DeepCOVIDNet-CXR: deep learning strategies for identifying COVID-19 on enhanced chest X-rays

Introduction

Coronavirus Disease 2019 (COVID-19) is a pandemic with high mortality and contagiousness rates in the global population. It is in the same respiratory disease family as severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS). Although SARS and MERS have affected specific locations in the world with mortality rates of 10 and 37 %, the prevalence of COVID-19 has already outperformed them in an exponential acceleration [1]. COVID-19 infects the lungs, reveals ruinous effects on the respiratory tract, and causes death related to gasping for each age group. Developments in medical image processing have enhanced computerized techniques for diagnosing and monitoring diseases.

Using novel techniques in diagnosis models procures quick and standardized abnormality detection on medical image processing [2].

Deep Learning (DL) has the ability to analyze the entire medical image through the advantages of feature learning. In contrast, conventional medical image processing depends on the local and morphological features to characterize the pathological region of interest. Convolutional neural networks (ConvNet), the most popular DL algorithms, extract low- and high-level features with sequential convolutions [3]. The depth of feature learning defines the complexity of the ConvNet architecture for feeding a deeper supervised classifier with responsible feature maps in DL. ConvNet comprises two main stages: feature learning and supervised learning. Whereas responsible feature maps are sequentially transferred among convolution blocks, fully connected (FC) layers support supervised learning with many units, regularization methods, and optimization algorithms. Besides the widespread use of ConvNet in various machine learning tasks, the primary justification of ConvNet is transfer learning that enables reusing the same knowledge to reduce the training time for related tasks. AlexNet [3], VGGNet [4], GoogLeNet [5], DarkNet19 [6], Inception modules, EfficientNet [7], and ResNet [8], 9] are among the most popular ConvNet architectures.

The easy accessibility of chest X-rays made them a widely used diagnostic tool for detecting COVID-19 [8], [10], [11], [12]. Even though most researchers shared the experimental results of chest X-ray analysis to catch COVID-19, the number of cases is small to produce a robust evaluation and modelling of DL architectures with high generalization capability.

Yoon et al. analyzed the correlation of chest X-ray and CT images in identifying COVID-19 with nine COVID-19-positive cases. They highlighted the efficiency of chest X-rays at least as well as CT [10]. Moreover, the popularity of feature learning on the ConvNet architectures brings stimulating approaches, including COVID-Net [12] and ResNet50 [8]. Nevertheless, the COVID-19-positive cases were insufficient in 31 and 25 for COVID-Net and ResNet50, respectively. Similarly, He et al. used a ConvNet to identify COVID-19-positive cases on 100 chest X-rays [9]. Das et al. fine-tuned InceptionNetV3 architecture to separate COVID-19 from healthy subjects. They experimented with various combinations of six COVID-19 chest X-ray datasets. In addition, they classified multi-cases (COVID-19, healthy, tuberculosis, and pneumonia cases) [13]. Asnaoui and Chawki also compared the deep architectures on chest X-rays and CT. They reported the performance of VGG16, VGG19, ResNet50, MobileNet, Inception-ResNetV2, and DenseNet201 on 231 COVID-19-positive cases [14]. Mukherjee et al. proposed a ConvNet architecture for various batch sizes, pooling, and convolutional filter sizes on 439 COVID-19-positive cases. They also outperformed popular ConvNet architectures, including, MobileNet, InceptionNetV3, and ResNet50 [15]. Minaee et al. utilized transfer learning and different adaptive optimization techniques and batch sizes on ResNet18, ResNet50, SqueezeNet, and DenseNet121 architectures. They experimented with two datasets on 184 COVID-19-positive cases and 5 K non-COVID cases [16]. Singh et al. proposed a multi-stage COVID-19 detection model using VGG16, ResNet50, DenseNet121, and DenseNet169 on 573 COVID-19-positive cases (473 train, 50 validation, and 50 test). They handled the lung segmentation module and generative adversarial networks to generate more practical chest X-rays in the preprocessing stage. They emphasized the effect of preprocessing for a robust evaluation, even for ConvNets [17]. Oh et al. also adapted a segmentation module and histogram equalization for the identification of multi-cases (healthy, pneumonia, and COVID-19) using ResNet-18 architecture on 180 COVID-19-positive cases (126 train, 18 validation, and 36 test) [18]. Ozturk et al. applied transfer learning using DarkNet19 architecture on 125 COVID-19-positive cases (5-fold cross-validation). They experimented with binary-case (COVID-19 and non-COVID-19) and multi-case classification (healthy, pneumonia, and COVID-19) and reported the achievements of DarkNet19 in learning realistic pathological patterns in chest X-rays [6]. Chowdhury et al. combined various COVID-19 chest X-ray databases for detecting COVID-19 using ConvNet architectures. They re-trained popular ConvNet architectures, including, MobileNetV2, SqueezeNet, ResNet18, ResNet101, and DenseNet201, on 423 COVID-19-positive cases (304 training, 34 validation, and 85 test). They reached the outperforming achievements against ChexNet using ResNet18 for binary-cases (COVID-19 and non-COVID-19) classification [19]. Rajpurkar et al. developed ChexNet, an initiator of reliable ConvNet analysis, using over 112 K chest X-rays with 14 pulmonary diseases [20]. Even though Conventional ChexNet did not include COVID-19-positive cases, the DenseNet architecture with 121 layers has become a baseline for much existing research on chest X-ray analysis. Haghanifar et al. re-trained the weights of ChexNet architecture, which used the DenseNet121 framework as a baseline feature learning, on 428 images (80 % train and 20 % test) for a variety of fully-connected layer parameters in the supervised stage [21]. Taspinar et al. used the feature learning stage of the VGG19 architecture to feed the conventional machine learning algorithms, including support vector machines and artificial neural networks on 800 COVID-19-positive cases. They reported the impact of feature learning on multi-case classification (healthy, pneumonia, and COVID-19) [22].

Among a considerable number of researchers on COVID-19 identification on chest X-rays in Table 1, most focused on proposing multi-stage ConvNet models (lung segmentation and pathology identification), compulsive generative DL algorithms for improving the quality, and global preprocessing stages for noise removals on small number of COVID-19-positive cases. The literature has limitations in being used as a guiding diagnostic with clinical relevance. The proposals must analyze large number of COVID-19 datasets for a high generalization capability. Moreover, the architectures need to apply adaptive preprocessing techniques with a detailed parameter presentation. Herein, the proposal aimed to identify multi-cases (COVID-19, healthy, and pneumonia) using adaptive techniques as preprocessing and designate the impact of localization parameters for chest X-rays with shallow ConvNet architectures. The main significance of the paper is analyzing COVID-19 on separate lung lobes for a specific impact definition on the performance of feature learning with popular ConvNet architectures, designating ConvNet-based adaptive preprocessing parameters for generating reliable lung anatomy of airways. In this paper, we generated new presentations of raw chest X-rays using various grid sizes and clip limits on Adaptive histogram equalization (AHE) [23], which enables global enhancing the chest X-ray without loss of contrast. We fed the popular shallow ConvNet architectures, including AlexNet, MobileNet, DarkNet19, and VGG16, using a reliable airway quality of the left lung, right lung, and both lungs from chest X-rays. In this manner, the generalization capacity of different chest X-ray sections for various AHE parameters could be designated, particularly for ConvNet architectures for future chest X-ray analysis. Feeding the ConvNet architectures with more significant and responsible pathology of lung airways to clinicians could implement more acceptable clinical relevance using the advantages of DL.

This paper presents a multi-case (COVID-19, pneumonia, and normal) identification evaluation for various AHE parameters with popular ConvNet architectures. Furthermore, we compared the effect of reliable AHE parameters for possible clinical relevance instead of raw chest X-rays. The key contributions of the proposed system are as follows:

1. Whereas most researchers analyzed a small number of COVID-19-positive cases, a disadvantage of ConvNet for unbalanced data, our proposal utilized one of the latest datasets with 3615 COVID-19-positive cases.

2. Rather than using raw chest X-rays, the proposal fed the ConvNet architectures using enhanced images with a more significant and responsible pathology visualization for a clinical relevance

3. The paper performed a complete comparison to state the impact of various AHE parameters with each ConvNet for creating a baseline for future chest X-ray strategies

4. Multi-case identification performances were evaluated separately for chest X-rays using fine-tuning on the best performance to reduce the training time of ConvNets.

5. Transfer learning on the weights of ConvNets trained small-scale datasets for large-scale COVID-19 dataset

The remaining paper gives detailed information about the chest X-ray database, AHE, ConvNet architectures, feature learning, and fine-tuning in Section 2. The structure of the proposed model, iteration parameters, AHE variations, experimental setup, and achievements are shared in Section 3. The efficiency of ConvNets in identifying COVID-19, superiority, and limited aspects of the proposed models are handled in the last section.

Materials and methods

Adaptive histogram equalization on chest X-rays

AHE is a prosperous image contrast enhancement technique that provides more precise presentations for various image types. Capabilities of contrast enhancement and noise reduction for medical images are the most frequent preference reasons for using AHE [23]. It enables reproducible enhancing presentation with many contrasts in chest X-rays using interactive contrast density windows and cumulative distribution of pixels [24]. It has a common use in medical images to improve the visibility of the patterns and support the diagnosis due to ensuring detailed presentations for the region surrounding the black pixels [25]. The complexity of AHE is O(nˆ2 (m + i)) for an n-by-n medical image with an intensity level of i and m-by-m contextual region size [23], 25].

AHE has two parameters the grid size, which divides the chest X-rays into contextual tiles, and the clip limit, which is a thresholding value for contrast limiting to prevent over-saturation in homogeneous subsections [25]. Herein, we experimented with a brute force algorithm using AHE parameters at the ranges of [0.1–1] by increasing the size of 0.1 for clip limit and 1/2, 1/4, 1/7, 1/8 [23], 1/16 [25], 1/28, 1/56 of input chest X-rays for grid sizes on each pre-trained architecture [32]. We shared the classification performances for only AHE parameters with the highest achievements for each pre-trained architecture in ascending order according to the highest accuracy.

Chest X-rays have a large amount of information belonging to many viscera, including the heart, diaphragm, lungs, hilar arteries, clavicle, and rib cage. As shown in Figure 1, the histogram plot for raw chest X-ray and AHE application present different densities. In comparison, local image enhancement techniques improve the presentation by equally analyzing pixel intensity over the possible grayscale; the variety of density in viscera necessitates regional adaptive pixel intensity proportional [25]. Hence, using a whole chest X-ray with AHE implements a density-based enhancement referring to global enhancement techniques, which are incompetent for medical images by maintaining in restricted size of sub-regions [24]. Raw chest X-ray, depicted in Figure 1a, is composed of similar grayscale density due to the similar X-ray permeability of the tissues and noise arising out of device specifications (see Figure 1b). On the other hand, chest X-ray with AHE enables the pixel density to reach a more uniform distribution with local enhancement methods, as is seen in Figure 1d. This density-based enhancement provides a sharper acquisition of pathological regions and airways, minor alterations (see Figure 1c), as well as the detection of distinctive features for machine learning.

Figure 1:

A random chest X-ray with COVID-19 (a), histogram plot for image (a) (b), AHE with clip limit of 0.4 and grid size of 112px (1/2) on image (a) (c), histogram plot for image (c) (d).

Results

Computer-aided identification of rare pathology and incipient abnormalities is of great importance for COVID-19. Furthermore, it is possible to reveal ConvNet models using the density of pathologies, obstructions, and fluid accumulation resulting from bacterial- and viral pulmonary diseases. Therefore, minimizing the adherence to clinical specialists for the robust and practical identification of pandemic diseases is the prime necessity to increase the prevalence. The structure of the proposed strategies is indicated in Figure 2. It depicts the sequence of the proposed three strategies for transfer learning. Strategy 1 is initially attaining the pre-trained weights for ConvNet architectures by training with the small-scale dataset to pave for other strategies with related data training. Strategy 2 is the test of ConvNet architectures on Strategy 1 using the large-scale dataset. Strategy 3 performs transfer learning on pre-trained weights from Strategy 1 on the large-scale dataset. Consequently, the proposed model visualizes the most responsible regions for a better understanding with clinical relevance for multi-cases on chest X-rays.

Figure 2:

The structure of the proposed model. Strategy 1 (transfer learning on small-scale dataset). Strategy 2 (testing strategy 1 using the large-scale dataset). Strategy 3 (transfer learning on the large-scale dataset).

We evaluated the impact of AHE with popular ConvNet architectures to identify chest X-rays with COVID-19, pneumonia, and healthy. We applied AHE for various parameters as a preprocessing stage to designate the most responsible vision on chest X-rays depending on each ConvNet architecture. Although ConvNet architectures have high performances for identifying COVID-19 using raw chest X-rays, we focused on ensuring intact lung anatomy by padding and enhancing airway visibility to support the clinical relevance and validity of the proposals.

We fined-tuned the pre-trained ImageNet weights on both ConvNet architectures using Keras implementation. We trained each ConvNet architecture (Strategy 1) using the cross-entropy loss function, a learning rate of 0.00001, a patience period of 5 epochs, and a batch size of 16 images for 100 epochs. We selected the adaptive moment estimation (Adam) optimizer to adaptively decrease the learning rate when learning stagnates in a period of epochs during training [31]. The cross-entropy loss function, which approaches minimizing the probabilistic differences between predicted and actual labels, is adapted to the fine-tuning procedures. Finally, we utilized weight decay, validation set (20 % of the training set), and L1 regularization in training to avoid overfitting.

The statistical test characteristics were calculated using a weighted average to evaluate the class-based efficiency of the experimented AHE parameters with each Strategy on ConvNet architectures [2]. The results were averaged to produce the performance of ConvNet architectures. Furthermore, we visualized the most responsible sections on chest X-rays for each ConvNet using gradient-based localization techniques on feature maps.

The highest classification performances for AHE on complete chest X-ray analysis were achieved using DarkNet19 architecture with rates of 98.26 , 98.33, 98.26, and 98.25 % for accuracy, precision, recall, and f1 score, respectively. The most responsible AHE parameters are 1/28 and 0.6 for AHE grid size and clip limit, respectively. Table 2 presents classification performances and AHE parameters for ConvNets. Applying AHE in various parameters has outperformed models without AHE for both ConvNet architectures. The highest performance improvement in terms of overall accuracy is reported using 1/8 of grid size and 0.1 of clip limit on AHE using MobileNet architecture for Strategy 1 by 84.62–95.46 %.

We tested the trained ConvNet architectures in Strategy 1 using a discrete large-scale testing fold to evaluate the generalization performances. The highest large-scale testing identification performances for AHE were achieved also using DarkNet19 architecture with rates of 71.33 , 76.81, 72.54, and 74.55 % for accuracy, precision, recall, and f1 score, respectively. The most responsible AHE parameters are 1/16 and 0.7 for AHE grid size and clip limit on complete chest X-rays, respectively. Table 3 presents the testing performances with the large-scale dataset for the best ConvNet architectures. Whereas DarkNet19 has the highest generalization capability with an accuracy rate of 71.33 %, the remaining ConvNet architectures predicted unsuccessfully for training with small-scale datasets. Especially, MobileNet is the weakest architecture for Strategy 2. This case indicates the disadvantages and the lack of clinical validity for related research.

Transfer learning was applied to the weights of the best five ConvNets on Strategy 1 using a large-scale database. The highest classification performances for Strategy 3 were achieved also using VGG16 architecture with rates of 95.04 , 95.11, 94.99, and 95.05 % for accuracy, precision, recall, and f1 score, respectively. The most responsible AHE parameters for VGG16 are 1/8 and 0.1 for AHE grid size and clip limit, respectively. Table 4 presents classification performances and the best AHE parameters for four ConvNets. Applying transfer learning on the weights of the small-scale COVID-19 dataset reached high classification performances for DarkNet19, VGG16, and AlexNet architectures. MobileNet reached a low generalization capability considering the classification performance on the small-scale dataset. Whereas the patience parameter for early stopping was set to 5 epochs, the training of Strategy 3 was halted at average epoch sizes of 18, 12, and 16 for AlexNet, DarkNet, and VGG16, respectively. Besides achieving the lowest classification performances for transfer learning on Strategy 3, MobileNet passed over the patience parameter by finalizing the training at 100 epochs.

It would be easy to understand the impact and responsibility of ConvNet architectures on different Strategies using the same AHE parameters. Therefore, we presented Table 5 that includes a cross check for the achievements with the same AHE parameters of the best pre-trained architectures on each strategy. Although the DarkNet19 architecture does not give the best results for each strategy on the same AHE parameters, it is clearly seen that it does not cause very large fluctuations on performance rates. The VGG16 architecture, on the other hand, showed inconstant rates up to almost 2x of overall accuracy. This situation shows that the VGG16 is quite ineffective compared to DarkNet19 in COVID-19 identification without fine-tuning on new data.

DarkNet19 reached the highest overall accuracy rates for Strategies 1 and 2. The best identification rates are achieved with rates of 100 and 99.10 % for pneumonia and normal chest X-ray using DarkNet19 on Strategy 1, respectively. Although Strategy 2 has low performance in terms of overall accuracy, DarkNet19 is also the best architecture and reaches a pneumonia identification accuracy of 96.00 %. VGG16 has the highest capacity for transfer learning on Strategy 3. Figure 3 depicts the confusion matrices for the best ConvNet architectures. The confusion matrix for DarkNet19 with the small-scale dataset on Strategy 2 has been plotted for 1,428 chest X-rays with homogeneously distributed multi-classes on the left of Figure 3. The confusion matrix of the highly responsible VGG16 architecture for Strategy 3 on the large-scale dataset has been plotted for 3,128 chest X-rays on the right of Figure 3. The receiver operating characteristic (ROC) and learning curves (loss-epoch) have been implemented in Figure 4 for a better understanding of the training performance of the models. We included the ROC and learning curves of ConvNet architectures with the best achievements for Strategies 1 and 3. VGG16 in Strategy 3 has a better and faster-converging capability than DarkNet19 in Strategy 1 in the training of the COVID-19 identification model on enhanced chest X-rays. Areas under ROC curves, the aggregate measure of model performance, are 94.98 and 95.87 % for Strategy 3 and Strategy 1, respectively. The learning curve measures the correlation between training losses and the epochs for ConvNet architectures. Learning curves indicates that training DarkNet19 in Strategy 1 and VGG16 in Strategy 3 (without early stopping) have similar convergence by epoch. Strategy 1 can reach a lower loss by epoch on training. In contrast, Strategy 1 is prone to stop training with sufficiently minimum loss at early epochs for a small patience value but reaches a slightly higher loss than Strategy 3.

Figure 3:

The confusion matrices for the best ConvNet architectures, DarkNet19 on strategies 1 and 2 (Left); VGG16 on strategy 3 (Right).

Figure 4:

The receiver operating characteristic (ROC) and learning (loss-epoch) curves for the ConvNet architectures with the highest achievements in strategies 1 and 3.

Discussion

As the amount of COVID-19 cases continues to increase with rapid acceleration, developing robust diagnostics with clinical validity is crucial to edge over the progression of the pandemic. In spite of the fact that the RT-PCR test is the most preferred diagnostic tool, its precision is also limited [28]. Although it gives definite results in identifying COVID-19-positive cases, definitive non-COVID-19 cases need to be evaluated at the various clinical stages. Hence, it is substantial to develop alternative diagnostic and visualization techniques to procure the analytical validity and testing for agreement. The majority of COVID-19 identification research on chest X-ray analysis has focused on obtaining a high generalization capability using small-scale datasets due to the scantiness in quantity. The chest X-ray analysis commonly focused on feeding ConvNet architectures with small-scale datasets due to the lack of chest X-rays with COVID-19. Table presents a complete comparison with related works on identifying multi-cases (COVID-19, pneumonia, and normal) in terms of databases, ConvNet architectures, preprocessing, number of subjects in the analysis, and classification performances.

Many researchers analyzed chest X-rays for binary case classification (COVID-19 vs. non-COVID-19 and COVID-19 vs. normal). Mukherjee et al. proposed a tailored ConvNet architecture for distinguishing two cases (COVID-19 – non-COVID-19). Their proposal outperformed popular architectures with rates of 99.69 , 100, 99.38, and 99.69 % for accuracy, recall, precision, and f1 score, respectively [15]. Minaee et al. re-trained ResNet18, ResNet50, SqueezeNet, and DenseNet121 architectures with adaptive optimization techniques by transfer learning on ImageNet weights. They reached identification performance rates of 92.28 , 92.10, 99.93, and 95.85 % for accuracy, recall, precision, and f1 score using a small-scale dataset for identifying COVID-19 and non-COVID-19 [16]. He et al. used a ConvNet to separate COVID-19 and non-COVID-19 cases. In training, they experimented with various threshold values with a stochastic gradient descent algorithm. They achieved high classification performance rates of 87.98 , 90.00, and 87.84 % for accuracy, recall, and specificity, respectively [9]. Haghanifar et al. also tried to train DenseNet121, which is the feature learning backbone of ChexNet architecture, for binary classification (COVID-19, normal). They distinguished COVID-19 cases from normal chest X-rays with performance rates of 98.68 , 99.34, 99.17, and 99.25 % for accuracy, precision, recall, and f1 score, respectively. However, they reported the inefficacy of models for the localization of reasonable lung region feature maps [21]. Ozturk et al. also proposed a binary case classification (COVID-19 vs. normal) model using DarkNet19 architecture. Although applying none of preprocessing except resizing, they iterated on various numbers of filters for the feature learning stage of ConvNet architecture. They separated COVID-19 cases from healthy chest X-rays with identification performance rates of 98.08 , 98.03, 95.13, 95.30, and 96.51 % for accuracy, precision, recall, specificity, and f1 score, respectively [6]. Ismael and Sengur applied local texture descriptors to classify chest X-rays (180 COVID-19 and 200 normal) using transfer learning on popular ConvNet architectures, including, ResNet18, ResNet50, ResNet101, VGG16, and VGG19. They also adapted multiple conventional machine learning algorithms to evaluate the impact of feature learning stages. They reached the highest identification achievements using ResNet50 with linear support vector machines function with rates of 94.74 , 91.00, 98.89, and 94.79 % for accuracy, recall, sensitivity, and f1 score, respectively [29]. Nayak et al. also performed a binary classification on chest X-rays (203 COVID-19 and 203 normal) to compare the impact of ConvNet architectures, including VGG16, InceptionV3, ResNet34, MobileNetV2, AlexNet, GoogleNet, ResNet50, and SqueezeNet. They augmented the dataset using scaling, flipping, rotation, and adding Gaussian noise. They produced the best achievements using ResNet34 with rates of 98.33 , 96.77, 96.67, and 98.36 % for accuracy, precision, specificity, and f1 score, respectively [5]. Taspinar et al. compared various machine learning algorithms by feeding ConvNet-based feature maps and highlighted the efficiency of feature learning on VGG19 with the stacking classifier. They reached the best achievements using the stacking classifier with rates of 96.86 , 96.88, and 96.89 % for accuracy, recall, and f1 score, respectively [22].

The researchers analyzed the chest X-rays for multi-case classification presented in Table 1 with a detailed comparison in terms of preprocessing, ConvNet architectures, additional optimization algorithms, and case-specific classification performances. Although the demonstrated achievements are relatively prosperous, the main deficiency of the related papers is that the analyses included a tiny amount of COVID-19 cases in common. Furthermore, using unbalanced datasets fails COVID-19 identification and learning COVID-19 characteristics on chest X-rays. Thanks to the recent updates on COVID-19 databases, our proposal on Strategy 3 is based on one of the foremost and most comprehensive chest X-ray analyses regarding the amount of COVID-19 positive cases. Hence, transfer learning on the weights of ConvNet architecture trained with the related small-scale dataset reached a high distinguishing capability for COVID-19 cases, just as providing precipitated optimization on detailed ConvNet architectures.

Even if the most relevant papers in preprocessing with AHE have been conducted on small-scale datasets within COVID-19 cases, they highlighted the effect of using enhanced lung pathology to support the clinical validity of results on the various ConvNet architectures [14], 17], 18], 21]. Singh et al. utilized generative adversarial networks on AHE representations to generate more visual lung patterns as a preprocessing. They produced the feature learning stage of ConvNet architectures, including VGG19, VGG16, DenseNet121, DenseNet169, and ResNet50. They reported the efficiency of feature maps in VGG19 with pruned Naive classifier with rates of 98.67 and 98 % for overall accuracy and kappa score, respectively [17]. However, using a deep generative model as segmentation and chest X-ray enhancement technique unveils additional workload in COVID-19 identification. Moreover, deep generative models have the possibility of constituting non-existing lung pathology. Hence, the clinical relevance of their proposal is still limited.

On the other hand, Oh et al. used a two-step ConvNet architecture to segment the lungs and classify multi-cases on chest X-rays. They excluded the non-lung sections from the analysis using DenseNet103 architecture and afterwards fed the ROIs of the lung into the ResNet18 architecture. They classified multi-cases with performance rates of 88.90 , 83.40, 85.90, and 84.40 % for accuracy, precision, recall, and f1 score, respectively [18]. Similarly, Haghanifar et al. utilized U-Net architecture for lung segmentation and afterwards applied AHE for default parameters to the segmented lung areas. They reached an identification accuracy rate of 87.21 % by feeding the segmented lungs into DenseNet121 architecture [21]. However, using deep ConvNet models as lung segmentation is a disadvantage for wasting training time. Moreover, the achievements are in the moderate performance ranges against our achievements for a majority of strategies. El Asnaoui and Chawki used AHE to generate more responsible chest X-rays in an approach similar to ours by image enhancement. They performed transfer learning on the VGG16, VGG19, DenseNet201, Inception ResNetV2, InceptionV3, Resnet50, and MobileNetV2 architectures. They reached multi-case identification performance using Inception ResNetV2 and L2 regularization with the rates of 92.18 , 92.19, 92.32, and 92.08 % for accuracy, recall, precision, and f1 score, respectively [14]. However, the paper doesn’t explain the most responsible AHE parameters and experimented ranges for COVID-19 identification. In particular, default AHE parameters for grid size and clip limit factor have limitations for representing true clinical evaluations considering the experiments on various ConvNet architectures. Whereas the state-of-the-art has disadvantages of clinical relevance and clinical validity due to a limited range of parameters in image enhancement and ConvNet architectures with a high number of layers, our proposal experimented with a wide range of AHE parameters with shallow ConvNet architectures. Whereas AHE with 1/56 and 1/16 of grid size presented the most responsible feature maps for DarkNet19, 1/8 of grid size for AHE has the most characteristic features for VGG16 architecture.

The highest classification performances for the large-scale COVID-19 dataset within Strategy 3 were achieved using VGG16 architecture with rates of 95.04 , 95.11, 94.99, and 95.05 % for accuracy, precision, recall, and f1 score, respectively. The most responsible AHE parameters are 1/8 and 0.1 for AHE grid size and clip limit, respectively (see Table 6).

Gradient-weighted class activation mapping (Grad-CAM) is a visualisation technique on CNN architectures to identify the most responsible feature activations with a heat map [30]. We customized Grad-CAM by setting a threshold rate for learned highly responsible features. Whereas Grad-CAM generates a heat map to visualize the level of responsibility for activation maps, we excluded the learned activation maps less than the threshold ratio and a single colour presentation to provide a clearer view of the highest responsibility for COVID-19. Figure 5 depicts the customized approach on Grad-CAM for random chest X-rays with COVID-19. The yellow sections are the highly responsible sections with a threshold rate of 0.75 for each Strategy. Most dispersion on chest X-rays mainly occurs on the lungs, even using no lung segmentation in the proposal. Results indicate the highly responsible feature activation map for COVID-19 for Strategy 1, Strategy 2, and Strategy 3. Strategy 1 highlighted the most responsible feature activation maps for a local region and non-lung (heart) region, as is seen in Figure 5a. Strategy 2 has similar achievements with a local pathological region and non-lung (armpit) section in Figure 5b. The highly responsible feature maps for identifying COVID-19 on Strategy 3 are mostly located in the lung regions for extended pathologies on sample chest X-rays with clinical relevance in Figure 5c.

Figure 5:

The highly responsible sections with Grad-CAM for strategy 1 (a), strategy 2 (b), and strategy 3 (c).

The necessity of clinical validation on predicted pathology constitutes the study’s weakness. Hence, it is more appropriate to use chest X-rays as a guiding evaluation or supportive visualization technique rather than an early diagnosis tool for COVID-19 due to the variety of medical devices with different specifications and patient-specific cases. Computerized techniques need a more detailed COVID-19 database with annotations and localization of pathological regions caused by COVID-19 for the clinical applicability of the proposals to overcome this limitation. An experienced clinician evaluated the Grad-CAM of the proposal for responsible lung regions with COVID-19. According to the clinician’s evaluation, although the proposal within Strategy 3 is relatively successful at identifying severe COVID-19 pathology, it still has limitations in locating mild-level abnormalities. Nevertheless, highly responsible feature maps can be located in the non-lung and cardiac regions. This assessment clearly shows that the ConvNet architectures need to be supported with larger numbers of COVID-19 cases before gaining clinical use. Generally, the most responsible case activation maps are on lung regions for the best proposals on each Strategy.

Shallow ConvNet architectures have the ability to perform accurate predictions with the advantages of chest X-ray enhancements in preprocessing and straight parameter optimization techniques in training. The lightweight concepts of shallow ConvNets gain an edge in the pre-definition of robust models using transfer learning. That is why we used the weights of Strategy 1 for Strategy 3 on a large-scale relevant dataset. It enables re-training the novel COVID-19 cases using shared weights. In accordance with the highest classification performances for four ConvNet architectures, DarkNet19 achieved the highest multi-case identification performances for Strategies 1 and 2. VGG16 is the highest-performance model in Strategy 3. Whereas training time is a curse for deep ConvNets, Strategy 3 reduced training time with transfer learning on the model trained using a relevant small-scale dataset.

Conclusions

The main superiorities of this study are feeding the ConvNet architectures with enhanced representations of chest X-rays and COVID-19 pathology using various AHE parameters on the largest balanced COVID-19 dataset. The AHE enhancement on chest X-rays took advantage of accurate multi-case classification performances with local preprocessing, which exhibits reproducible presentation for lung patterns. The achievements prove that.

1. AHE with ConvNets is a pioneering approach to designate multi-case models for distinguishing similar pathological metaplasia such as pneumonia and COVID-19.

2. It is a pioneering transfer learning application that asserts a base and open-upgrade architecture for future COVID-19 cases for better generalization.

3. The most responsible AHE parameters for future strategies on chest X-rays for identification of COVID-19 was designated

4. The impact of the left lung, right lung, and complete chest X-rays for multi-case classification individually was revealed

5. Transfer learning on the weights of ConvNets, which was trained using balanced small-scale COVID-19 datasets, was evaluated using extended balanced COVID-19 dataset for better generalization.

Therefore, DL-supported computer-aided techniques can significantly identify and visualize pathologies for diagnosing pneumonia and COVID-19 cases.