Empirical analysis on retinal segmentation using PSO-based thresholding in diabetic retinopathy grading

Dataset collection and description

The current work utilized the Indian Diabetic Retinopathy Image Dataset (IDRiD) [20]. The IDRiD Dataset is a publicly accessible benchmark dataset. The fundus images were captured by a retinal specialist using a Kowa VX-10 alpha digital fundus camera and all are centered near to the macula. The resolution of an image is 4,288 × 2,848 pixels. This dataset incorporates pixel-level annotated data and image-level grading data with binary mask ground-truth images to evaluate the performance of individual lesion segmentation techniques and grading classification.

Image pre-processing

The input data and the architecture of the networks play an important role in building an effective deep-leaning-based classification system. Pre-processing is an important step in improving the quality of the input image [21]. Improvement in preprocessing can help in eliminating artifacts [6]. This study has utilized normalization, augmentation, and resizing for preprocessing.

The image pixel values were originally in the range 0–255. These can lead to exploding or vanishing problems during the training process. Normalizing images to the range 0–1 helps to improve the image data, training stability, and performance of the model [22]. There are many normalization approaches for scaling the attribute values [23]. For this study, the rescale parameter from Keras ImageDataGenerator, a specific case of min-max normalization has used.

Image augmentation increases the size and diversity of the dataset artificially to improve the model’s generalizability. The horizontal and vertical flipping is applied on the training image to augment the image using the ImageDataGenerator. The ImageDataGenerator avoids overfitting because it will not add the transformed images to the range of original images [24]. The image resize is used in preprocessing to transform different dimensional images to a consistent scale [8]. The image resize helps to minimize the running time and required memory of the model to load and train the images [25]. In this work, the fundus image is resized to downscale the image size by passing a fixed dimension value.

Proposed PSO-based retinal segmentation

Prior studies have made significant contributions to DR classification by enhancing image quality and facilitating feature analysis using various approaches including CLAHE, Color extraction, Feature extraction, Blood vessel segmentation, and Optic Disc segmentation. The artifacts may potentially cause the model to concentrate on inappropriate information. It is illustrated in the following Figure 2a using Explainable AI SHAP values tools. SHAP helps to understand how each feature impacts the model’s prediction. SHAP gives an important value on each feature for a prediction that has been identified [26]. As emphasized in the figure, the model focuses on undesired black background information, impacting its accuracy and leading to misdiagnoses. This study suggests a retinal segmentation to segment the retinal area from the background area. This approach reduces background information to further enhance the model’s accuracy and reduces the size of the image for more efficient processing. For foreground segmentation, thresholding is one of the most popular approaches for converting an image into a binary image. The optimal threshold value setting is the primary challenge in thresholding approaches, which is determined by the user. If the user is unable to choose an ideal threshold setting, it may cause to miss certain valuable areas of an image [27].

Figure 2:

Result of SHAP interpretation on (a) non-segmented images and (b) segmented images using PSO-E.

Thresholding techniques include local and global thresholding algorithms. Global thresholding uses a single threshold to divide the image into foreground and background pixels and this study focuses on global threshold approaches. Otsu’s method is a popular state-of-the-art global thresholding method, though it may struggle with certain images since it expects that the foreground and background have similar Gaussian distributions, which is a highly complex order that real-world images are unable to satisfy [28]. Otsu may not perform well with images that have more than two peaks in the histograms.

In recent studies, bio-inspired optimization algorithms have been utilized to tackle the problems of traditional optimization algorithms [29]. The bio-inspired PSO algorithm has the potential to produce good results while addressing the limitations of Otsu’s technique, such as establishing an ideal threshold for accurate image segmentation and the images having a large intraclass variance difference and uneven illumination respectively. In 1990, Kennedy and Eberhart introduced the PSO [30]. PSO is a population and random-based approach inspired by the collective behavior of bird flocks or fish schools. The ideal strategy for the birds is to follow the one closest to the food. PSO treats each potential solution as a particle within the search space, much like a bird. Each particle has a fitness value, which is defined by a fitness function that requires optimization, as well as a velocity that regulates its movement. Particles move through the problem space by seeking the particle in the best-known position [31].

For segmenting the retinal area from the unwanted background information, the PSO algorithm is used in this study due to its ease of implementation, low parameter needs, fast convergence, and scalability [32]. The purpose of using the PSO algorithm is to determine the threshold value automatically for the binary conversion of an image to segment the retinal region from images. In order to get the best threshold value, the entropy approach is used as an objective function with the PSO. The PSO with Entropy (PSO-E)-based optimization algorithm is implemented for auto threshold value determination and the detected threshold value is applied on the image to segment the retinal region. As shown in Figure 2b, the majority of background pixels are removed using the PSO-E algorithm, and the SHAP tool is applied to interpret the result. The SHAP shows the model is focused on the appropriate information after retinal segmentation. The general steps for the PSO-E algorithm for thresholding are as follows:

• Step 1: Input the grayscale image, set the number of particles to 0, maximum iterations to 20, PSO parameters: inertia weight (w) to 0.5, cognitive and social components (c1, c2) are to1.5.

• Step 2: Define the fitness function based on image entropy. The entropy of the binary image is used as the fitness to obtain the threshold value.

where p _j is the probability of foreground pixel value in the binary image.

• Step 3: Initialize the positions of particles with a predefined threshold range [10], 20], personal best positions are to be equal to the initial positions, global best position and fitness are to 0, all particle velocities are set to 0, and personal best fitness is to 0 for all particles.

• Step 4: Iteration: begin;

  • Calculate the fitness of each particle.

  • Adjust the velocity and position of each particle.

where

Update personal and global best positions based on fitness values.

where

Check the convergence criteria. Stop if the global fitness reaches the desired entropy of 1.0.

• Step 5: Get the output of the binarized image using the global best threshold value.

Multi-class classification using ResNet50

The non-segmented and PSO-based retinal-segmented images are inputted separately into the ResNet50 transfer learning model [33] to evaluate the segmentation and grading performance. The Residual Neural Network is a Convolutional Neural Network (CNN) that integrates residual connections to preserve network connectivity and prevent gradient vanishing by adding the normalization layers. These connections work as shortcuts, allowing information to jump to certain levels and proceed directly to the output. This technique enables the network to learn residual functions and make modest parameter changes, resulting in faster convergence and better performance. ResNet enhances overall performance by prioritizing residual functions over complex input-output mappings.

The 50-layer ResNet (Residual Neural Network) architecture extracts a feature from the input image, and then the Global Average Pooling (GAP) [34] layer reduces the high dimensional feature maps from ResNet50 into lower-dimension vectors, and finally fully connected (dense) layers perform classification for grading the fundus images. This procedure aids in customizing the underlying model’s higher-level features to the particular task at hand [35]. ResNet50 residual networks keep consistent computational complexity by doubling filters when halving feature map size and using the same filter count based on the output feature map size with bottleneck design to reduce the number of parameters and matrix multiplications. Grading results are analyzed to provide insights into the significance of retinal segmentation.

Experimental dataset distribution

The experimental investigation of the proposed PSO-based retinal segmentation was executed on the images acquired from the IDRiD dataset. In total 516 images, for thresholding analysis, 516 images were employed, and for assessing the grading accuracy 330 images were allocated for training, 83 images for validation, and 103 images for testing. All computation, training, testing, and validations were performed in Google Colab environments using GPU runtime, equipped with an Intel i5 -1035G1 CPU @ 1.00 GHz processor, and 8 GB of physical memory hardware.

Performance metrics

To measure the performance of the retinal segmentation, kappa score, mean accuracy, multi-class confusion matrix, and SHAP XAI, are employed. In a multi-class confusion matrix rows reflect actual classes, and columns indicate expected classes. The kappa score is determined from the multi-class confusion matrix. It is more reliable than basic accuracy since it shows how much better the segmentation results are than probability. The segmentation model’s mean accuracy provides an overview of how well it works across all classes, rather than concentrating on just one. The multi-class confusion matrix makes it possible to analyse errors for each class in greater detail and shows how frequently the model confuses one class with another. SHAP can explain which features are most crucial for predicting particular outcomes. The model’s accuracy, efficiency, and reliability in retinal segmentation are evaluated using these metrics.

Experimental analysis of thresholding

Figure 3 presents the visualization results obtained from the state-of-the-art Otsu thresholding, histogram-based sigma thresholding, Entropy thresholding, and proposed PSO-E thresholding techniques.

Figure 3:

Comparative performance of the thresholding with various methods.

The various thresholding algorithms such as Otsu’s algorithm, the histogram-based sigma approach, the entropy-based thresholding algorithm, and the PSO optimization are effective for automatic thresholding methods. However, Otsu, histogram-based sigma, and entropy methods lead to incorrect segmentation due to large intraclass variance differences and uneven illumination in images. In contrast, the proposed PSO algorithm, which uses entropy-based optimization, works well with both unimodal and multimodal histogram distributions. It also eliminates manual threshold setting, resulting in a more robust and efficient solution.

Experimental analysis of grading

The effectiveness of retinal segmentation is assessed using ResNet50 architecture. The various threshold techniques are applied to segment the retinal area. Once the retinal area was segmented, the segmented images were trained and validated using the classification model. The performance of different retinal segmentation methods was evaluated using training accuracy, validation accuracy and the mean kappa score, as shown in Table 2.

Based on the retinal segmentation table results, Otsu’s method, while effective in training, suffered from overfitting, the histogram-based approach had lower overall performance but a better balance between training and validation results, and the Entropy-based approach performed worse than Otsu or sigma-based approach, however, PSO-E-based thresholding indicates the highest agreement and showing good generalization on unseen data.

Ablation studies

To separately evaluate the impact of retinal segmentation, the deep learning model is trained using non-segmented images and PSO-based retinal-segmented images. Then the result is obtained separately using the ResNet-50 baseline model. The confusion matrix is a table that compares the grading performance of the two approaches, shown in Figure 4.

Figure 4:

Result of multi-class confusion matrix on (top) non-segmented images (bottom) segmented images.

The figure shows retinal segmentation can significantly improve the accuracy of grading in deep learning. The findings suggest that retinal segmentation can help a deep learning model focus on the most important regions of an image, decreasing complexity and distractions. This can help the model learn crucial features and patterns for the task at hand. Table 3 (top) shows the class-wise performance.

From the above table, training with segmentation images yields superior results than non-segmentation images across all classes. Non-segmented images perform slightly better in validation in certain Class 0 and Class 3 cases. However, Segmentation approach provides a significant speed improvement for specific classes, such as Class 1 and Class 2. It is evident that performance is generally improved by retinal-segmented images. For training, validation, and overall performance of a task, the mean kappa score, a measure of agreement between expected and actual outcomes is assessed.

The model with the proposed retinal segmentation image accuracy is compared with the non-segmented image approach. The model trained using the proposed segmentation images gave an accuracy of 93.9 % using Kappa’s score. The model with the non-segmented images obtained an accuracy of 88.6 %. Table 3 (bottom) indicates that the retinal-segmented approach performs much better than the non-segmented approach.

Compared to non-segmented images, the model performs much better when retinal-segmented images are used, especially when testing and validation stages are used to reflect real-world events. In order to improve generalization, the segmentation procedure aids the model in concentrating on important aspects inside the images. Figure 5 illustrates how test predictions increased for all classes when employing retinal-segmented images; nevertheless, the non-segmented approach was unable to meet this accuracy, especially for Grade 1, which is indicated by a red bounding box.

Figure 5:

Prediction result on the test dataset.

Finally, the experimental results show that the majority of background pixels affect the DR grading. Therefore, retinal segmentation using a PSO-based thresholding approach is proposed to segment the foreground retinal region from the background. Hence, the retinal segmented images not only reduce the complexity but also obtain a modest improvement in the performance of the model and grading accuracy.