Automated mitosis detection in stained histopathological images using Faster R-CNN and stain techniques

2.1 Mitosis detection techniques

The mitosis detection methodologies can be classified into three categories [1]. The first one, called handcrafted features methods extracts manually features from data and trains Machine Learning algorithms to recognize or classify patterns. Initially, the input image is processed to detect candidate cells or nuclei. Then, there is a feature extraction stage. Finally, there is a extraction and classification stage for the candidate cells. This stage generally uses standard algorithms such as Support Vector Machines (SVM), Random Forest (RF), Linear Discriminant Analysis (LDA) or Multi Layer Perceptron (MLP). These algorithms classify the candidate cells into mitotic cell or non-mitotic cell, thus completing the mitosis detection process [1].

The second method is based on Deep Learning. Nowadays, Deep Learning methods have become extensively used in medical systems for image-processing tasks, including mitosis detection, cell nucleus segmentation and tissue classification. Detection methods based on DL exploits the abilities of neural network self-learning to automatically extract features and train features. To tackle the challenges of mitosis detection, methods that utilize Deep Convolutional Neural Network (DCNN) are commonly employed due to their effectiveness in achieving accurate results [1], 8]. Approaches based on Convolutional Neural Network (CNN) are also particularly notable in this regard. CNN-based methods are widely used in medical image analysis as they improve computer vision tasks, including image classification [9], object detection [7], 10], semantic segmentation [11] and instance segmentation [12]. In our research, we employ Deep Learning models, specifically CNN-based methods, to tackle challenges in mitosis detection and improve medical image analysis, with a focus on object detection.

The third one is a combination of both methods. Methodologies for mitosis detection utilize either layered handcrafted features or features extracted through CNNs. Nevertheless, using only handcrafted features results in low detection accuracy, while CNN-based techniques are limited by their computational complexity [13]. The integration of both methodologies can potentially enhance the overall performance of the mitosis detection system.

2.2 Object detection models

Object detection is a task within computer vision that involves identifying objects in an image by determining both their category and location. This process not only classifies the objects but also predicts their position using bounding boxes [14]. As a result, the aim of object detection is twofold: locating objects in the image (object localization) and assigning them to the correct category (object classification).

Recently, there has been growing interest in object detection tasks, particularly within the field of histopathology [15] and mitosis detection [8], 16]. The models are generally divided into two main categories: one-stage and two-stage approaches. In general terms, two-stage models typically achieve superior accuracy but demand higher computational resources compared to one-stage models. The accuracy-computational trade-off is heavily influenced by the choice of the backbone network and the hyperparameter configuration [17].

DCNN are the backbone network for object detection models. To improve feature representation performance continue, network architectures become increasingly complex, with deeper layers and more parameters. Consequently, networks known as Complex Backbone Network (CBN) has been proposed. However, in environments with limited computing power and storage, Lightweight Backbone Network (LBN) structures are used to simplify the network structure without compromising accuracy [6].

In addition, to enhance accuracy, the depth of Complex Backbone Networks has been increased. Some examples are VGGNet [18], GoogLeNet [19] and ResNet [9]. All of this DCNN came up after the success of AlexNet [20], known as the first CNN.

One-stage models perform region proposal and classification in a single stage. A single feed-forward fully CNN directly outputs both the bounding boxes and the object classification. RetinaNet and You Only Look Once (YOLO) [21] are examples of these models.

Two-stage frameworks split the detection process into two stages: region proposal and classification. Firstly, these models utilize reference boxes known as anchors and generate multiple object candidates, referred to as Region of Interest (RoI). Subsequently, the proposed regions are classified, and their localization is refined. Examples of the two-stage approaches most commonly used are R-CNN (Region-based Convolutional Neural Network), Fast R-CNN, Faster R-CNN and Cascade R-CNN [14].

Since we are using Faster R-CNN, we explain this model in more detail. Faster R-CNN consists of a single and unified network for object detection, its consists of two modules. The first module is a DCNN that proposes regions and the second module is the Fast R-CNN detector [22] that uses the proposed regions. Faster R-CNN utilizes the recently popular terminology of neural networks with “attention” [23] mechanisms, the Region Proposal Network module tells the Fast R-CNN module where to look.

Furthermore, other studies have demonstrated the efficiency of Faster R-CNN for mitosis detection in histopathological images [16], 24], 25]. Focusing on how this model performs detection on various datasets different from the one we will use, and how stain techniques enhance the model’s performance. However, our study employs a novel and more complex dataset, along with a different staining technique implementation, to address the domain shift problem in the dataset.

2.3 Techniques for addressing domain shift

DS is considered a challenge for the computational pathology community. Domain shift occurs when there are discrepancies in data distribution between the source and target domains, making the direct applicability of trained models to unseen data difficult. Deep Learning models in computational pathology have demonstrated vulnerability to domain shift, as well as typical corruptions and perturbations [26], 27].

Among all the domain generalization methods developed, in this work we apply stain normalization and stain augmentation. Stain normalization serves as a preprocessing stage, this method aims to correct inconsistencies in the colours of histological images resulting from different staining procedures and differences between scanners [28]. Stain augmentation methods aim to generate new images to enhance robustness to colour variations, under the assumption that objects of interests are invariant to changes in colour intensity and illumination [29]. Various studies [29], [30], [31] have shown that stain techniques effectively address the domain shift problem, and that applying stain augmentation after stain normalization achieves better results than using stain normalization alone.

3.1 Dataset

In our experiments, we utilize the MIDOG++ dataset [32], the most recent dataset from the original MIDOG challenge [2]. MIDOG++ expands the MIDOG dataset with the addition of new images and/or annotations for more cases, and two additional tumor types: canine soft tissue sarcoma and human melanoma. Figure 1 presents examples of mitotic figures from this extended dataset.

Figure 1:

Mitotic figures candidates from all domains [32]. Note that h stands for human, c for canine, carci for carcinoma, t for tumor, cut for cutaneous and sarc for sarcoma.

MIDOG++ stands out by providing images across a diverse array of domains, with a strong focus on various tumor types. In fact, this is the first dataset that encompasses multiple sources of domain shifts that are crucial for pathological diagnosis [32].

The MIDOG++ dataset includes region-of-interest images from 503 histological specimens representing seven distinct tumor types with diverse morphologies: breast carcinoma, lung carcinoma, lymphosarcoma, neuroendocrine tumor, cutaneous mast cell tumor, cutaneous melanoma, and (sub)cutaneous soft tissue sarcoma. Both human and canine samples were processed and stained at various human and veterinary pathology laboratories using standard Hematoxylin and Eosin (H&E) staining. The images were digitized by one of five whole slide scanners, each at either 0.23 μm/px or 0.25 μm/px resolution. The dataset contains labels for 11,937 mitotic figures, which were distinguished from 14,351 imposter cells. The labeling process involved a blinded consensus by two pathologists, with a final review by a third pathologist for any disagreements [32].

3.2 Implementation of Faster R-CNN

In order to compare our previous results developed with one-stage models [5], here we implement a two-stage object detection model, namely Faster Region-based Convolutional Neural Network (Faster R-CNN). We choose Faster R-CNN since it is a widely-used in other studies related to mitosis detection [16], 33], 34]. Two-stage models generally achieve higher confidence predictions and better results compared to one-stage models, but they come with significantly higher computational costs and time-complexity during training and inference. For the implementation of Faster R-CNN we use PyTorch due to its widespread use and scalability.

The experiments are conducted across two distinct domains: single-domain and leave-one-out. The single-domain approach trains the model on a single tumor type and evaluates it across all tumor types. In the leave-one-out approach, the model is trained on all tumor types except one and then it is evaluated on all tumor types.

We adapt the training and evaluation procedure to use PyTorch for the utilization of Faster R-CNN model. This training process involves adapting the code from the fastai framework to PyTorch, integrating additional libraries such as Albumentations to ensure precise data transformations, and adding custom classes and elements to facilitate an effective training stage. Our proposed Faster R-CNN model also undergoes fine-tuning and utilizes a ResNet-50-FPN backbone. In particular, our model represents an improved version of the conventional pretrained Faster R-CNN model. Note that using a denser backbone offers benefits like improved precision and generalization but comes with drawbacks such as longer training times and higher computational costs.

For the evaluation stage, we adjust the process to rely solely on the PyTorch framework and revised the implementation of Non-Maximum Suppression (NMS). The results are primarily evaluated using the F1-score, the key metric in the MIDOG challenge.

The F1-score has range [0,1] and is calculated using Equation (1), where TP represents true positives, FP are false positives and FN are false negatives.

3.3 Applying stain augmentation and normalization techniques

In order to tackle the domain shift in the MIDOG++ [32] dataset, we apply stain normalization and stain augmentation technique to our proposed model. We select them due to two main factors. First, stain normalization helps to standardize the variability in visual features, allowing the models to become less sensitive to these inconsistencies and focus on more critical image characteristics. Second, stain augmentation expands the dataset by introducing variations in colour intensity and staining patterns, enhancing the models’ robustness to unseen variations in new domains. We rely on existing implementations of these stain techniques [30], 35] and are not introducing any new methods.

Stain normalization technique addresses stain variation in digital pathology, including the MIDOG challenge [2]. The Vahadane stain normalization method is commonly used since it preserves the structural properties of stained tissue samples and it is robust to stain sparsity in pathology images. Additionally, Vahadane stain normalization has been shown to be superior to other state-of-the-art methods [28].

Vahadane stain normalization employs Sparse Non-negative Matrix Factorization (SNMF) to estimate the stain matrix (S) and concentration matrix (C) from both source and target images. It then scales the concentration map of the source image and combines it with the stain matrix of the target image to achieve normalization [30].

Our model is enhanced by incorporating stain augmentation in training, that involves randomly altering the concentration of H&E stains in the source image. Using the SNMF algorithm, we first extract the source stain matrix (S) and concentration matrix (C). We then scale and shift the stain concentrations, and finally convert the modified stain information back to RGB space, resulting in an augmented image, denoted as Î [36].

The augmented image Î is created according to Equation (2), where I₀ represents the incident intensity of the light source derived from the source image I. Here, α ∼ U(0.75, 1.25) and β ∼ U(−0.2, 0.2) are the stain concentration scale and shift factors, respectively, which are randomly selected from uniform distributions [36].

We perform stain normalization and stain augmentation simultaneously by setting the S matrix in Equation (2) to a pre-extracted target stain matrix. The target stain matrix is obtained by setting a target image and extracting its stain matrix. In our experiments, we utilize “009.tiff” as the target image due to its comprehensive H&E stain colour spectrum. Furthermore, previous works related to the MIDOG challenge have demonstrated its effectiveness for stain augmentation techniques [36].

In our implementation, we utilize TIAToolbox [35] library to realize both stain normalization and stain augmentation techniques.

We first extract the stain matrix from the target image. Then by using TIAToolbox, we integrate the stain normalization and stain augmentation techniques into the Albumentations pipeline developed. Finally, we repeat the experiments with our proposed models to evaluate their performance. We use this new pipeline to train Faster R-CNN with PyTorch, while maintaining the same training configuration of both models.

4.1 Analysis of Faster R-CNN

For Faster R-CNN, in the training phase, the average epoch duration is 63.72 s. Additionally, the average inference time per image is 5.65 s for Faster R-CNN.

For both average training time and average inference time, we observe higher times for Faster R-CNN compared to our previous research [5]. This time increase is due to Faster R-CNN architecture. As a two-stage approach, Faster R-CNN first performs an object proposal stage, and then a classification and regression stage, these two stages increase its complexity. In addition, we are using a denser backbone (ResNet-50-FPN) that implies longer training times and higher computational costs.

We now analyze the results of Faster R-CNN in terms of the F1-score. Figure 2 shows the mean F1-score for each tumor type in each single-domain experiment using Faster R-CNN. The diagonal generally presents higher F1-scores compared to off-diagonal experiments. This is expected, as it shows the model performs best when trained and tested on the same cancer type. For example, human breast carcinoma achieves a strong F1-score when trained and tested on the same type (diagonal), and it also demonstrates good generalization when tested on human melanoma. In addition, canine cutaneous mast cell tumor achieves the highest F1-score of 0.85. However, human neuroendocrine tumor struggles to obtain high F1-scores, and canine lung carcinoma and canine soft tissue sarcoma only achieve good scores when trained with all tumor types. These low scores can be attributed to two main factors: the quantity of mitotic figures associated with each specific tumor type, and the morphological similarities between different tumor types, that limit the model’s generalization capabilities when trained on a single tumor type. Nevertheless, in all tumor types experiment, the primary reason for the low F1-scores remains the limited number of mitotic figures. Table 1 shows the mean and standard deviation of the F1-score achieved using Faster R-CNN in each single-domain experiment.

Figure 2:

Domain matrix for single-domain training for Faster R-CNN. Note that h stands for human, c for canine, carci for carcinoma, t for tumor, cut for cutaneous and sarc for sarcoma.

Figure 3 shows the mean F1-score for each tumor type in each leave-one-out experiment using Faster R-CNN. We observe significant generalization across all experiments when leaving one tumor type out for training. Notably, the canine cutaneous mast cell tumor achieves the highest F1-score of 0.84, with canine lymphosarcoma and human melanoma also yielding strong results. However, the human neuroendocrine tumor type shows the lowest F1-scores, likely due to the smaller number of mitotic figures and images compared to the other tumor types. Table 2 shows the mean and standard deviation of the F1-score achieved using Faster R-CNN in each leave-one-out experiment.

Figure 3:

Domain matrix for leave-one-out training for Faster R-CNN. Note that h stands for human, c for canine, carci for carcinoma, t for tumor, cut for cutaneous and sarc for sarcoma.

As Section 4.3 shows, results obtained by Faster R-CNN surpass those obtained in our previous research [5]. This demonstrates that a two-stage approach model, despite their higher cost in time and computation, achieves better results than a one-stage model in MIDOG++ dataset.

4.2 Analysis of stain augmentation and normalization techniques

We now evaluate and analyze results after implementing the stain augmentation and stain normalization for Faster R-CNN. During the training phase, there is an increase in the average epoch duration due to the addition of the stain techniques. However, the average inference time remains unaffected by this additional processing.

Figure 4 presents the mean F1-score for each tumor type in each single-domain experiment using Faster R-CNN with stain techniques. We observe that results have improved in most cases, highlighting the achievement of better generalization in all experiments. Regarding the diagonal results, F1-scores have improved in all cases, except for canine cutaneous mast cell tumor, which has slightly decreased from 0.85 to 0.83 while still maintaining the highest F1-score, and human melanoma, which has remained the same F1-score. When training with all tumor types, we appreciate a similar pattern as with the diagonal results: each tumor type has slightly improved except for canine cutaneous mast cell tumor, which has decreased from 0.85 to 0.82, and human melanoma, which has remained unchanged. Table 3 shows the mean and standard deviation of the F1-score achieved using Faster R-CNN in each sigle-domain experiment with stain techniques.

Figure 4:

Domain matrix for single-domain training for Faster R-CNN with stain techniques. Note that h stands for human, c for canine, carci for carcinoma, t for tumor, cut for cutaneous and sarc for sarcoma.

Figure 5 presents the mean F1-score for each tumor type in each leave-one-out experiment using Faster R-CNN with stain techniques. The improvement in F1-scores across most cases is appreciable. Notably, the generalization of models has enhanced even when predicting tumor types not included in the training set. Among the different tumor types, the F1-scores for human breast cancer, human neuroendocrine tumor, and canine soft tissue sarcoma stand out. Additionally, in the experiment where canine lymphosarcoma is excluded from training, the F1-score for this tumor type improved from 0.55 to 0.62. However, we also observe a slight decrease in the F1-score for canine cutaneous mast cell tumor, with the previous maximum of 0.84 dropping to 0.82. Table 4 shows the mean and standard deviation of the F1-score achieved using Faster R-CNN in each leave-one-out experiment with stain techniques.

Figure 5:

Domain matrix for leave-one-out training for Faster R-CNN with stain techniques. Note that h stands for human, c for canine, carci for carcinoma, t for tumor, cut for cutaneous and sarc for sarcoma.

4.3 Overall analysis

Finally, we analyze results obtained from all the experiments conducted in our research, to provide a comprehensive overview of which model performs best. Therefore, we compare results from the models developed in our previous research [5] to those obtained by the models proposed in this study.

We first compare Faster R-CNN to RetinaNet-PyTorch and RetinaNet-fastai [5]. We focus on determining which model performs better in terms of F1-score or AP across different scenarios and cases, without considering stain techniques.

Tables 5 and 6 summarize results for F1-score and AP, respectively, with Faster R-CNN, RetinaNet-PyTorch and RetinaNet-fastai, indicating the number of cases where each model obtains the highest F1-score for single-domain cases, along with the mean, maximum and minimum.

Table 5 shows that Faster R-CNN stands out as the best model by winning the most cases compared to other models across both domains. It achieves the highest mean and maximum F1-scores, and its minimum F1-score is acceptable. Regarding RetinaNet-PyTorch, we note its consistency across both domains in comparison to other models. This is evident in the number of cases where it achieves the highest F1-score in leave-one-out, only one case behind from Faster R-CNN.

Table 6 shows that RetinaNet-fastai achieves the best results and wins most cases, surpassing Faster R-CNN. However, the performance of Faster R-CNN is close to that of RetinaNet-fastai. This difference is primarily due to the detection threshold set for each model. A higher detection threshold tends to make more conservative detections, causing the rejection of predictions that might actually be true positives. Consequently, this can increase false negatives and cause a decrease in recall, resulting in a lower AP score. In the experiments, RetinaNet-fastai’s detection thresholds never exceed 0.65, while Faster R-CNN’s thresholds are typically set higher than 0.9 in all cases. During the evaluation stage, we establish the optimal detection threshold by testing different thresholds and selecting the one that achieves the highest F1-score. We observe that this optimal detection threshold depends on the model used, with two-stage models achieving higher detection thresholds compared to one-stage models. RetinaNet-PyTorch falls slightly behind the other models in terms of AP.

Regarding our implementation with the stain techniques Tables 7 and 8 shows results for F1-score and AP, respectively, with Faster R-CNN, Faster R-CNN (S), RetinaNet-PyTorch and RetinaNet-fastai, indicating the number of cases where each model obtains the highest F1-score for single-domain cases, along with the mean, maximum and minimum. Note that (S) stands for models implemented with stain augmentation and normalization.

For F1-score, Table 7 shows that Faster R-CNN with stain techniques achieves the best results among all models, with the highest number of cases and closely obtains the highest mean and maximum F1-scores. Notably, Faster R-CNN also achieves the highest minimum mean F1-score. Nevertheless, the utilization of stain augmentation and normalization techniques improves results for RetinaNet-PyTorch in both domains, but Faster R-CNN without stain techniques still achieves superior results.

For AP, Table 8 shows that, again, RetinaNet-fastai is the best option obtaining always the highest values. The choice of detection thresholds impacts all models in this scenario, since it affects the mean AP score achieved by the models. However, note that Faster R-CNN with stain techniques achieves AP scores close to those of RetinaNet-fastai in single-domain, although it slightly lags behind in leave-one-out experiments.

Finally, we compare the models performance in terms of both training and inference time. Additionally, we analyze the mean detection threshold range of the models, obtained by studying all experiments and the various detection thresholds employed. The summary of all models performance are shown in Table 9.

In terms of average training time per epoch, RetinaNet models achieve lower training times compared to Faster R-CNN. This is attributed to their one-stage architecture, which contrasts with the two-stage architecture of Faster R-CNN. Furthermore, the application of the stain techniques increases the average training time of models. In terms of average inference time per image, one-stage models performs faster inferences compared to two-stage models, and RetinaNet-fastai presents the lowest average inference time. This can be attributed to its lighter ResNet backbone [18] compared to the one used by RetinaNet-Pytorch [5]. The application of stain techniques does not affect in average inference time of models.

Regarding the mean detection threshold range, we observe a notable contrast between the thresholds used by one-stage models and those used by two-stage models. One-stage models commonly set detections threshold that never exceeds 0.7. Among these one-stage models, we appreciate how RetinaNet-PyTorch tends to use higher detection thresholds than RetinaNet-fastai. On the other hand, we observe how two-stage models employs high detection thresholds, often approaching or exceeding 0.9. Furthermore, we notice how the utilization of stain augmentation and normalization tends to make more conservative detections using a higher threshold that models without stain techniques. Notably, the Faster R-CNN model with stain techniques achieves the highest range of detection threshold reaching even 0.98 in different cases.

As a summary, Faster R-CNN with stain augmentation and normalization is the top choice due to its higher detection thresholds, despite longer training and inference times. It provides more reliable predictions compared to RetinaNet. However, if speed is prioritized, RetinaNet is preferable.

In terms of results, Faster R-CNN with stain techniques generally provides the highest F1-scores, which is crucial since F1-score is the main metric in MIDOG challenges. While RetinaNet-fastai achieves the best AP, Faster R-CNN is close behind, particularly in single-domain scenarios. However, its conservative detection approach can slightly lower AP by increasing false negatives. During inference, we observe that Faster R-CNN models generated more predictions compared to the other models, particularly the variant with stain augmentation and normalization, further supporting the application of these Faster R-CNN models. Finally, model performance is impacted by the variability in mitotic figures and the amount of images from different tumor types. Notably, canine lung cancer and human neuroendocrine tumors achieve the lowest results, due to the lower number of mitotic figures present in their images.