Utilization of convolutional neural networks to analyze microscopic images for high-throughput screening of mesenchymal stem cells

MuYun Liu; XiangXi Du; JunYuan Hu; Xiao Liang; HaiJun Wang

doi:10.1515/biol-2022-0859

Article Open Access

Utilization of convolutional neural networks to analyze microscopic images for high-throughput screening of mesenchymal stem cells

MuYun Liu , XiangXi Du , JunYuan Hu , Xiao Liang and HaiJun Wang

Published/Copyright: July 10, 2024

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

From the journal Open Life Sciences Volume 19 Issue 1

Abstract

This work investigated the high-throughput classification performance of microscopic images of mesenchymal stem cells (MSCs) using a hyperspectral imaging-based separable convolutional neural network (CNN) (H-SCNN) model. Human bone marrow mesenchymal stem cells (hBMSCs) were cultured, and microscopic images were acquired using a fully automated microscope. Flow cytometry (FCT) was employed for functional classification. Subsequently, the H-SCNN model was established. The hyperspectral microscopic (HSM) images were created, and the spatial-spectral combined distance (SSCD) was employed to derive the spatial-spectral neighbors (SSNs) for each pixel in the training set to determine the optimal parameters. Then, a separable CNN (SCNN) was adopted instead of the classic convolutional layer. Additionally, cultured cells were seeded into 96-well plates, and high-functioning hBMSCs were screened using both manual visual inspection (MV group) and the H-SCNN model (H-SCNN group), with each group consisting of 96 samples. FCT served as the benchmark to compare the area under the curve (AUC), F1 score, accuracy (Acc), sensitivity (Sen), specificity (Spe), positive predictive value (PPV), and negative predictive value (NPV) between the manual and model groups. The best classification Acc was 0.862 when using window size of 9 and 12 SSNs. The classification Acc of the SCNN model, ResNet model, and VGGNet model gradually increased with the increase in sample size, reaching 89.56 ± 3.09, 80.61 ± 2.83, and 80.06 ± 3.01%, respectively at the sample size of 100. The corresponding training time for the SCNN model was significantly shorter at 21.32 ± 1.09 min compared to ResNet (36.09 ± 3.11 min) and VGGNet models (34.73 ± 3.72 min) (P < 0.05). Furthermore, the classification AUC, F1 score, Acc, Sen, Spe, PPV, and NPV were all higher in the H-SCNN group, with significantly less time required (P < 0.05). Microscopic images based on the H-SCNN model proved to be effective for the classification assessment of hBMSCs, demonstrating excellent performance in classification Acc and efficiency, enabling its potential to be a powerful tool in future MSCs research.

Graphical abstract

Keywords: microscopic images; separable convolutional neural network; human bone marrow mesenchymal stem cells; classification accuracy

1 Introduction

In modern biomedical research, mesenchymal stem cells (MSCs) have garnered significant attention [1]. They possess essential characteristics such as self-renewal, immunomodulation, and multi-lineage differentiation, and are widely distributed in various sources, including bone marrow, adipose tissue, and placenta [2,3]. Clinical studies have indicated the potential of MSCs in tissue engineering, regenerative medicine, and immunotherapy [4,5,6]. It is worth noting that MSCs from different sources exhibit distinct characteristics; for instance, adipose-derived MSCs primarily focus on self-renewal and differentiation [7], umbilical cord MSCs are mainly associated with proliferation and immunomodulation [8], and bone marrow MSCs primarily emphasize multi-lineage differentiation [9]. Quality control is of paramount importance in working with MSCs, as poor quality can lead to less effective treatments or adverse reactions [10,11].

Traditional methods for screening MSCs involve laboratory techniques such as flow cytometry (FCT) or immunohistochemistry. However, these methods are time-consuming, labor-intensive, and require specialized expertise, rendering them unsuitable for high-throughput screening [12,13]. Recently, microscopy analysis has emerged as a highly promising approach for evaluating MSCs’ characteristics [14]. Microscopic images can provide information about cell morphology, distribution, structure, quantity, and functionality, simplifying the differentiation of various MSC types [15]. Nevertheless, manually counting and classifying MSCs through visual inspection often suffer from low efficiency and inadequate classification accuracy (Acc), posing challenges in delivering timely results for clinical applications.

The rapid evolution of artificial intelligence technologies has led to the widespread use of deep learning algorithms for image classification and processing. Within this landscape, the rapid development of convolutional neural networks (CNN) has opened up possibilities for the classification of cells in microscopic images [16]. Experts have already applied CNN for classifying microscopic images, such as using CNN models for microalgae microscopic image classification or using Faster R-CNN and deep CNN for the classification of multi-stage mitotic cell classification and detection [17,18]. CNN has become an essential tool in the field of biomedical image analysis. CNN not only enhances feature extraction and classification capabilities but also possesses the ability to automatically learn [19,20]. As a result, CNN can automatically learn the morphological features of images, facilitating the efficient processing of large-scale image data, thus avoiding the inefficiencies associated with manual feature extraction. This positions CNN as an ideal tool for processing microscopic images [21,22]. Kim et al. [23] confirmed that deep learning models represent a convenient high-throughput method for evaluating the classification efficacy of MSCs and can be used as an effective quality control method in future clinical bio-manufacturing processes. However, traditional CNN methods in the classification of microscopic images only capture information related to cell colors, lacking insight into their underlying biochemical characteristics. As a solution, experts have proposed combining the “all-in-one” characteristic of microscopic images with CNN for cell microscopy image classification. Research has demonstrated that a CNN model incorporating the “spectrum-all-in-one” feature of hyperspectral imaging can not only comprehensively capture information in microscopic cell images but also rapidly and accurately analyze a large number of cell images. Furthermore, it possesses automatic learning capabilities, reducing manual intervention and simplifying the processing, thus positively impacting the advancement of clinical biomanufacturing and cell research [24].

In summary, this work represented the inaugural application of the hyperspectral imaging-based separable CNN (H-SCNN) model combined with hyperspectral imaging technology for the analysis of microscopic images of MSCs, assessing the model’s classification performance on MSCs. The aim of this work is to develop an effective screening method that can automatically learn and extract morphological features from images, thereby mitigating the inefficiencies of manual feature extraction, by harnessing the biological characteristics of MSCs and the computational capabilities of H-SCNN. This empowers clinicians to rapidly and accurately identify MSCs with specific characteristics, promoting further progress in stem cell research and providing robust support for the clinical applications, drug discovery, and fundamental research related to MSCs.

2 Research methods

2.1 Cell culture

In this work, Human bone marrow mesenchymal stem cell (hBMSCs) were sourced from Guangzhou GeniBio Biotechnology Co., Ltd, and were cultivated in vitro for subsequent investigations. The in vitro cultivation of hBMSCs typically necessitates specialized culture media and conditions to maintain their growth and functionality [25]. The specific cultivation method was as follows:

First, the culture medium was prepared, which involved using Dulbecco’s Modified Eagle’s Medium/Ham’s F-12 (DF12), obtained from Guangdong EnviroBio Technology Co., Ltd, as the basal culture medium. In addition, 10–20% fetal bovine serum from Thermo Fisher Scientific, China was incorporated into the medium, followed by the addition of 1% l-glutamine (Jiangsu Pules Biological Technology Co., Ltd) and antibiotics, typically 100 IU/mL of penicillin and 100 μg/mL of streptomycin (Beijing Soleibao Technology Co., Ltd). Subsequently, cell cultivation was initiated: hBMSCs were placed into culture dishes (Thermo Fisher Scientific, China), covered with sterile coverslips from the same source, and incubated in an environment maintained at 37°C with 5% CO₂ gas for 2 weeks. During this period, the culture medium was refreshed every 2–3 days. Growth of hBMSCs was periodically observed to ensure they exhibited their typical fibroblast-like morphology. When the cell density reached a certain level, typically at 80–90% confluence, cell passaging was performed to separate and redistribute hBMSCs into new culture dishes to increase the cell population.

2.2 Acquisition and processing of microscopic images

Under the controlled conditions of 37°C with 5% CO₂, the microscopic imaging of hBMSCs cells was observed using an automated microscope provided by Meigu Molecular Instruments (Shanghai) Co., Ltd. Microscopic images were captured using phase objectives (40× and 100×). A total of 3,200 8-bit grayscale images were collected and were subjected to adjustment based on the hue (H), saturation (S), and value (V) of the image to minimize their impact on the experiments. Images with an average V value exceeding 240 were excluded because excessively high brightness could cause cell boundaries to merge with the background, making differentiation challenging. The average V value of the remaining images was adjusted to approximately 130. Subsequently, the image size was resized to 220 × 300 pixels using interpolation techniques available in the Python OpenCV Toolbox. Ultimately, 1,400 microscopic images were obtained and utilized for subsequent research.

2.3 FCT

Following the acquisition of microscopic images, the research collected corresponding cells for FCT to assess the expression levels of the surface antigens CD73 and CD90. First, hBMSCs were carefully gathered and rinsed with phosphate-buffered saline from Sigma-Aldrich to eliminate culture media and impurities. The cell count was determined using the Countstar, fully automated cell counter from Shanghai Ruiyu Biotech Co., Ltd, and a cell suspension was prepared, maintaining a concentration ranging from 1–5 × 10⁶ cells/mL. Next the required number of cells was taken and placed in Nunc 1.5 mL centrifuge tubes from Thermo Fisher Scientific, China. Subsequently, the cell suspension was combined with fluorescein isothiocyanate (FITC)-labeled CD73 antibody and phycoerythrin (PE)-labeled CD90 antibody from Shanghai Ruiyu Biotech Co., Ltd, both at a concentration of 10 μg/mL. The cells and antibodies were mixed and incubated at 4°C for 30 min. To eliminate unbound antibodies, samples were washed with fluorescence-activated cell sorting (FACS) buffer from Thermo Fisher Scientific, China and then subjected to a 5-min centrifugation at 1,500 rpm for discharging the supernatant. FACS buffer was added to the cell pellet, and the cells were suspended. Flow cytometric analysis of the cell samples was performed using the CytoFLEX S flow cytometer from Beckman Coulter International Trading (Shanghai) Co., Ltd. The instrument was configured to excite and detect FITC and PE fluorescence signals. By detecting the fluorescence signal of each cell, the FCT could determine whether CD73 and CD90 were expressed on the cell surface. The data obtained were subsequently analyzed using DIVA software to gauge the expression levels of CD73 and CD90, providing valuable insights into the cellular properties of hBMSCs. High functionality was defined as CD73 and CD90 positive expression levels exceeding 95%, while lower levels were categorized as indicating reduced functionality.

2.4 Model establishment

2.4.1 Construction of hyperspectral microscopic (HSM) images

The construction of HSM imaging involved leveraging spatial-spectral feature (SSF) information from hyperspectral images to enhance the classification efficacy of microscopic images. Constructing SSF-based microscopic images is an image processing technique that combines spectral and spatial data, typically employed in fields such as materials science and biology. This technology aids in the identification and analysis of the composition, distribution, and properties of different materials or substances. The general steps for constructing SSF-based microscopic images are illustrated in Figure 1. First, the data were acquired. In this work, the hBMSCs properties were assessed using FCT to distinguish between high-performance and low-performance hBMSCs within the microscopic images. Spectral and spatial information for both types of cells were then collected to facilitate the classification. Subsequently, data were preprocessed, including correcting and denoising spectral information and aligning and correcting spatial information to correspond with spectral information. After that, spectral data were merged with spatial information. Finally, microscopic images were constructed. The fused data were adopted to construct microscopic images, with the aid of interpolation techniques available in the Python OpenCV Toolbox for image reconstruction. Furthermore, additional steps such as denoising, enhancement, and contrast adjustment were also carried out.

Figure 1

Construction of HSM images.

Image denoising: (a) the median filtering denoising algorithm was selected to mitigate noise in the image; (b) denoising parameters were fine-tuned to balance denoising effectiveness and the preservation of image details; (c) image denoising tools were employed to apply the selected denoising method for noise reduction.

Image enhancement: (a) image contrast was enhanced initially to highlight target features and reduce background interference; (b) brightness and saturation were adjusted to improve the visual quality of the image; (c) histogram equalization or other enhancement techniques were utilized to optimize the image’s histogram distribution.

Contrast adjustment: (a) image editing tools or dedicated image processing software (such as Adobe Photoshop) were employed to adjust the image’s contrast to ensure that target features were more clearly visible; (b) linear or nonlinear contrast adjustment methods can be employed based on specific requirements; (c) whether the adjusted image meets the analysis or visualization needs was evaluated, making iterative adjustments as per the specific application.

HSM images exhibited a noticeable spatial correlation among pixel distributions, with pixels in close spatial proximity tending to share the same characteristics [26,27]. In this work, spatial-spectral neighbors (SSNs) were selected based on the similarity of joint spatial-spectral information in the neighborhood. This approach can help increase the training samples, as depicted in Figure 2. It was assumed that the dataset of HSM images was represented as X = [ x 1 , x 2 , … , x i ] , where X constituted the data matrix and x _i referred to the spectral vector of the ith pixel. Taking the example of pixels x _i and x _j in HSM images of MSCs, their neighboring spatial regions were represented as S(x _i) and S(x _j), respectively, and the spatial-spectral combined distance SSCD could be expressed as follows:

(1) D ( x i , x j ) = D ( S ( x i ) + S ( x j ) ) .

Figure 2

Calculation process of SSCD.

In the above equation, D ( S ( x i ) + S ( x j ) ) referred to the distance from the pixels x _i to x _j in the neighboring spatial regions S(x _i) and S(x _j), respectively.

2.4.2 SCNN

SCNN, commonly abbreviated as depthwise separable convolution or depthwise separable ConvNet, is a CNN architecture frequently used for image processing and computer vision tasks [28]. The SCNN structure maintains model performance while reducing the number of parameters, thereby lowering computational costs and memory consumption [29].

SCNN is composed of two principal components: depthwise convolution (DC) and pointwise convolution (PC) [30].

DC: In traditional convolutional operations, each input channel undergoes convolution with a convolutional kernel, yielding a single output channel. However, DC is the first step in separable convolution, enabling each input channel to be convolved independently with its respective convolutional kernel, generating output channels equal in number to the input channels, without mixing information between channels. In this work, it was considered that the input feature map possessed C channels and the size of the convolutional kernel is K × K . In this case, DC can be calculated as follows:

For each channel c:

Input of the feature map: I _c, with the size of H × W (H denotes the height and W denotes the weight);

Convolution kernel: K _c, K × K ;

Output of the feature map: O _c, ( H − K + 1 ) × ( W − K + 1 ) ;

DC can be calculated with following equation:

(2) O c [ i , j ] = sum ( I c [ i : i + K , j : j + K ] ⁎ K c ) for i = 0 to H − K , j = 0 to W − K .

PC: It is the second step in separable convolution and involves traditional 1 × 1 convolution. It was employed to condense the quantity of output channels from the DC to the desired number. PC convolved the output of DC using a 1 × 1 convolutional kernel to generate the final output. The specific calculation method for PC is as follows:

It was assumed that there were E output channels for DC:

Input of the feature map: the output O _c of DC, with the size of ( H − K + 1 ) × ( W − K + 1 ) ;

Convolution kernel: there were E convolution kernels with a size of 1 × 1, representing as K1, K2,…, KD;

Output of the feature map: the final output feature map, ( H − K + 1 ) × ( W − K + 1 )

Calculation expression of PC is given in equation (3).

(3) E [ i , j ] = sum ( O c i [ i , j ] ⁎ K i ) for i = 0 to H − K , j = 0 to W − K .

In the above expression, E [ i , j ] refers to each channel of the output feature map and O c i denotes the ith output channel of the DC.

Difference between CNN and SCNN is illustrated in Figure 3.

Figure 3

Comparison between standard CNN and SCNN.

2.4.3 Evaluation methods

This work aimed to assess the effectiveness of the methods employed for classifying HSM images. To achieve this, an initial training dataset comprising of 1,400 microscopic images was applied to obtain the optimal parameter samples for the H-SCNN model. Subsequently, sample sets of sizes 20, 30, 40, 50, 60, 70, 80, 90, and 100 were each selected for analysis to evaluate the classification Acc of SCNN and other CNN models (using FCT detection results as the reference standard). These additional CNN models primarily included well-known ResNet and VGGNet models. Concurrently, the training times required for various CNN models were compared.

2.5 Cell grouping

In Section 2.1, this work involved seeding cultured cells into a 96-well culture plate. Subsequently, an automated microscope was utilized to observe the hBMSCs present in each well. Following this, high-functioning hBMSCs were screened through two distinct approaches: manual visual inspection (MV group) and the H-SCNN model. These two screening methods were designated as the MV group and the H-SCNN group, respectively, each consisting of 96 samples. The evaluation of both screening methods was performed, with FCT detection results serving as the reference standard.

2.6 Observation parameters

Furthermore, the analysis effectiveness of classification methods for hBMSCs in different groups was compared using distinct metrics, including area under the curve (AUC), F1 score, Acc, sensitivity (Sen), specificity (Spe), positive predictive value (PPV), and negative predictive value (NPV). Additionally, the time differences between distinct classification methods were observed to identify an efficient and effective screening method with strong classification efficacy.

(4) Acc = TP + TN TP ,

(5) Sen = TP TP + FN ,

(6) Spe = TN TN + FP ,

(7) PPV = TP TP + FP ,

(8) NPV = TN TN + FN .

where TP represents the number of samples that are actually positive and correctly predicted as positive by the classifier; TN refers to the number of samples that are actually negative and correctly predicted as negative by the classifier; FP signifies the number of samples that are actually negative but incorrectly predicted as positive by the classifier; and FN indicates the number of samples that are actually positive but incorrectly predicted as negative by the classifier.

2.7 Methods for statistical analysis

Data were processed using SPSS 26.0. Continuous data were displayed as mean value ± standard deviation and were compared using the t-test. Categorical data were presented as frequencies or percentages (%) and were compared using the χ² test. P < 0.05 was considered statistically significant.

3 Results

3.1 Construction of HSM images and classification efficacy

Based on the FCT results, the different functional levels of hBMSCs were labeled in the corresponding microscopic images. White represented the background, red indicated high functionality, and yellow represented low functionality. Simultaneously, the distribution of hBMSCs in the corresponding hyperspectral microscopy ground truth images was observed, where blue signified the background, orange indicated high functionality, and white represented low functionality. Through comparative observations, the distribution of hBMSCs in both scenarios was found to be quite consistent, as displayed in Figure 4a–c. To obtain the optimal algorithm parameters, this work further compared the classification Acc under different Window size (WS) and SSNs numbers. WS was selected from 1, 3, 5, 7, 9, 11, and 13, while SSN numbers were chosen sequentially from 2, 4, 6, 8, 10, 12, 14, and 16. The classification Acc is shown in Figure 4d. It was found that when WS was set to 5–11, Acc was higher, and when the number of SSNs was 8–12, Acc was higher. When WS was set to 9 and the number of SSNs was 12, the classification Acc was 0.862, reaching the highest, indicating that it was the best result. In addition, these parameters were also the basis for subsequent research.

Figure 4

Distribution of hBMSCs and classification Acc: (a) original image; (b) FCT results; (c) hyperspectral hBMSCs microscopic truth map; and (d) results with WS of WS 1, 3, 5, 7, 9, 11, and 13, respectively.

3.2 Classification performance of various CNN models

Sample sizes of 20, 30, 40, 50, 60, 70, 80, 90, and 100 microscopic images were employed for training to compare the classification Acc of the hyperspectral hBMSCs microscopic images among the SCNN, ResNet, and VGGNet models. Additionally, the classification efficiencies of the three models were evaluated. As the sample size increased, the classification Acc of all three models gradually increased (Figure 5a). When the sample size reached 100, each model obtained the highest classification Acc. The classification Acc of SCNN, ResNet, and VGGNet models were 89.56 ± 3.09, 80.61 ± 2.83, and 80.06 ± 3.01%, respectively. The classification efficiency of the SCNN model was much higher than that of the ResNet and VGGNet models (P < 0.05) (Figure 5b). Figure 5c shows that the training time of SCNN, ResNet, and VGGNet models was 21.32 ± 1.09, 36.09 ± 3.11, and 34.73 ± 3.72 min, respectively. Compared to ResNet and VGGNet models, SCNN models had a shorter training time (P < 0.05). Figures 5d–g represent the classification power diagram, indicating that the SCNN model exhibited significantly superior classification performance and was more similar to ground real images.

Figure 5

Comparison of classification performance of SCNN, ResNet, and VGGNet models. (a) Acc of SCNN, ResNet, and VGGNet models, respectively; (b) Acc when the sample size was 100; (c) training time; (d) ground truth image; (e) SCNN model; (f) ResNet model; (g) VGGNet model; “*” indicated a statistical significance (P < 0.05) compared to ResNet and VGGNet models.

3.3 Comparison on screening efficacy in the H-SCNN and MV groups

In this work, several metrics, including AUC, F1 score, Acc, Sen, Spe, PPV, and NPV, were selected to analyze the classification efficacy of hBMSCs in both the MV group and H-SCNN group. Figure 6a displays the ROC curve. According to the ROC analysis, the AUC, F1 score, Acc, Sen, Spe, PPV, and NPV for hBMSCs classification in the MV group were 0.908, 0.826, 0.817, 0.819, 0.816, 0.853, and 0.822, respectively. For the H-SCNN group, the corresponding values were 0.968, 0.918, 0.908, 0.951, 0.928, 0.955, and 0.912, respectively. Comparatively, the classification AUC, F1 score, Acc, Sen, Spe, PPV, and NPV for the H-SCNN group were all higher than those for the MV group, exhibiting obvious differences (P < 0.05), as explicated in Figure 6d. Furthermore, it was observed that the MV group required 60.28 ± 4.16 min to classify 96 microscopic images, whereas the H-SCNN group completed the task in only 20.11 ± 2.17 min, which was obviously faster (P < 0.05), as depicted in Figure 6e.

Figure 6

Classification efficacy for hBMSCs in MV and H-SCNN groups. (a) ROC curve; (b) AUC; (c) F1 score; (d) Acc, Sen, Spe, PPV, and NPV; (e) time. Note: * suggested a substantial difference with P < 0.05 in contrast to the MV group.

4 Discussion

In this work, FCT was utilized to assess the levels of CD73 and CD90 in hBMSCs. CD73, also known as 5′-nucleotidase, is a surface molecule typically expressed in BMSCs. Its primary role involves the conversion of adenosine monophosphate into adenosine on the cell surface, thereby regulating immune responses and cell signal transduction [31]. CD90, also known as Thy-1 or THY1, is a common marker for BMSCs and serves as a surface antigen. It is often utilized for identifying and isolating BMSC populations [32]. Based on the evaluation of functional levels of hBMSCs from the test results, the corresponding hBMSCs in the microscopic images were classified as high-functioning or low-functioning. Once the data were collected, the HSM images were constructed, and the optimal algorithm parameters were determined by comparing the classification Acc under various WS and SSN values. The results suggested that the best parameter combination was WS = 9 and SSN = 12, which achieved a classification Acc of 0.862, making it the best parameter combination for this study. This finding underscores the importance of parameter selection for accurate classification and provides a strong benchmark for subsequent research.

Based on the results, this work further compared the performance of the SCNN, ResNet, and VGGNet models in classifying high-spectral hBMSCs microscopic images. These three models are all commonly utilized in deep learning, are variants of CNN, and are constructed with components like convolutional layers, pooling layers, and fully connected layers for tasks such as image classification and feature extraction. However, they differ in network depth, the number of parameters, and their suitability for various tasks [33,34,35]. VGGNet is relatively shallow, featuring either 16 or 19 convolutional layers and a large number of parameters. ResNet, on the other hand, is very deep, typically having 50, 101, or even more convolutional layers, but fewer parameters compared to VGGNet. In contrast, SCNN is a specialized CNN designed for semantic segmentation, typically consisting of convolutional and deconvolutional layers for pixel-level labeling. It usually falls between VGGNet and ResNet in terms of the number of parameters. While the first two are often used for image classification tasks, SCNN excels in assigning each pixel in an image to a specific category and is typically used for image segmentation tasks. All three have found applications in cell classification studies [36], but this work represented the first comparison of their classification performance. SCNN is a neural network architecture specifically designed for image segmentation tasks, often used to segment different cell structures or nuclei in cell images [37]. In cell classification, SCNN can be used to locate and segment cell nuclei and other cellular components, providing valuable data for subsequent classification tasks to achieve more accurate cell classification and identification [38]. Given that hyperspectral images often contain a substantial number of parameters, efficient training is a key challenge. SCNN excels in handling high-dimensional, large-scale hyperspectral data. Its architectural design effectively reduces the number of model parameters, enhances feature extraction, mitigates overfitting, and improves computational efficiency [39,40]. This work revealed that the SCNN model achieved the highest classification Acc, significantly outperforming the performance of the ResNet and VGGNet models. Furthermore, the training time required for the SCNN model was notably lower in contrast to the other two models. These findings indicate the advantages of the H-SCNN model in terms of classification Acc and efficiency.

In conclusion, the H-SCNN model and MV group methods were adopted to classify high and low functional hBMSCs. The results demonstrated the superiority of the H-SCNN group over the MV group in terms of classification AUC, F1 score, Acc, Sen, Spe, PPV, and NPV. Additionally, the H-SCNN group required significantly less time compared to the MV group. This further emphasizes the clear advantages of the H-SCNN model in both classification performance and efficiency. Manual cell classification often relies on the subjective judgment and expertise of trained biologists or medical professionals. Acc can be influenced by subjective factors, leading to potential errors. Moreover, manual classification is labor-intensive and can significantly impact processing speed when dealing with large datasets [41]. In contrast, machine learning algorithms can be trained on extensive and well-labeled datasets, enabling rapid and highly accurate classification [42]. Therefore, the H-SCNN model offers distinct advantages over manual methods. Lyu et al. [43] and Honrado et al. [44] have also proposed through their research that machine learning methods offer greater speed, efficiency, and consistency in cell classification. Lien et al. [45] proposed a multi-layer tensor model, which is an improved CNN that can classify cells derived from induced pluripotent stem cells and evaluate their differentiation efficiency. This model demonstrated the ability to classify MSCs, retinal ganglion cells, and retinal pigment epithelial cells with an Acc of 97.8%. Additionally, it demonstrated the potential to identify candidate cells with ideal characteristics while excluding cells with immature/abnormal phenotypes. Wang et al. [46] proposed an analysis method based on cell physical characteristics and a deep learning method for identifying cell types. By analyzing the processed image using an optimized CNN, two sets of cells (HL-7702 and SMMC-7721, SGC-7901 and GES-1) can be identified. The results showed that using deep learning technology to recognize the physical characteristics of cells can be a universal and effective automatic analysis method for cell information. It is evident that machine learning-based cell classification is typically faster, more consistent, and adaptable, making it particularly well-suited for large-scale cell classification tasks.

5 Conclusion

In conclusion, the results and discussions presented above clearly demonstrated the effectiveness of utilizing HSM images and machine learning models for the classification of hBMSCs. In particular, the H-SCNN model exhibited outstanding performance in terms of classification Acc and efficiency, positioning it as a powerful tool for future MSCs research. This work yielded strong support and methods for further exploration of the biological characteristics and clinical applications of MSCs. However, it is essential to acknowledge that the success of these machine learning models hinges on the availability of a substantial amount of labeled data and the fine-tuning of algorithms. Manual cell classification, though time-consuming and subjective, remained useful in certain cases, particularly in scenarios where there was insufficient training data available for machine learning or when complex cell classification situations require the expertise of human professionals. Therefore, in the field of machine learning-based cell classification, researchers should direct their efforts toward refining deep learning algorithms to reduce the reliance on a large amount of labeled data. Techniques such as transfer learning, weakly supervised learning, and self-supervised learning can help enhance algorithm generalization, thus reducing the need for labeled data. This avenue of research holds the potential to further advance the field of automated cell classification while maintaining the flexibility and expertise of human judgment when needed.

Acknowledgments

The software of analysis of physical morphological characteristics of cells used for the Convolutional Neural Network analysis of microscopic images for high-throughput screening of MSCs has been provided by Shenzen CellAuto Company.

Funding information: This work was supported by Innovation Capacity Building Project, National Engineering Research Center of Foundational Technologies for CGT Industry (NDRC-High-Technology [2023] No. 447 to M.L.) and Shenzhen Non-invasive Cell Quality Online Monitoring and Analysis Platform (F-2022-Z99-502233 to M.L.).
Author contributions: Muyun Liu designed the study, wrote the paper, supervised project administration and acquired funding. JunYuan Hu and Xiao Liang prepared mesenchymal stem cells and collect microscopic images. Haijun Wang wrote software program and performed convolutional neural network analysis. Xiangxi Du made coordination of the study. All authors discussed the results and commented on the manuscript.
Conflict of interest: Haijun Wang, who wrote software program and performed Convolutional Neural Network analysis and Xiangxi Du, who made coordination of the study, were employees of Shenzen CellAuto Company during the research conduction. This fact did not affect the results of the study.
Data availability statement: The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

[1] Tamaddon AM, Bashiri R, Najafi H, Mousavi K, Jafari M, Borandeh S, et al. Biocompatibility of graphene oxide nanosheets functionalized with various amino acids towards mesenchymal stem cells. Heliyon. 2023;9(8):e19153.10.1016/j.heliyon.2023.e19153Search in Google Scholar PubMed PubMed Central

[2] Tammam BMH, Habotta OA, El-Khadragy M, Abdel Moneim AE, Abdalla MS. Therapeutic role of mesenchymal stem cells and platelet-rich plasma on skin burn healing and rejuvenation: A focus on scar regulation, oxido-inflammatory stress and apoptotic mechanisms. Heliyon. 2023;9(9):e19452.10.1016/j.heliyon.2023.e19452Search in Google Scholar PubMed PubMed Central

[3] Zakiyah N, Wanandi SI, Antarianto RD, Syahrani RA, Arumsari S. Mesenchymal stem cell-derived extracellular vesicles increase human MCF7 breast cancer cell proliferation associated with OCT4 expression and ALDH activity. Asian Pac J Cancer Prev. 2023;24(8):2781–9.10.31557/APJCP.2023.24.8.2781Search in Google Scholar PubMed PubMed Central

[4] Maroquenne M, Bourguignon M, Larochette N, El-Hafci H, Margottin M, Potier E, et al. The lower in vivo osteogenicity of adipose tissue-derived stem cells correlates with a higher innate immune response. Stem Cell Rev Rep. 2023;1:2869–85. 10.1007/s12015-023-10614.Search in Google Scholar

[5] Alves DVL, Claudio-da-Silva C, Souza MCA, Pinho RT, Silva WSD, Sousa-Vasconcelos PS, et al. Adipose tissue-derived mesenchymal stromal cells from ex-morbidly obese individuals instruct macrophages towards a M2-like profile in vitro. Int J Stem Cell. 2023;16:425–37. 10.15283/ijsc22172. Search in Google Scholar

[6] Ortiz GGR, Zaidi NH, Saini RS, Ramirez Coronel AA, Alsandook T, Hadi Lafta M, et al. The developing role of extracellular vesicles in autoimmune diseases: special attention to mesenchymal stem cell-derived extracellular vesicles. Int Immunopharmacology. 2023;122:110531.10.1016/j.intimp.2023.110531Search in Google Scholar PubMed

[7] Kwapisz A, Bowman M, Walters J, Cosh H, Burnikel B, Tokish J, et al. Human adipose- and amnion-derived mesenchymal stromal cells similarly mitigate osteoarthritis progression in the Dunkin-Hartley guinea pig. Am J Sports Med. 2022;50(14):3963–73.10.1177/03635465221126683Search in Google Scholar PubMed

[8] Perez JA, Parcero Valdes JJ, Corral Moreno R, Gomez Cuevas LI, Lopez JJ, Ichim T, et al. Intravenous administration of umbilical cord-derived mesenchymal stem cells (UC-MSC) for acute respiratory distress syndrome due to COVID-19 infection. Cureus. 2023;15(8):e44110.10.7759/cureus.44110Search in Google Scholar PubMed PubMed Central

[9] Fiévet L, Espagnolle N, Gerovska D, Bernard D, Syrykh C, Laurent C, et al. Single-cell RNA sequencing of human non-hematopoietic bone marrow cells reveals a unique set of inter-species conserved biomarkers for native mesenchymal stromal cells. Stem Cell Res Ther. 2023;14(1):229.10.1186/s13287-023-03437-xSearch in Google Scholar PubMed PubMed Central

[10] Arango-Rodríguez ML, Mateus LC, Sossa CL, Becerra-Bayona SM, Solarte-David VA, Ochoa Vera ME, et al. A novel therapeutic management for diabetes patients with chronic limb-threatening ischemia: comparison of autologous bone marrow mononuclear cells versus allogenic Wharton jelly-derived mesenchymal stem cells. Stem Cell Res Ther. 2023;14(1):221.10.1186/s13287-023-03427-zSearch in Google Scholar PubMed PubMed Central

[11] Husakova J, Echalar B, Kossl J, Palacka K, Fejfarova V, Dubsky M. The effects of immunosuppressive drugs on the characteristics and functional properties of bone marrow-derived stem cells isolated from patients with diabetes mellitus and peripheral arterial disease. Biomedicines. 2023;11(7):1872.10.3390/biomedicines11071872Search in Google Scholar PubMed PubMed Central

[12] Zhao X, Wu J, Yuan R, Li Y, Yang Q, Wu B, et al. Adipose-derived mesenchymal stem cell therapy for reverse bleomycin-induced experimental pulmonary fibrosis. Sci Rep. 2023;13(1):13183.10.1038/s41598-023-40531-9Search in Google Scholar PubMed PubMed Central

[13] Yildirim S, Oylumlu E, Ozkan A, Sinen O, Bulbul M, Goksu ET, et al. Zinc (Zn) and adipose-derived mesenchymal stem cells (AD-MSCs) on MPTP-induced Parkinson’s disease model: A comparative evaluation of behavioral and immunohistochemical results. Neurotoxicology. 2023;97:1–11.10.1016/j.neuro.2023.05.002Search in Google Scholar PubMed

[14] Enrriques AE, Howard S, Timsina R, Khadka NK, Hoover AN, Ray AE, et al. Atomic force microscopy cantilever-based nanoindentation: mechanical property measurements at the nanoscale in air and fluid. J Visualized Exp. 2022;190. 10.3791/64497. Search in Google Scholar

[15] Lubis AMT, Aprianto P, Pawitan JA, Priosoeryanto BP, Dewi TIT, Kamal AF. Intra-articular injection of secretome, derived from umbilical cord mesenchymal stem cell, enhances the regeneration process of cartilage in early-stage osteo-arthritis: an animal study. Acta Orthop. 2023;94:300–6.10.2340/17453674.2023.12359Search in Google Scholar PubMed PubMed Central

[16] Ahmad A, Sala F, Paiè P, Candeo A, D’Annunzio S, Zippo A, et al. On the robustness of machine learning algorithms toward microfluidic distortions for cell classification via on-chip fluorescence microscopy. Lab a Chip. 2022;22(18):3453–63.10.1039/D2LC00482HSearch in Google Scholar PubMed

[17] Wongchai A, Jenjeti DR, Priyadarsini AI, Deb N, Bhardwaj A, Tomar P. Farm monitoring and disease prediction by classification based on deep learning architectures in sustainable agriculture. Ecol Model. 2022;474:110167.10.1016/j.ecolmodel.2022.110167Search in Google Scholar

[18] Kumar A, Sinha N, Bhardwaj A, Goel S. Clinical risk assessment of chronic kidney disease patients using genetic programming. Comput Methods Biomech Biomed Eng. 2022;25(8):887–95.10.1080/10255842.2021.1985476Search in Google Scholar PubMed

[19] Mochalova EN, Kotov IA, Lifanov DA, Chakraborti S, Nikitin MP. Imaging flow cytometry data analysis using convolutional neural network for quantitative investigation of phagocytosis. Biotechnol Bioeng. 2022;119(2):626–35.10.1002/bit.27986Search in Google Scholar PubMed

[20] Hussain E, Mahanta LB, Das CR, Choudhury M, Chowdhury M. A shape context fully convolutional neural network for segmentation and classification of cervical nuclei in Pap smear images. Artif Intell Med. 2020;107:101897.10.1016/j.artmed.2020.101897Search in Google Scholar PubMed

[21] Huang Y, Ding X, Zhao Y, Tian X, Feng G, Gao Z. Automatic detection and segmentation of chorda tympani under microscopic vision in otosclerosis patients via convolutional neural networks. Int J Med Robot + Comput Assist Surg. 2023;19:e2567.10.1002/rcs.2567Search in Google Scholar PubMed

[22] Ma Y, Zhong Y, Su Q, Xu L, Song H, Wen C. Study on identification algorithm of traditional Chinese medicinals microscopic image based on convolutional neural network. Medicine. 2023;102(25):e34085.10.1097/MD.0000000000034085Search in Google Scholar PubMed PubMed Central

[23] Kim G, Jeon JH, Park K, Kim SW, Kim DH, Lee S. High throughput screening of mesenchymal stem cell lines using deep learning. Sci Rep. 2022;12(1):17507.10.1038/s41598-022-21653-ySearch in Google Scholar PubMed PubMed Central

[24] Ma L, Zhou X, Little JV, Chen AY, Myers LL, Sumer BD et al. Hyperspectral microscopic imaging for the detection of head and neck squamous cell carcinoma on histologic slides. Proceedings of SPIE the International Society for Optical Engineering; 2021. p. 1160310.1117/12.2581970Search in Google Scholar PubMed PubMed Central

[25] Ajeeb B, Detamore M. Comparison of multiple synthetic chondroinductive factors in pellet culture against a TGF-β positive control. Osteoarthr Cartil open. 2023;5(3):100369.10.1016/j.ocarto.2023.100369Search in Google Scholar PubMed PubMed Central

[26] Yin J, Zhang M, Tan Y, Guo Z, He H, Lan L, et al. Video-rate mid-infrared photothermal imaging by single-pulse photothermal detection per pixel. Sci Adv. 2023;9(24):eadg8814.10.1126/sciadv.adg8814Search in Google Scholar PubMed PubMed Central

[27] Treepong P, Theera-Ampornpunt N. Early bread mold detection through microscopic images using convolutional neural network. Curr Res Food Sci. 2023;7:100574.10.1016/j.crfs.2023.100574Search in Google Scholar PubMed PubMed Central

[28] Riyad M, Khalil M, Adib A. MI-EEGNET: A novel convolutional neural network for motor imagery classification. J Neurosci Methods. 2021;353:109037.10.1016/j.jneumeth.2020.109037Search in Google Scholar PubMed

[29] Kong X, Zhang K. A novel text sentiment analysis system using improved depthwise separable convolution neural networks. PeerJ Comput Sci. 2023;9:e1236.10.7717/peerj-cs.1236Search in Google Scholar PubMed PubMed Central

[30] Tseng FH, Yeh KH, Kao FY, Chen CY. MiniNet: Dense squeeze with depthwise separable convolutions for image classification in resource-constrained autonomous systems. ISA Trans. 2023;132:120–30.10.1016/j.isatra.2022.07.030Search in Google Scholar PubMed

[31] Stefańska K, Nemcova L, Blatkiewicz M, Pieńkowski W, Ruciński M, Zabel M, et al. Apoptosis related human Wharton’s jelly-derived stem cells differentiation into osteoblasts, chondrocytes, adipocytes and neural-like cells-complete transcriptomic assays. Int J Mol Sci. 2023;24(12):10023.10.3390/ijms241210023Search in Google Scholar PubMed PubMed Central

[32] Shah AA, Sheikh AA, Hasin D, Shah F, Aarif O, Shah RA, et al. Isolation, in vitro expansion and characterization of ovine fetal adnexa-derived mesenchymal stem cells reveals a source dependent trilineage differentiation and growth kinetics. Anim Biotechnol. 2023;34:1–12.10.1080/10495398.2023.2238015Search in Google Scholar PubMed

[33] Mohanty R, Allabun S, Solanki SS, Pani SK, Alqahtani MS, Abbas M, et al. NAMSTCD: a novel augmented model for spinal cord segmentation and tumor classification using deep nets. Diagnostics. 2023;13(8):1417.10.3390/diagnostics13081417Search in Google Scholar PubMed PubMed Central

[34] Ji L, Wei Z, Hao J, Wang C. An intelligent diagnostic method of ECG signal based on Markov transition field and a ResNet. Comput Methods Prog Biomed. 2023;242:107784.10.1016/j.cmpb.2023.107784Search in Google Scholar PubMed

[35] Mirzabagherian H, Menhaj MB, Suratgar AA, Talebi N, Abbasi Sardari MR, Sajedin A. Temporal-spatial convolutional residual network for decoding attempted movement related EEG signals of subjects with spinal cord injury. Comput Biol Med. 2023;164:107159.10.1016/j.compbiomed.2023.107159Search in Google Scholar PubMed

[36] Hansang L, Helen H, Junmo K, Dae Chul J. High content imaging of early morphological signatures predicts long term mineralization capacity of human mesenchymal stem cells upon osteogenic induction. Med Phys. 2018;45(4):1550–61.Search in Google Scholar

[37] Shan T, Yan J, Cui X, Xie L. DSCA-Net: A depthwise separable convolutional neural network with attention mechanism for medical image segmentation. Math Biosci Eng. 2023;20(1):365–82.10.3934/mbe.2023017Search in Google Scholar PubMed

[38] Balasubaramanian S, Cyriac R, Roshan S, Maruthamuthu Paramasivam K, Chellanthara Jose B. An effective stacked autoencoder based depth separable convolutional neural network model for face mask detection. Array. 2023;19:100294.10.1016/j.array.2023.100294Search in Google Scholar PubMed PubMed Central

[39] Asghar MZ, Albogamy FR, Al-Rakhami MS, Asghar J, Rahmat MK, Alam MM, et al. Facial mask detection using depthwise separable convolutional neural network model during COVID-19 pandemic. Front Public Health. 2022;10:855254.10.3389/fpubh.2022.855254Search in Google Scholar PubMed PubMed Central

[40] Le DN, Parvathy VS, Gupta D, Khanna A, Rodrigues JJPC, Shankar K. IoT enabled depthwise separable convolution neural network with deep support vector machine for COVID-19 diagnosis and classification. Int J Mach Learn Cybern. 2021;12(11):3235–48. 10.1007/s13042-020-01248-7 Search in Google Scholar PubMed PubMed Central

[41] Kamyab-Hesari K, Azhari V, Ahmadzade A, Asadi Amoli F, Najafi A, Hasanzadeh A, et al. Machine learning for classification of cutaneous sebaceous neoplasms: implementing decision tree model using cytological and architectural features. Diagn Pathol. 2023;18(1):89.10.1186/s13000-023-01378-wSearch in Google Scholar PubMed PubMed Central

[42] Piansaddhayanon C, Koracharkornradt C, Laosaengpha N, Tao Q, Ingrungruanglert P, Israsena N, et al. Label-free tumor cells classification using deep learning and high-content imaging. Sci Data. 2023;10(1):570.10.1038/s41597-023-02482-8Search in Google Scholar PubMed PubMed Central

[43] Lyu M, Xin L, Jin H, Chitkushev LT, Zhang G, Keskin DB, et al. Protocol for classification single-cell PBMC types from pathological samples using supervised machine learning. Methods Mol Biol. 2023;2673:53–67.10.1007/978-1-0716-3239-0_4Search in Google Scholar PubMed

[44] Honrado C, Salahi A, Adair SJ, Moore JH, Bauer TW, Swami NS. Automated biophysical classification of apoptotic pancreatic cancer cell subpopulations by using machine learning approaches with impedance cytometry. Lab a chip. 2022;22(19):3708–20.10.1039/D2LC00304JSearch in Google Scholar PubMed PubMed Central

[45] Lien CY, Chen TT, Tsai ET, Hsiao YJ, Lee N, Gao CE, et al. Recognizing the differentiation degree of human induced pluripotent stem cell-derived retinal pigment epithelium cells using machine learning and deep learning-based approaches. Cells. 2023;12(2):211.10.3390/cells12020211Search in Google Scholar PubMed PubMed Central

[46] Wang J, Gao M, Yang L, Huang Y, Wang J, Wang B, et al. Cell recognition based on atomic force microscopy and modified residual neural network. J Struct Biol. 2023;215(3):107991.10.1016/j.jsb.2023.107991Search in Google Scholar PubMed

Received: 2023-10-01

Revised: 2023-11-21

Accepted: 2023-11-26

Published Online: 2024-07-10

This work is licensed under the Creative Commons Attribution 4.0 International License.

Articles in the same Issue

https://doi.org/10.1515/biol-2022-0859

Keywords for this article

microscopic images; separable convolutional neural network; human bone marrow mesenchymal stem cells; classification accuracy

Creative Commons

BY 4.0