Home Performance of the automated digital cell image analyzer UIMD PBIA in white blood cell classification: a comparative study with sysmex DI-60
Article Open Access

Performance of the automated digital cell image analyzer UIMD PBIA in white blood cell classification: a comparative study with sysmex DI-60

  • Hongkyung Kim ORCID logo , Oh Joo Kweon ORCID logo , Sumi Yoon ORCID logo , Yong Kwan Lim ORCID logo and Bohyun Kim ORCID logo EMAIL logo
Published/Copyright: January 23, 2025

Abstract

Objectives

This study aimed to evaluate the performance of PBIA (UIMD, Seoul, Republic of Korea), an automated digital morphology analyzer using deep learning, for white blood cell (WBC) classification in peripheral blood smears and compare it with the widely used DI-60 (Sysmex, Kobe, Japan).

Methods

A total of 461 slides were analyzed using PBIA and DI-60. For each instrument, pre-classification performance was evaluated on the basis of post-classification results verified by users. Pre- and post-classification results were compared with manual WBC differentials, and the ability to identify abnormal cells was assessed.

Results

The pre-classification performance of PBIA was better than that of DI-60 for most cell classes. PBIA had an accuracy of 90.0 % and Cohen’s kappa of 0.934, higher than DI-60 (45.5 % accuracy and 0.629 kappa) across all cell classes. The pre-classification performance of both instruments decreased when abnormal cells were observed in manual counts, but PBIA still performed better. PBIA also appeared to show better correlation with manual WBC differential counts, particularly in pre-classification (Pearson’s correlation coefficient: 0.696–0.944 vs. 0.230–0.882 for neutrophils, lymphocytes, monocytes, eosinophils, basophils, and blasts), although the mean differences varied by cell class. For abnormal cells identified in manual counts, PBIA exhibited more false positives for blasts (30.5 vs. 2.3 %), while DI-60 had a higher rate of false negatives (42.1 vs. 6.1 %). Both instruments exhibited high false negative rates for atypical lymphocytes.

Conclusions

PBIA demonstrated better performance than DI-60, highlighting its clinical utility. Further multicenter studies are required for full validation.

Introduction

In clinical hematology, the examination of peripheral blood smears (PBS) is essential for diagnosing hematologic disorders [1], 2]. When abnormalities are flagged by automated hematology analyzers (approximately 10–15 % of blood samples) or clinical concerns arise, further morphological verification is needed [2], [3], [4]. Traditionally, it is performed through a manual microscopic review, which is the gold standard for white blood cell (WBC) differential counts [1], 2], 5]. However, manual review is labor intensive, time consuming, and prone to inter- and intra-observer variability, thus affecting accuracy and consistency [1], 6], 7].

Automated digital morphology analyzers have been developed to address these limitations, thereby providing faster, more consistent results with less reliance on expert labor [6], 8]. Recently, their accuracy in manual count tasks has been improved by artificial intelligence integration [9]. With these features, digital morphology analyzers are increasingly valuable in modern hematology labs; however, their performance must be thoroughly validated before clinical implementation, particularly their ability to identify abnormal findings [1], 7].

PBIA (UIMD, Seoul, Republic of Korea) is a system designed to automate the pre-classification of WBCs via deep learning algorithms [10], 11]. Similar to the comprehensively studied DI-60 (Sysmex, Kobe, Japan) [6], 8], [12], [13], [14], PBIA automates the examination of PBS, thus offering rapid WBC differential results. Despite its potential, studies on the performance of PBIA are limited [10], 11].

In this study, we evaluated the WBC classification performance of PBIA and compared it with DI-60. This comparative study aimed to provide valuable insights into the clinical utility of PBIA.

Materials and methods

Samples

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board/Ethics Committee of Soonchunhyang University College of Medicine, Cheonan, Republic of Korea (approval no. 2024-02-009). The need for informed consent was waived because residual samples were used after the requested complete blood count (CBC) and PBS examinations were completed.

A total of 461 PBS slides were selected from our routine laboratory workload, mostly from patient samples. Venous blood was collected using a K2-EDTA tube (BD, Fanklin Lakes, NJ, USA). CBC for WBC count was performed using XN-9000 (Sysmex, Kobe, Japan) within 4 h of sample collection. PBS slides were prepared and stained by an automated SP-10 slide maker (Sysmex) using Wright–Giemsa stain (Sysmex) and buffer (CellaVision AB, Lund, Sweden), with following protocol: 2 min for Wright pure, 3 min for Wright diluted (1:10), 5 min for Giemsa diluted (1:25), 15 s for rinsing, and 10 min for drying. In each slide, manual WBC differential counts (200 cells) were performed by two well-trained laboratory experts (a skilled medical technician with over 10 years of experience and a laboratory medicine doctor) using the battlement track pattern. If nucleated RBCs (nRBCs) were observed, they were counted per 100 WBCs. Of the 461 PBS slides analyzed, 114 had blasts, 28 had atypical lymphocytes, and 79 had nRBCs. When categorized by WBC count, 53 slides had <1.00 × 109/L; 85 had 1.00–2.00 × 109/L; 84 had 2.00–4.00 × 109/L; 83 had 4.00–10.00 × 109/L; and 156 had >10.00 × 109/L. Among the >10.00 × 109/L group, 75 (48.1 %) contained blasts on manual inspection, whereas none <1.00 × 109/L did. Of the 4,514 blasts identified by manual count, 4,373 (96.9 %) were observed in slides with WBC counts >10.00 × 109/L.

PBIA

This automated digital image analyzing system is composed of hardware and software components. The hardware component includes 12-slide cassettes, a slide barcode scanner, a precision stage, and a microscope optical system. The software component comprises a user interface, motion control, image processing, and deep learning functionalities. Manual re-classification (verification) is facilitated through a drag-and-drop function on related pages.

For deep learning, the Cell i Network optimized for cell images is utilized. It is based on a database of more than three million slide images, which have been used for training and tuning to achieve optimized classification performance. UIMD’s database accounts for variations in staining, patient pathology, and examining professionals’ expertise. Additionally, a custom model can be developed by fine-tuning the base model with 200 additional slides (100 normal and 100 abnormal) that have been confirmed by users at the implementing institution. In this study, PBIA was evaluated after it was fine-tuned.

Captured WBC images were pre-classified into 13 types of cells: segmented neutrophils, banded neutrophils, metamyelocytes, myelocytes, promyelocytes, lymphocytes, abnormal lymphocytes, reactive lymphocytes, monocytes, eosinophils, basophils, blasts, and plasma cells. nRBCs, giant platelets, platelet aggregation, malaria, and artifacts were also counted and displayed separately.

DI-60

The DI-60 system is a fully integrated cell image analyzer that combines two Sysmex XN hematology analyzers, namely, the Sysmex SP-10 slide maker/stainer and a CellaVision instrument. It operates using the same software and a modified camera lens system as the CellaVision DM1200 analyzer. Its performance has been thoroughly evaluated in recent studies, which showed that its WBC classification for normal and abnormal samples is reliable, and its accuracy improves after user verification [1], 6], 8].

DI-60 pre-classified 12 WBC classes: segmented neutrophils, banded neutrophils, metamyelocytes, myelocytes, promyelocytes, lymphocytes, atypical lymphocytes, monocytes, eosinophils, basophils, blasts, and plasma cells. It also separately counted and displayed nRBCs, giant platelets, platelet clumps, smudged cells, artifacts, and unidentified cells.

Study design

A total of 461 PBS slides were analyzed using PBIA and DI-60 for pre-classification. Verification was performed by two well-trained experts, yielding post-classification results. The classification process involved 14 different cell classes: segmented neutrophils, banded neutrophils, metamyelocytes, myelocytes, promyelocytes, lymphocytes, abnormal lymphocytes, reactive lymphocytes, monocytes, eosinophils, basophils, blasts, plasma cells, nRBC, and artifacts. We used each instrument’s default settings: PBIA attempted to capture about 200 cells per slide, whereas DI-60 attempted to process roughly 300 cells but displayed approximately 100 images for review. In PBIA, abnormal lymphocytes and reactive lymphocytes were categorized as atypical lymphocytes for analysis. In DI-60, smudge cells and unidentified cells were classified as artifacts, and plasma cells were classified as atypical lymphocytes during verification. Unlike PBIA, which verified plasma cells as a distinct category, DI-60 and manual WBC differential counts used our institution’s workflow, merging any suspected plasma cells into the ‘atypical lymphocyte’ unless specific circumstances – such as plasma cell leukemia – required a separate count. Giant platelets, platelet aggregation or clumps, and malaria were excluded from this analysis.

For each instrument, the pre-classification performance, including accuracy, false positive rate (FPR), and false negative rate (FNR), was calculated on the basis of post-classification results, and the agreement between pre- and post-classification was assessed. The pre-classification performance for both instruments was evaluated by categorizing WBC counts into five ranges: <1.00 × 109/L, 1.00–2.00 × 109/L, 2.00–4.00 × 109/L, 4.00–10.00 × 109/L, and >10.00 × 109/L. The pre-classification performance of both instruments was evaluated when key abnormal cells, such as blasts and atypical lymphocytes, were present in manual counts.

Manual WBC differential counts were used as the reference, and the WBC differentials from pre- and post-classification results were compared. In this study, manual counts did not include plasma cells and artifacts; as such, they were excluded from the analysis when comparing manual counts with the results from each instrument. Therefore, the following cell types were examined: segmented neutrophils, banded neutrophils, metamyelocytes, myelocytes, promyelocytes, lymphocytes, atypical lymphocytes, monocytes, eosinophils, basophils, blasts, and nRBCs. For nRBCs in PBIA and DI-60, the counts were converted to numbers per 100 WBCs to match the units used in the manual count. The mean differences in each cell class were calculated for both instruments. The correlation between pre- and post-classification with manual counts was determined for each cell type. Differences in the correlation between pre- and post-classification within the same instrument were also examined. Passing–Bablok regression was performed for six WBC types (neutrophils, lymphocytes, monocytes, eosinophils, basophils, and blasts) to compare both pre- and post-classification with manual counts. Results were visualized.

In cases where abnormal cells such as blasts, atypical lymphocytes, and nRBCs were observed in the manual count, the ability of each instrument to detect these cells was evaluated. The accuracy, FPR, and FNR of each instrument were calculated and compared accordingly.

Statistical analysis

Data were analyzed using R (version 4.2.1; R Core Team, 2022). All tests were two sided with p<0.05 considered significant. Categorical data were examined using Fisher’s exact test. The agreement between pre- and post-classification was assessed with Cohen’s kappa value (κ) interpreted as follows: κ≤0.2, poor; 0.2<κ≤0.4, fair; 0.4<κ≤0.6, moderate; 0.6<κ≤0.8, substantial; and κ>0.8: good [15]. The mean differences between WBC differentials from pre-classification, post-classification, and manual counts were calculated via a paired t-test. For correlation analysis, Pearson’s correlation coefficients (r) were calculated and interpreted as follows: <0.30, negligible; 0.30–0.50, low; 0.50–0.70, moderate; 0.70–0.90, high; and 0.90–1.00: very high [16]. The Z-difference method was used to compare these correlations. Passing–Bablok regression was performed using the R package “mcr” (version 1.3.3). Results were provided for neutrophils, lymphocytes, monocytes, eosinophils, basophils, and blasts, where the slope was between 0.1 and 100, comparing post-classification WBC differentials with manual counts.

Results

Pre-classification performance

Table 1 shows the pre-classification performance of various cell classes based on post-classification results for PBIA and DI-60. Figure 1 presents the confusion matrices comparing the pre- and post-classification results for PBIA and DI-60. A total of 119,632 and 82,078 cells were analyzed by PBIA and DI-60, respectively. In PBIA, promyelocytes were excluded from the analysis because only two cells were identified. In DI-60, plasma cells were classified as atypical lymphocytes during verification; consequently, analyzing the pre-classification performance for plasma cells was difficult. Therefore, they were excluded from the analysis.

Table 1:

Performance of WBC pre-classification based on the post-classification for PBIA and DI-60.

PBIA (Total cell count=119,632) DI-60 (Total cell count=82,078)
Cell class Cell count Accuracy % (95 % CI) FPR % (95 % CI) FNR % (95 % CI) Kappa (95 % CI) Cell count Accuracy % (95 % CI) FPR % (95 % CI) FNR % (95 % CI) Kappa (95 % CI)
NES 42,603 99.0 (99.0–99.1) 1.1 (1.1–1.2) 0.6 (0.6–0.7) 0.979 (0.978–0.980) 19,707 93.3 (93.2–93.5) 3.2 (3.0–3.3) 16.7 (16.2–17.2) 0.822 (0.817–0.826)
NEB 1,898 99.0 (99.0–99.1) 0.5 (0.5–0.6) 30.6 (28.5–32.7) 0.674 (0.657–0.692) 3,738 95.8 (95.7–96) 3.8 (3.7–4.0) 33.9 (31.0–36.9) 0.259 (0.243–0.276)
ME 883 99.6 (99.5–99.6) 0.3 (0.3–0.3) 25.9 (22.7–29) 0.669 (0.643–0.696) 432 99.3 (99.3–99.4) 0.5 (0.4–0.5) 75.7 (70.3–81.2) 0.168 (0.129–0.207)
MY 700 99.7 (99.6–99.7) 0.2 (0.2–0.2) 27.0 (23.6–30.3) 0.709 (0.682–0.737) 1,093 98.7 (98.7–98.8) 1.2 (1.1–1.3) 34.5 (27.3–41.7) 0.172 (0.143–0.200)
PR 2 1,387 97.7 (97.6–97.8) 1.4 (1.4–1.5) 77.2 (74.5–79.8) 0.178 (0.156–0.199)
LY 22,499 98.1 (98.1–98.2) 1.3 (1.2–1.4) 4.4 (4.1–4.7) 0.938 (0.936–0.941) 9,405 91.5 (91.3–91.7) 3.6 (3.5–3.8) 39.4 (38.5–40.3) 0.613 (0.605–0.621)
AT 210 98.9 (98.8–98.9) 0.1 (0.1–0.1) 92.1 (90.7–93.6) 0.134 (0.111–0.157) 1,633 97.9 (97.8–98.0) 1.9 (1.8–2.0) 65.0 (59.3–70.8) 0.092 (0.074–0.110)
MO 8,563 98.8 (98.8–98.9) 0.9 (0.8–0.9) 5.6 (5.1–6.1) 0.909 (0.904–0.914) 1,883 96.3 (96.1–96.4) 0.5 (0.4–0.5) 64.0 (62.5–65.4) 0.482 (0.466–0.497)
EO 1,652 100.0 (100.0–100.0) 0.0 (0.0–0.0) 0.4 (0.1–0.7) 0.994 (0.991–0.996) 493 99.6 (99.5–99.6) 0.1 (0.1–0.1) 39.1 (35.4–42.8) 0.700 (0.669–0.730)
BA 776 99.9 (99.9–99.9) 0.1 (0.1–0.1) 4.0 (2.5–5.4) 0.916 (0.901–0.931) 1,390 98.6 (98.5–98.7) 1.4 (1.3–1.4) 12.8 (9.2–16.5) 0.322 (0.293–0.351)
BL 6,881 98.3 (98.3–98.4) 0.4 (0.3–0.4) 19.8 (18.9–20.6) 0.858 (0.852–0.864) 1,548 95.7 (95.5–95.8) 0.4 (0.4–0.5) 72.7 (71.4–74) 0.387 (0.371–0.403)
PC 242 99.8 (99.8–99.9) 0.2 (0.1–0.2) 6.7 (0.4–13) 0.370 (0.301–0.440) 388
NR 1,493 99.7 (99.6–99.7) 0.2 (0.2–0.3) 8.1 (6.7–9.6) 0.855 (0.841–0.869) 4,397 96.2 (96.1–96.3) 3.8 (3.7–3.9) 4.3 (3.3–5.4) 0.449 (0.433–0.465)
AR 31,230 99.2 (99.1–99.2) 0.7 (0.6–0.7) 1.3 (1.2–1.4) 0.978 (0.977–0.979) 34,584 85.4 (85.2–85.7) 11.6 (11.3–11.9) 18.4 (18.0–18.8) 0.702 (0.697–0.707)
  1. AR, artifact; AT, atypical lymphocyte; BA, basophil; BL, blast; CI, confidence interval; EO, eosinophil; FNR, false negative rate; FPR, false positive rate; LY, lymphocyte; ME, metamyelocyte; MO, monocyte; MY, myelocyte; NEB, neutrophil banded; NES, neutrophil segmented; NR, nucleated red blood cell; PC, plasma cell; PR, promyelocyte.

Figure 1: 
Confusion matrices comparing the pre- and post-classification performance of (A) PBIA and (B) DI-60. Columns and rows represent pre- and post-classification counts, respectively, by cell class. Color indicates the percentage of post-classification counts based on pre-classification counts for each cell class. In DI-60, plasma cells are excluded from post-classification; as a result, all post-classification counts for plasma cells are equivalent to zero. AR, artifact; AT, atypical lymphocyte; BA, basophil; BL, blast; EO, eosinophil; LY, lymphocyte; ME, metamyelocyte; MO, monocyte; MY, myelocyte; NEB, neutrophil banded; NES, neutrophil segmented; NR, nucleated red blood cell; PC, plasma cell; PR, promyelocyte.
Figure 1:

Confusion matrices comparing the pre- and post-classification performance of (A) PBIA and (B) DI-60. Columns and rows represent pre- and post-classification counts, respectively, by cell class. Color indicates the percentage of post-classification counts based on pre-classification counts for each cell class. In DI-60, plasma cells are excluded from post-classification; as a result, all post-classification counts for plasma cells are equivalent to zero. AR, artifact; AT, atypical lymphocyte; BA, basophil; BL, blast; EO, eosinophil; LY, lymphocyte; ME, metamyelocyte; MO, monocyte; MY, myelocyte; NEB, neutrophil banded; NES, neutrophil segmented; NR, nucleated red blood cell; PC, plasma cell; PR, promyelocyte.

Overall, the pre-classification performance of PBIA was better than that of DI-60, with higher accuracy, lower FPR and FNR, and stronger agreement. PBIA outperformed DI-60 in most comparisons (p<0.001) except in banded neutrophils (p=0.073) and myelocytes (p=0.056), where no significant differences in FNR were observed. For atypical lymphocytes, PBIA had a significantly higher FNR of 92.1 % (95 % confidence interval: 90.7–93.6) than the FNR of DI-60, which was 65.0 % (59.3–70.8; p<0.001). The pre-classification accuracies based on post-classification results for all cell classes were 90.0 % (95 % confidence interval: 89.8–90.1) for PBIA and 45.5 % (45.2–45.9) for DI-60. All cell classes had κ of 0.934 (0.933–0.936) for PBIA and 0.629 (0.625–0.633) for DI-60. For major cell classes such as segmented neutrophils, lymphocytes, monocytes, eosinophils, and basophils, PBIA demonstrated an accuracy of 98.1–100.0 %, FPR of 0.0–1.3 %, and FNR of 0.4–5.6 %. Its κ ranged from 0.909 to 0.994, indicating good agreement. PBIA also outperformed DI-60 for blasts, with κ indicating good agreement. Both instruments showed a generally poor pre-classification performance on immature granulocytes (promyelocytes, myelocytes, and metamyelocytes) and banded neutrophils. For artifacts, PBIA had lower misclassification rates and stronger agreement than DI-60.

For both instruments, when WBC counts were divided into five ranges (<1.00 × 109/L, 1.00–2.00 × 109/L, 2.00–4.00 × 109/L, 4.00–10.00 × 109/L, and >10.00 × 109/L), variability in the pre-classification performance was observed. The performance was better in leukopenic cases, but tended to decrease as WBC counts increased (Supplemental Table 1). In addition, the pre-classification performance of PBIA and DI-60 was analyzed for 114 PBS slides where blasts were identified in the manual count and for 7 PBS slides where atypical lymphocytes were present in numbers greater than 20 in the manual count; these specimens were from patients previously diagnosed with lymphoma (Supplemental Table 2). The FPR of the cell classes that could potentially be confused with blasts, such as lymphocytes, atypical lymphocytes, monocytes, myelocytes, and artifacts, tended to increase compared with the overall results for both instruments. Specifically, in the 121 slides with abnormal cells, the FPR of lymphocytes (3.1 vs. 1.3 %), atypical lymphocytes (0.2 vs. 0.1 %), monocytes (2.6 vs. 0.9 %), myelocytes (0.5 vs. 0.2 %), and artifacts (2.0 vs. 0.7 %) in PBIA increased compared with that in the overall group. Similarly, the FPR of lymphocytes (8.0 vs. 3.6 %), atypical lymphocytes (3.2 vs. 1.9 %), monocytes (1.5 vs. 0.5 %), myelocytes (2.7 vs. 1.2 %), and artifacts (17.0 vs. 11.6 %) in DI-60 showed an increase. The FPR in PBIA was consistently lower than that in DI-60 across all relevant cell classes. However, for atypical lymphocytes, PBIA showed a significantly higher FNR of 93.0 % (91.2–94.8), while DI-60 showed an FNR of 68.5 % (59.8–77.3; p<0.001).

Comparison with manual WBC differentials

Table 2 presents the median manual WBC differential counts for each cell class and, for each instrument, the mean differences between those manual counts and both the pre- and post-classification WBC differentials. In PBIA, the pre-classification results did not significantly differ from the manual counts for eosinophils and nRBCs. The post-classification results did not significantly vary for atypical lymphocytes, eosinophils, and nRBCs. In DI-60, the pre-classification results for segmented neutrophils did not significantly differ from those for the manual counts. The post-classification results showed no significant differences in segmented neutrophils, myelocytes, lymphocytes, atypical lymphocytes, and eosinophils.

Table 2:

Comparison of WBC differential counts between manual differential count, PBIA, and DI-60.

Median, % [IQR] Mean difference, % (95 % CI)
PBIA DI-60
Cell class Manual Pre-classification vs. Manual p-Value Post-classification vs. Manual p-Value Pre-classification vs. Manual p-Value Post-classification vs. Manual p-Value
NES 45.0 [24.0–63.0] 2.8 (1.9–3.6) <0.001 2.0 (1.2–2.8) <0.001 −0.3 (−1.4 to 0.8) 0.556 0.8 (−0.2 to 1.7) 0.110
NEB 0.0 [0.0–0.0] 1.3 (1.0–1.7) <0.001 1.6 (1.1–2.0) <0.001 7.0 (6.2–7.8) <0.001 1.5 (1.1–2.0) <0.001
ME 0.0 [0.0–0.0] 0.8 (0.6–1.0) <0.001 0.6 (0.4–0.7) <0.001 0.9 (0.7–1.1) <0.001 0.3 (0.–0.5) <0.001
MY 0.0 [0.0–0.0] 0.3 (0.2–0.4) <0.001 0.3 (0.1–0.4) <0.001 2.1 (1.8–2.4) <0.001 −0.1 (−0.3 to 0.1) 0.387
PR 0.0 [0.0–0.0] 0.0 (0.0–0.0) 0.042 0.0 (0.0–0.0) 0.049 2.9 (2.2–3.5) <0.001 1.5 (0.5–2.4) 0.002
LY 25.0 [9.0–43.0] −1.4 (−2.3 to −0.4) 0.005 −1.7 (−2.4 to −1.0) <0.001 −3.7 (−5.2 to −2.2) <0.001 −0.3 (−1.5 to 0.8) 0.583
AT 0.0 [0.0–0.0] −1.2 (−2.0 to −0.4) 0.005 0.1 (−0.3 to 0.5) 0.498 2.5 (1.5 to 3.4) <0.001 −0.7 (−1.6 to 0.2) 0.115
MO 8.0 [3.0–15.0] −1.3 (−2.0 to −0.6) <0.001 −1.8 (−2.4 to −1.2) <0.001 −6.8 (−7.6 to −6.0) <0.001 −1.9 (−2.5 to −1.3) <0.001
EO 1.0 [0.0–3.0] 0.0 (−0.1 to 0.2) 0.664 0.0 (−0.1 to 0.2) 0.752 −0.5 (−0.8 to −0.3) <0.001 −0.1 (−0.4 to 0.2) 0.433
BA 0.0 [0.0–1.0] 0.2 (0.7–1.1) <0.001 0.2 (0.7–1.0) 0.010 2.8 (2.4–3.1) <0.001 0.2 (0.0–0.3) 0.023
BL 0.0 [0.0–0.0] −1.9 (−2.7 to −1.1) <0.001 −1.0 (−1.5 to −0.5) <0.001 −6.8 (−8.6 to −5) <0.001 −2.3 (−3.4 to −1.2) <0.001
NR 0.0 [0.0–0.0] −0.5 (−1.2 to 0.1) 0.126 −0.3 (−1.0 to 0.4) 0.387 21.2 (15–27.3) <0.001 2.4 (0.9–3.9) 0.002
  1. AT, atypical lymphocyte; BA, basophil; BL, blast; CI, confidence interval; EO, eosinophil; IQR, Interquartile range; LY, lymphocyte; ME, metamyelocyte; MO, monocyte; MY, myelocyte; NEB, neutrophil banded; NES, neutrophil segmented; NR, nucleated red blood cell; PR, promyelocyte.

Table 3 presents the comparison of the correlation of pre- and post-classification WBC differentials with manual counts for each instrument. The z-difference was used to assess whether the correlation results significantly differed between the pre- and post-classification. For major cell classes (segmented neutrophils, lymphocytes, monocytes, eosinophils, and basophils), PBIA showed high to very high correlation in post-classification; conversely, the correlation between pre- and post-classification for neutrophils, eosinophils, and basophils did not significantly vary. In DI-60, the post-classification results showed high to very high correlation for neutrophils, lymphocytes, and monocytes, while eosinophils and basophils showed moderate correlation. In DI-60, the correlation of pre-classification with manual counts was lower than that of post-classification.

Table 3:

Comparison of WBC differential correlation between manual differential count, PBIA, and DI-60.

PBIA DI-60
Pre-classification vs. Manual Post-classification vs. Manual Z-difference p-Value Pre-classification vs. Manual Post-classification vs. Manual Z-difference p-Value
Cell class r p-Value r p-Value r p-Value r p-Value
NES 0.944 <0.001 0.940 <0.001 −0.494 0.621 0.882 <0.001 0.925 <0.001 3.568 <0.001
NEB 0.460 <0.001 0.351 <0.001 −1.984 0.047 0.101 0.030 0.484 <0.001 6.445 <0.001
ME 0.532 <0.001 0.526 <0.001 −0.129 0.897 0.168 <0.001 0.371 <0.001 3.321 0.001
MY 0.695 <0.001 0.717 <0.001 0.655 0.512 0.262 <0.001 0.325 <0.001 1.050 0.294
PR −0.007 0.878 −0.007 0.880 0.002 0.999 0.007 0.879 0.002 0.971 −0.082 0.935
LY 0.886 <0.001 0.935 <0.001 4.449 <0.001 0.692 <0.001 0.840 <0.001 5.582 <0.001
AT 0.456 <0.001 0.887 <0.001 13.823 <0.001 −0.024 0.609 0.037 0.429 0.921 0.357
MO 0.775 <0.001 0.848 <0.001 3.270 0.001 0.681 <0.001 0.842 <0.001 5.993 <0.001
EO 0.875 <0.001 0.872 <0.001 −0.193 0.847 0.624 <0.001 0.660 <0.001 0.923 0.356
BA 0.696 <0.001 0.713 <0.001 0.526 0.599 0.230 <0.001 0.627 <0.001 7.610 <0.001
BL 0.958 <0.001 0.975 <0.001 4.056 <0.001 0.865 <0.001 0.884 <0.001 1.208 0.227
NR 0.905 <0.001 0.915 <0.001 0.930 0.352 0.322 <0.001 0.842 <0.001 13.559 <0.001
  1. AT, atypical lymphocyte; BA, basophil; BL, blast; EO, eosinophil; LY, lymphocyte; ME, metamyelocyte; MO, monocyte; MY, myelocyte; NEB, neutrophil banded; NES, neutrophil segmented; NR, nucleated red blood cell; PR, promyelocyte; r, Pearson’s correlation coefficient.

For blasts, PBIA showed a very high correlation with manual counts in both pre- (r=0.905) and post-classification (r=0.915). In DI-60, the correlation coefficients for blasts were 0.322 for pre-classification (low correlation) and 0.842 for post-classification (high correlation). For atypical lymphocytes, PBIA showed a low correlation in pre-classification (r=0.456) and a high correlation in post-classification (r=0.887). Conversely, DI-60 did not show a significant correlation for atypical lymphocytes. Regarding nRBCs, PBIA demonstrated a very high correlation in pre- and post-classification (r=0.905 and r=0.915, respectively). For DI-60, pre- and post-classification had r of 0.865 and 0.884, respectively. Both PBIA and DI-60 exhibited relatively lower correlation for immature granulocytes compared with major cell classes.

The results of Passing–Bablok regression for the major cell classes and blasts are presented in Figure 2. The scatter plots with r for the remaining cell classes are provided in Supplemental Figure 1.

Figure 2: 
Passing–Bablok regression analysis comparing pre- and post-classification percent counts with manual differential counts. Cell classes are shown on the left side of the figure, and the corresponding variables for the x- and y-axes of each graph are displayed at the top. The Passing–Bablok regression line, the corresponding regression equation, and the correlation coefficient (r) are provided in each graph. The regression for the DI-60 pre-classification of basophils is not provided. BA, basophil; BL, blast; EO, eosinophil; LY, lymphocyte; MO, monocyte; NES, neutrophil segmented.
Figure 2:

Passing–Bablok regression analysis comparing pre- and post-classification percent counts with manual differential counts. Cell classes are shown on the left side of the figure, and the corresponding variables for the x- and y-axes of each graph are displayed at the top. The Passing–Bablok regression line, the corresponding regression equation, and the correlation coefficient (r) are provided in each graph. The regression for the DI-60 pre-classification of basophils is not provided. BA, basophil; BL, blast; EO, eosinophil; LY, lymphocyte; MO, monocyte; NES, neutrophil segmented.

Ability to identify abnormal cells

We compared the ability of PBIA and DI-60 to identify key abnormal cells when at least one was observed in the manual WBC differential count (Table 4). PBIA produced more false positive results for blasts, with an FPR of 30.5 % (25.9–35.6), while DI-60 recorded a lower FPR of 2.3 % (1.2–4.5). Conversely, DI-60 had a higher FNR of 42.1 % (33.4–51.3) than PBIA (6.1 %, 3.0–12.1; p<0.001 for all). For atypical lymphocytes and nRBCs, PBIA showed higher accuracy and lower FPR than DI-60, although their FNR did not significantly differ. Atypical lymphocytes exhibited high FNRs in both instruments; specifically, PBIA and DI-60 had FNR of 50.0 % (32.6–67.4) and 57.1 % (39.1–73.5), respectively.

Table 4:

Performance for identifying abnormal cells in post-classification when present in manual differential count.

PBIA post-classification DI-60 post-classification Fisher’s exact test, p-value
Cell class Accuracy % (95 % CI) FPR % (95 % CI) FNR % (95 % CI) Accuracy % (95 % CI) FPR % (95 % CI) FNR % (95 % CI) Accuracy FPR FNR
BL 75.5 (71.4–79.2) 30.5 (25.9–35.6) 6.1 (3.0–12.1) 87.9 (84.6–90.5) 2.3 (1.2–4.5) 42.1 (33.4–51.3) <0.001 <0.001 <0.001
AT 83.5 (79.9–86.6) 14.3 (11.3–17.9) 50.0 (32.6–67.4) 75.1 (70.9–78.8) 22.9 (19.2–27.0) 57.1 (39.1–73.5) 0.002 0.002 0.789
NR 60.3 (55.8–64.7) 44.8 (39.9–49.8) 15.2 (8.9–24.7) 79.2 (75.2–82.6) 22.8 (18.9–27.2) 11.4 (6.1–20.3) <0.001 <0.001 0.640
  1. AT, atypical lymphocyte; BL, blast; CI, confidence interval; FNR, false negative rate; FPR, false positive rate; NR, nucleated red blood cell.

Discussion

DI-60 provides an acceptable WBC classification performance for normal and abnormal samples, with overall pre-classification accuracy reaching 86–90 % and correlates well with manual WBC differential counts after user verification [1], 6], 8], 17]. These previous results differ from our findings possibly because of variations in the sample types analyzed and the potential differences in staining protocols [1], 17], 18]. Notably, a previous study showed that DI-60 performance could depend on the staining method and proportion of abnormal samples [17]. Our use of a different Wright–Giemsa stain and the likely higher proportion of abnormal cases in our real-world sample may both have contributed to the observed discrepancies [8], 17].

To the best of our knowledge, only two studies have reported on the performance of PBIA WBC classification [10], 11]. One study reported a high pre-classification accuracy of 97 % for overall WBC counts and approximately 94 % for blasts. Its accuracy (81 %) for abnormal lymphocytes is lower than that for other cell classes although reactive lymphocytes are separately classified, achieving a higher accuracy of 94 %. For nRBCs, the pre-classification accuracy of PBIA is approximately 96 % [10]. The other study demonstrated over 90 % accuracy for all cell types except atypical lymphocytes in samples from leukopenic patients [11].

Under the conditions of this study, PBIA outperformed DI-60 in pre-classification and showed stronger correlation with manual counts, resulting in fewer discrepancies between pre- and post-classification. However, for specific cell classes, such as atypical lymphocytes, PBIA showed a lower pre-classification performance than DI-60 did. In addition, the performance of PBIA for immature granulocytes and banded neutrophils remained poor. One possible explanation is the inherent subjectivity in classifying these transitional forms, leading to substantial inter-observer variability [1], 18]. Although verification reduced differences in WBC differentials for some cell classes, improvements varied by class. Therefore, users should understand which cell classes might be prone to misclassification when using PBIA, and they should take additional care during verification.

The presence of abnormal cells may reduce pre-classification performance [8], 18], 19], as our findings also indicated, underscoring the need for verification [6]. However, PBIA performed better under these conditions, potentially helping users verify images more effectively. In DI-60, various cell classes, including blasts, were more frequently misclassified as artifacts, demanding greater vigilance during reclassification.

When evaluating abnormal cell detection, DI-60 showed a better accuracy for blasts and nRBCs in terms of manual count results, whereas PBIA performed better for atypical lymphocytes. Notably, PBIA produced more false positives for blasts, whereas DI-60 had a higher false negative rate. These findings suggested that manual inspection is recommended for PBIA when blasts are observed in post-classification results. This aligns with the International Council for Standardization of Hematology recommendations for digital morphology analyzers, which specify that blasts require manual review [1]. Both instruments showed high false negative rates for atypical lymphocytes. Therefore, digital morphology analyzers alone may be insufficient for detecting these cells.

A previous study demonstrated that digital morphology analyzers can exhibit reduced pre-classification performance in leukopenia. Notably, it showed that leukopenia groups likely contained more abnormal cells [14]. Conversely, more blasts and atypical lymphocytes in the groups with higher WBC counts were observed in our study. This different distribution of abnormal cells, as noted above, potentially influenced the pre-classification results in our study [8], 18], 19].

PBIA exhibits a strong WBC classification performance possibly because of deep learning algorithms, which are based on a vast WBC image database, and the ability of users to fine-tune the base model. This adjustment may also help compensate for variations in staining methods [1], 9], 20]. Although it improves the alignment between pre- and post-classification, it may introduce a risk of bias [1]. Therefore, after the base model is fine-tuned, a comparison study with manual counts is recommended. Multicenter studies should also be performed to further assess the overall effect on the performance of PBIA.

Morphological identification inherently involves an aspect of subjectivity and requires experienced technologists [1], 9], 18]. We believe that PBIA, with its high accuracy, can assist less experienced users in reducing missed diagnoses of abnormal cells, especially clinically significant blasts [9]. Additionally, PBIA is expected to support morphologists in achieving faster cell classification [9], 11], although this study does not specifically address time efficiency.

This study has several limitations. First, although PBS samples were randomly selected from our routine workload, we neither stratified healthy vs. patient groups nor met the recommendation of Clinical and Laboratory Standards Institute H20-A2 guideline [5]. This may have created a skewed distribution of certain pathologic cells, potentially affecting the overall performance metrics for each analyzer [8], 18], 19]. Nevertheless, we believe that this provides insight into how these instruments perform in everyday practice. Second, although we compared both instruments by using the same set of samples, their performance might still differ because of sample-specific factors, particularly for DI-60, which is influenced by slide quality and staining methods; therefore, performance variations may occur under different conditions [1], 17], 18]. Third, the difference in cell count analyzed may influence performance comparisons. Fourth, this study could not fully separate the effect of abnormal cells on performance across different WBC counts. Fifth, analyzing the performance for low-count cell types, such as promyelocytes, was challenging. Sixth, comparison of plasma cells was limited. Lastly, we did not evaluate reproducibility or throughput of both instruments.

In conclusion, our study demonstrates that PBIA may offer a better pre-classification performance than DI-60 in this research setting; consequently, users may easily verify results in a clinical laboratory. Since PBIA can be fine-tuned, cell classes with significant differences from manual counts and any changes in overall performance should be evaluated prior to implementation. Multicenter studies should also be conducted to further explore and validate the WBC classification performance of PBIA.


Corresponding author: Bohyun Kim, Department of Laboratory Medicine, Soonchunhyang University Cheonan Hospital, Soonchunhyang University College of Medicine, 31, Soonchunhyang 6-gil, Dongnam-gu, Cheonan-si, Chungcheongnam-do, Gyeonggi-Do 31151, Republic of Korea, E-mail:
Hongkyung Kim and Oh Joo Kweon contributed equally to this work and share first authorship.
  1. Research ethics: The study was conducted in accordance with the Declaration of Helsinki, and the Institutional Review Board/Ethics Committee of Soonchunhyang University College of Medicine, Cheonan, Republic of Korea (approval no. 2024-02-009).

  2. Informed consent: Not applicable.

  3. Author contributions: The authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  4. Use of Large Language Models, AI and Machine Learning Tools: None declared.

  5. Conflict of interest: The authors state no conflict of interest.

  6. Research funding: This work was supported by Soonchunhyang University Research Fund.

  7. Data availability: The raw data can be obtained on request from the corresponding author.

References

1. Kratz, A, Lee, SH, Zini, G, Riedl, JA, Hur, M, Machin, S. Digital morphology analyzers in hematology: ICSH review and recommendations. Int J Lab Hematol 2019;41:437–47. https://doi.org/10.1111/ijlh.13042.Search in Google Scholar PubMed

2. Palmer, L, Briggs, C, McFadden, S, Zini, G, Burthem, J, Rozenberg, G, et al.. ICSH recommendations for the standardization of nomenclature and grading of peripheral blood cell morphological features. Int J Lab Hematol 2015;37:287–303. https://doi.org/10.1111/ijlh.12327.Search in Google Scholar PubMed

3. Bain, BJ. Diagnosis from the blood smear. N Engl J Med 2005;353:498–507. https://doi.org/10.1056/nejmra043442.Search in Google Scholar

4. Gulati, G, Song, J, Florea, AD, Gong, J. Purpose and criteria for blood smear scan, blood smear examination, and blood smear review. Ann Lab Med 2013;33:1–7. https://doi.org/10.3343/alm.2013.33.1.1.Search in Google Scholar PubMed PubMed Central

5. Clinical and Laboratory Standards Institute (CLSI). Reference leukocytes (WBC) differential count (proportional) and evaluation of instrumental methods; approved standard, 2nd ed. Wayne, PA, USA: CLSI Document H20-A2. CLSI; 2007.Search in Google Scholar

6. Tabe, Y, Yamamoto, T, Maenou, I, Nakai, R, Idei, M, Horii, T, et al.. Performance evaluation of the digital cell imaging analyzer DI-60 integrated into the fully automated Sysmex XN hematology analyzer system. Clin Chem Lab Med 2015;53:281–9. https://doi.org/10.1515/cclm-2014-0445.Search in Google Scholar PubMed

7. Da Costa, L. Digital image analysis of blood cells. Clin Lab Med 2015;35:105–22. https://doi.org/10.1016/j.cll.2014.10.005.Search in Google Scholar PubMed

8. Kim, HN, Hur, M, Kim, H, Kim, SW, Moon, HW, Yun, YM. Performance of automated digital cell imaging analyzer Sysmex DI-60. Clin Chem Lab Med 2017;56:94–102. https://doi.org/10.1515/cclm-2017-0132.Search in Google Scholar PubMed

9. Xing, Y, Liu, X, Dai, J, Ge, X, Wang, Q, Hu, Z, et al.. Artificial intelligence of digital morphology analyzers improves the efficiency of manual leukocyte differentiation of peripheral blood. BMC Med Inf Decis Making 2023;23:50. https://doi.org/10.1186/s12911-023-02153-z.Search in Google Scholar PubMed PubMed Central

10. Lee, J-M, Kim, M, Han, K, Kim, Y. Evaluation of a digital image analyzer UIMD PBIA for determining white blood cell differential count. Lab Med Online 2023;13:7–12. https://doi.org/10.47429/lmo.2023.13.1.7.Search in Google Scholar

11. Kim, HW, You, E, Lee, JY. Assessment of clinical utility of digital morphology analyzer uimd pbia in leukopenia samples. Lab Med Online 2024;14:321–9. https://doi.org/10.47429/lmo.2024.14.4.321.Search in Google Scholar

12. Leung, E, Johnston, A, Olsen, B, Chang, H, Martin, T, Wozniak, M, et al.. Laboratory practices for manual blood film review: results of an IQMH patterns of practice survey. Int J Lab Hematol 2021;43:184–90. https://doi.org/10.1111/ijlh.13343.Search in Google Scholar PubMed

13. Kweon, OJ, Lim, YK, Lee, MK, Kim, HR. Red and white blood cell morphology characterization and hands-on time analysis by the digital cell imaging analyzer DI-60. PLoS One 2022;17:e0267638. https://doi.org/10.1371/journal.pone.0267638.Search in Google Scholar PubMed PubMed Central

14. Yoon, S, Hur, M, Lee, GH, Nam, M, Kim, H. How reproducible is the data from sysmex Di-60 in leukopenic samples? Diagnostics 2021;11. https://doi.org/10.3390/diagnostics11122173.Search in Google Scholar PubMed PubMed Central

15. Landis, JR, Koch, GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. https://doi.org/10.2307/2529310.Search in Google Scholar

16. Mukaka, MM. Statistics corner: a guide to appropriate use of correlation coefficient in medical research. Malawi Med J 2012;24:69–71.Search in Google Scholar

17. Kim, HN, Hur, M, Kim, H, Park, M, Kim, SW, Moon, HW, et al.. Comparison of three staining methods in the automated digital cell imaging analyzer Sysmex DI-60. Clin Chem Lab Med 2018;56:e280–3. https://doi.org/10.1515/cclm-2018-0539.Search in Google Scholar PubMed

18. Rollins-Raval, MA, Raval, JS, Contis, L. Experience with CellaVision DM96 for peripheral blood differentials in a large multi-center academic hospital system. J Pathol Inf 2012;3:29. https://doi.org/10.4103/2153-3539.100154.Search in Google Scholar PubMed PubMed Central

19. Lee, LH, Mansoor, A, Wood, B, Nelson, H, Higa, D, Naugler, C. Performance of CellaVision DM96 in leukocyte classification. J Pathol Inf 2013;4:14. https://doi.org/10.4103/2153-3539.114205.Search in Google Scholar PubMed PubMed Central

20. Lincz, LF, Makhija, K, Attalla, K, Scorgie, FE, Enjeti, AK, Prasad, R. A comparative evaluation of three consecutive artificial intelligence algorithms released by Techcyte for identification of blasts and white blood cells in abnormal peripheral blood films. Int J Lab Hematol 2024;46:92–8. https://doi.org/10.1111/ijlh.14180.Search in Google Scholar PubMed


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/cclm-2024-1323).


Received: 2024-11-12
Accepted: 2025-01-13
Published Online: 2025-01-23
Published in Print: 2025-06-26

© 2025 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Articles in the same Issue

  1. Frontmatter
  2. Editorials
  3. The journey to pre-analytical quality
  4. Manual tilt tube method for prothrombin time: a commentary on contemporary relevance
  5. Reviews
  6. From errors to excellence: the pre-analytical journey to improved quality in diagnostics. A scoping review
  7. Advancements and challenges in high-sensitivity cardiac troponin assays: diagnostic, pathophysiological, and clinical perspectives
  8. Opinion Paper
  9. Is it feasible for European laboratories to use SI units in reporting results?
  10. Perspectives
  11. What does cancer screening have to do with tomato growing?
  12. Computer simulation approaches to evaluate the interaction between analytical performance characteristics and clinical (mis)classification: a complementary tool for setting indirect outcome-based analytical performance specifications
  13. Genetics and Molecular Diagnostics
  14. Artificial base mismatches-mediated PCR (ABM-PCR) for detecting clinically relevant single-base mutations
  15. Candidate Reference Measurement Procedures and Materials
  16. Antiphospholipid IgG Certified Reference Material ERM®-DA477/IFCC: a tool for aPL harmonization?
  17. General Clinical Chemistry and Laboratory Medicine
  18. External quality assessment of the manual tilt tube technique for prothrombin time testing: a report from the IFCC-SSC/ISTH Working Group on the Standardization of PT/INR
  19. Simple steps to achieve harmonisation and standardisation of dried blood spot phenylalanine measurements and facilitate consistent management of patients with phenylketonuria
  20. Inclusion of pyridoxine dependent epilepsy in expanded newborn screening programs by tandem mass spectrometry: set up of first and second tier tests
  21. Analytical performance evaluation and optimization of serum 25(OH)D LC-MS/MS measurement
  22. Towards routine high-throughput analysis of fecal bile acids: validation of an enzymatic cycling method for the quantification of total bile acids in human stool samples on fully automated clinical chemistry analyzers
  23. Analytical and clinical evaluations of Snibe Maglumi® S100B assay
  24. Prevalence and detection of citrate contamination in clinical laboratory
  25. Reference Values and Biological Variations
  26. Temporal dynamics in laboratory medicine: cosinor analysis and real-world data (RWD) approaches to population chronobiology
  27. Establishing sex- and age-related reference intervals of serum glial fibrillary acid protein measured by the fully automated lumipulse system
  28. Hematology and Coagulation
  29. Performance of the automated digital cell image analyzer UIMD PBIA in white blood cell classification: a comparative study with sysmex DI-60
  30. Cancer Diagnostics
  31. Flow-cytometric MRD detection in pediatric T-ALL: a multicenter AIEOP-BFM consensus-based guided standardized approach
  32. Impact of biological and genetic features of leukemic cells on the occurrence of “shark fins” in the WPC channel scattergrams of the Sysmex XN hematology analyzers in patients with chronic lymphocytic leukemia
  33. Assessing the clinical applicability of dimensionality reduction algorithms in flow cytometry for hematologic malignancies
  34. Cardiovascular Diseases
  35. Evaluation of sex-specific 0-h high-sensitivity cardiac troponin T thresholds for the risk stratification of non-ST-segment elevation myocardial infarction
  36. Retraction
  37. The first case of Teclistamab interference with serum electrophoresis and immunofixation
  38. Letters to the Editor
  39. Is this quantitative test fit-for-purpose?
  40. Reply to “Is this quantitative test fit-for-purpose?”
  41. Short-term biological variation of coagulation and fibrinolytic measurands
  42. The first case of Teclistamab interference with serum electrophoresis and immunofixation
  43. Imlifidase: a new interferent on serum protein electrophoresis looking as a rare plasma cell dyscrasia
  44. Research on the development of image-based Deep Learning (DL) model for serum quality recognition
  45. Interference of hypertriglyceridemia on total cholesterol assay with the new CHOL2 Abbott method on Architect analyser
  46. Congress Abstracts
  47. 10th Annual Meeting of the Austrian Society for Laboratory Medicine and Clinical Chemistry (ÖGLMKC)
Downloaded on 12.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cclm-2024-1323/html
Scroll to top button