Evaluation for serum glucose standardization in clinical laboratories of Southern China by consecutive 6 years proficiency testing based on JCTLM-recommended reference methods

Samples collection, preparation and evaluation

Two concentrations of TVP samples were prepared from leftover serum pools, which collected in the clinical laboratory of Guangdong Provincial Hospital of Chinese Medicine, with the approval of the hospital’s Ethics Committee (No. Z2017-152). All collected serum samples were tested to ensure that they were not reactive for anti-human immunodeficiency virus antibodies (anti-HIV-1, anti-HIV-2), hepatitis C virus (HCV), treponema pallidum antibody (TP), and hepatitis B surface antigen (HBsAg). The samples were pooled and thoroughly mixed after being filtered with 0.45 μm and 0.22 μm filter membrane, then they were sub-packaged into 2.0 mL cryotube vials with 0.5 mL content of each vial, and stored at −70 °C till distributed to clinical laboratories by dry ice within three months.

The homogeneity, stability and commutability of the TVP samples were evaluated according to ISO GUIDE35:2017 [8], CLSI EP30-A [9] and our previous article [10]. Briefly, 15 samples were randomly selected and tested for 3 times for homogeneity, and one-way ANOVA was used for analysis. Stability at room temperature (R.T.), 2–8 °C, −80 °C and during transportation were evaluated, and regression analysis was used for data statistics. The commutability of TVP samples was assessed in four routine systems, Roche, Beckman, Mindray, and Maccura at 2017. The verification samples and 20 individual serum samples were measured twice simultaneously by reference method and routine systems. The data was analyzed by linear regression using the least square method.

Reference and clinical measurement systems

The target values were assigned by two reference laboratories accredited by the China National Accreditation Service for Conformity Assessment (CNAS), in compliance with ISO 15195 [11] and ISO/IEC 17025 [12]. The reference measurement service for serum glucose provide by the two reference laboratories are also listed in JCTLM database [13], and their proficiency is verified annually through their participation in IFCC External Quality assessment scheme for Reference Laboratories in Laboratory Medicine (RELA, Labs No. 61 and No. 65) [14]. The traceability of the results is ensured using certified reference material NIST 917c. TVP samples were analyzed three times each day over three consecutive days each year. The average value of the two reference laboratories was calculated as the target value (Supplementary Table S1).

There were 58, 62, 54, 70, 57, and 88 clinical laboratories reported their data in this TVP program from 2017 to 2022, respectively. The ratio for laboratories using matched system was 44.9 , 38.7, 46.3, 52.9, 57.9, and 62.5 % over the six years. Biochemical analysis systems of Roche, Beckman, Mindray, Hitachi, Siemens, Abbott, Olympus, Toshiba, Johnson & Johnson, and other manufacturers were used for serum glucose measurement. In the majority of clinical laboratories, the hexokinase method and the glucose oxidase method were the two mainly used methods. Besides, the oxygen electrode method, dry chemistry method, and O-toluidine method were used by a few laboratories. The situation of the clinical measurement systems were summarized in Table 1. Each concentration of the TVP samples was determined by clinical systems for 5 times on 3 consecutive days. The result of 15 data for each concentration is required to report.

Statistical analysis for the Trueness Verification Plan

The reported data of each clinical laboratory were first inspected for outliers (truncated at±3SD). Bias between the tested mean and target value, as well as intra-laboratory precision (coefficient variation, CV) was calculated. The total error (TE) was calculated as |bias|+1.65CV. The allowable bias (Ba), allowable CV (CVa), and allowable total error (TEa) were derived from European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Biological Variation Database [15], which were 3.6 %, 3.8 % and 9.8 %, respectively. The laboratories are considered qualified if their bias, CV, and TE of both samples are all within the allowable limit. The pass rate refers to the percentage of qualified laboratories in total participating laboratories. The sigma value (σ) for each laboratory was calculated as (TEa-bias)/CV. The measurement capability of the clinical laboratory is acceptable with σ ≥3, and excellent with σ ≥6. Otherwise, the Quality Goal Index (QGI) was calculated as bias/(1.5CV) if σ <3, which indicates the results of the laboratory is not acceptable. QGI<0.8, 0.8≤QGI≤1.2, QGI>1.2 indicate the precision, both precision and accuracy, and accuracy need to be improved [16], [17], [18].

Microsoft Excel 2010, GraphPad Prism 9.3.1, SPSS 17.0 were used for data analysis and presentation.

Characteristic of the TVP samples

The TVP samples were homogeneous with p value >0.05 (Supplementary Table S2), as well as stable at R.T. for 8 h, at 2–8 °C for 24 h, at −80 °C for at least 6 months, and during transportation for 4 days (Supplementary Table S3). The commutability of TVP samples at 2017 was also confirmed to be commutable in four clinical commonly used systems, Roche, Beckman, Mindray, and Maccura system (Supplementary Figure S1).

The pass rate of clinical laboratories

From 2017 to 2022, the relative bias in each year across all clinical laboratories ranged as: −7.9 %–8.7 %, −5.5 %–9.8 %, −6.8 %–8.3 %, −7.6 %–7.9 %, −3.90 %–8.0 %, −7.0 %–12.7 %, respectively. The CV ranged as: 0.10 %–4.93 %, 0.13 %–9.36 %, 0.16 %–5.63 %, 0.12 %–7.39 %, 0.12 %–4.05 %, 0.12 %–6.22 %, respectively. The TE were 1.08 %–14.73 %, 1.07 %–16.18 %, 0.91 %–14.15 %, 0.85 %–17.18 %, 0.56 %–11.04 %, 0.94 %–17.23 %, respectively. The average bias, CV and TE for the 12 TVP samples across all clinical laboratories meet the specified criteria, the observed average bias all displayed a positive trend (Table 2).The pass rates evaluated by relative bias, CV and TE for all laboratories were 45.2 %–64.8 %, 96.8 %–98.9 %, and 83.9 %–97.1 %, respectively, for the six years, the details have been listed in Table 2.

The pass rate of matched systems vs. un-matched systems

The pass rate by bias of matched systems and un-matched systems are 62.5 %–70.6 % and 30.4 %–62.1 %, respectively. The pass rate by CV for the two kinds of systems is 91.7–100.0 % and 93.9–100.0 %, and the pass rate by TE is 83.3–100.0 % and 75.8–93.9 %. The measurement results of each clinical laboratory classified by matched and un-matched systems over the six years are displayed in Figure 1.

Figure 1:

The measurement results of matched-systems and un-matched systems from clinical laboratories in the Trueness Verification Plan from 2017 to 2022. The quadrilateral box denote the allowable range of relative bias. (A) In 2017, 64.0 % (16/25) of the measurement results for the matched systems were within the allowable range of relative bias, and 36.4 %(12/33) for un-matched systems. (B) In 2018, 62.5 % (15/24) of the measurement results for the matched systems were within the allowable range of relative bias, and 34.2 %(13/38) for un-matched systems. (C) In 2019, 68.0 % (17/25) of the measurement results for the matched systems were within the allowable range of relative bias, and 62.1 %(18/29) for un-matched systems. (D) In 2020, 64.9 % (24/37) of the measurement results for the matched systems were within the allowable range of relative bias, and 36.4 % (12/33) for un-matched systems. (E) In 2021, 70.6 % (24/37) of the measurement results for the matched systems were within the allowable range of relative bias, and 30.4 % (7/23) for un-matched systems. (F) In 2022, 65.5 % (36/55) of the measurement results for the matched systems were within the allowable range of relative bias, and 45.5 % (15/33) for un-matched systems.

The pass rate of hexokinase method vs. glucose oxidase method

There are several methods used for serum glucose measurement in this TVP program from 2017 to 2022. And the hexokinase method and glucose oxidase method are the most commonly used, accounting for 51.6 %–64.9 % and 31.6 %–48.4 %, respectively. The pass rates for the two methods by bias are 53.1 %–78.6 % and 29.2 %–52.2 %, respectively. The pass rates by CV for the two methods are 93.8–100.0 % and 94.4–100.0 %, and the pass rates by TE are 75.0–98.2 % and 64.5–95.8 %. All have been summarized in Table 3.

The pass rate for different biochemical analyzers

The commonly used biochemistry analyzers were Roche, Beckman, Hitachi, Mindray, Siemens, and Abbott, their performance are also evaluated in this six-year TVP program. The pass rate by bias for the six detection systems are 64.3 %–88.9 %, 40.0 %–62.1 %, 15.4 %–63.6 %, 0.0 %–71.4 %, 0.0 %–75.0 %, and 0.0 %–100.0 %, respectively, which exhibited significant variability, especially for the latter three systems due to the limited number of participating biochemical analyzer systems (Figure 2).

Figure 2:

The average pass rate evaluated by relative bias for six biochemistry analysis systems in Trueness Verification Plan from 2017 to 2022. The pass rate by bias over the six years for Roche system is 64.3 %–88.9 %, for Beckman system is 40.0 %–62.1 %, for Hitachi system is 15.4 %–63.6 %, for Mindray system is 0.0 %–71.4 %, for Siemens system is 0.0 %–75.0 %, and for Abbott system is 0.0 %–100.0 %, respectively.

Analyzed by Six Sigma metrics and QGI for clinical laboratories

Six Sigma metrics is a management methodology aimed at improving the quality and repeatability of inspection results by reducing errors and variability. Sigma value (σ)=6” signifies that 99.99966 % of the results are error-free, with only 3.4 defects per million opportunities (DPMO), which is considered as “world-class” measurement level. Otherwise, “σ=3” indicates that 93.3 % of the results are error-free, corresponding to 66807 DPMO, which is considered as “poor” measurement [17, 18]. In this program, the sigma values were also calculated for each clinical laboratory, which has been shown in Figure 3. The laboratories with σ ≥6 for both concentrations, indicate their measurement capability is qualified as “world-class”, account for 41.4 %, 45.2 %, 35.2 %, 51.4 %, 61.4 %, and 45.5 %, respectively, from 2017 to 2022. Laboratories with σ ≥3 for both concentrations were considered to have an “acceptable” measurement capability, with percentages of 81.0 , 74.2, 79.6, 85.7, 84.2, and 76.1 % over the six years. Otherwise, the measurement capability of laboratories were regarded as “unacceptable” with σ <3, were 6.9 %, 4.8 %, 7.4 %, 1.4 %, 8.8 %, and 9.1 %, respectively.

Figure 3:

Sigma values for two concentration samples of each laboratory from 2017 to 2022. The x-axis and y-axis represent the sigma values of sample A and sample B for each laboratory. The small dots in the upper right quadrant represent the laboratories with σ ≥6, indicate their measurement capability is “world-class” The small dots in the bottom-left box represent the laboratories with σ <3, indicate their measurement capability are “unacceptable”. The other small dots represent the laboratories with 3≤σ<6, indicate their measurement capability is “acceptable” The measurement capability for laboratories with σ≥6, σ≥3 and σ<3 for both samples in (A) 2017 were 41.4 %, 81.0 % and 6.9 %, respectively; (B) 2018 were 45.2 %, 74.2 % and 4.8 %, respectively; (C) 2019 were 35.2 %, 79.6 % and 7.4 %, respectively; (D) 2020 were 51.4 %, 85.7 % and 1.4 %, respectively; (E) 2021 were 61.4 %, 84.2 % and 8.8 %, respectively; (F) 2022 were 45.5 %, 76.1 % and 9.1 %, respectively.

The QGI was calculated to evaluate the factors leading to unsatisfactory results for laboratories with σ <3. The percentage of laboratories with QGI values more than or equal to 1.2 ranged from 0.0 to 79.0 %, indicate the main factor of accuracy. While the percentage of laboratories with QGI values less than 0.8 ranged from 9.1 % to 100.0 %, indicate the main factor of precision. The percentage of laboratories with QGI values between 1.2 and 0.8 varied from 0.0 to 27.3 % over the six years, indicate the factors of both accuracy and precision. The details are shown in Table 4.