Detection and management of differences in biochemical results across identical analyzers in high-volume laboratories: a practical internal quality control-based method

Introduction

Clinical laboratories are essential for patient management, disease monitoring, and treatment evaluation. Compliance with the International Organization for Standardization (ISO) 15,189:2022 becoming increasingly necessary [1]. One of the technical requirements of this standard (ISO 15189:2022, clause 7.3.7.4) emphasizes the need to ensure the comparability of patient results when the same test is performed on multiple devices.

Clinical assessment of patient results often relies solely on reference ranges, which cover only 95 % of the population. Furthermore, using reference ranges in tests where individual biological variation (BV) is lower than group variation may fail to detect minor but clinically important changes in an individual’s results, which is a characteristic of most clinical biochemistry tests. In such cases, criteria such as critical difference and relative change value (RCV), which represent the variation between consecutive measurements and consider within-individual variability, become relevant [2], 3]. Therefore, using multiple devices for the same test, even if they are of the same brand and model, highlights the importance of minimizing analytical variability.

Results from different devices, even of the same brand and model, must be comparable to ensure consistent patient care. While comparison studies are conducted, there is no consensus on how to perform and evaluate them in daily practice [4].

The literature review reveals a need for more consensus on evaluating the significance of differences between results and using different criteria for this purpose [5], [6], [7].

Our university laboratory, accredited under ISO 15189 since 2012, has consistently maintained this status and initiated systematic studies to compare results from analyzers of the same brand and model. The primary goal was to standardize a model that is easy to implement in routine practice, requires minimal additional workload, and is cost-effective.

In this study, we aimed to demonstrate how, in our institution, a more practical approach can be used to compare the results of the same test performed on multiple identical analyzers by relying solely on internal quality control (IQC) data.

Materials and methods

Methods

As summarized and illustrated in Figure 1, the study utilized two-level IQC data that Westgard criteria (22 S, 13 S, R4S) and were approved during the first five uninterrupted working days of each month between May 2022 and April 2023. In our study, the CV was calculated using L1 and L2 IQC data obtained over five consecutive working days that met the Westgard rule acceptance criteria. For each test, a CV value was first calculated from the L1 data and another from the L2 data, and the mean of these two values was taken as the CV for that test. Although the number of IQC results used varied according to the characteristics of the test, between 15 and 200 IQC measurements were analyzed for each level, as presented under the “Number of controls” column in Table 2. The first IQC result produced was accepted as the target value (initial result) for that month, and all subsequent calculations were based on this value. This approach supplemented the use of Westgard rules. The rationale behind this method was to enable the detection of minor variations between analyzers.

Figure 1:

Flowchart depicting the evaluation process for two-level IQC data according to westgard rules (22 S, 13 S, R4S) and comparision of calculated total error with total acceptable error from EFLM and Rili-BAEK. EFLM, european federation of clinical chemistry and laboratory medicine; IQC, internal quality control; Rili-BAEK, German guidelines for quality assurance; TE, total error; TEa, total allowable error.

Table 2:

CV and bias values calculated separately for each month using internal quality control data for 20 parameters.

Test name	Number of controlsa, Median (range)	CV, %, Median (range)	Bias, %, Median (range)
Albumin	121 (94–142)	2.13 (1.20–4.26)	2.12 (0.96–4.77)
ALT	134 (111–148)	3.26 (2.15–6.47)	3.42 (1.84–6.46)
AST	120 (88–150)	2.95 (2.14–4.61)	4.1 (1.23–6.68)
BUN	132 (112–156)	2.99 (2.14–4.89)	2.99 (2.42–6.85)
Calcium	82 (56–92)	2.34 (1.22–2.85)	2.19 (1.35–4.12)
Creatinine	96 (68–120)	3.96 (2.25–4.52)	3.91 (3.15–6.51)
CRP	170 (117–200)	2.13 (1.34–2.68)	2.19 (1.25–3.97)
Direct bilirubin	63 (46–78)	3.04 (1.55–6.17)	3.81 (1.34–11.03)
Ethanol	25 (17–47)	4.33 (2.18–6.34)	4.06 (2.5–8.09)
Glucose	126 (100–165)	2.54 (1.64–3.77)	2.28 (0.84–4.92)
hCG	31 (24–36)	4.09 (2.69–8.24)	5.42 (3.21–26.15)
Hs-TnI	59 (31–40)	9.03 (5.08–12.87)	10.68 (4.64–16.9)
Potassium	42 (36–47)	2.33 (1.58–3.80)	2.1 (1.45–5.58)
Sodium	40 (35–49)	2.22 (1.56–2.61)	1.95 (1.4–3.09)
Triglyceride	34 (19–48)	2.70 (1.31–4.52)	2.41 (1.4–5.79)
Total bilirubin	50 (37–74)	2.99 (1.60–4.40)	3.42 (0.01–5.24)
Total cholesterol	26 (15–35)	2.51 (1.49–4.99)	2.87 (2.02–5.5)
Total protein	59 (46–71)	2.02 (1.11–3.62)	1.7 (1.39–3.74)
TSH	20 (20–30)	3.80 (2.26–9.22)	4.56 (2.96–24.46)
Uric acid	55 (44–73)	2.51 (1.44–3.21)	2.4 (1.42–5.7)

1. ^aNumber of IQC, measurements obtained during the uninterrupted first five consecutive working days of each representative month and accepted according to Westgard rules.

Based on the initial result, the coefficients of variation (CV) and bias for all subsequent IQC results within the five-day period were calculated using the formulas shown below.

% CV = Standard   Deviation Mean x 100 ,

% Bias = Measured   Mean − Target   Value Target   Value x 100

CV and bias values of representative five days of the month were then used to obtain pooled CV and bias. Pooled bias values were derived using the root mean square (RMS) deviation method [8]. A single total error (TE) value was calculated using the corresponding pooled CV and bias values. The TE was determined using the formula: TE=Bias + 1.65 × CV, and the acceptability of the error was evaluated according to predefined analytical quality specifications. For this purpose, monthly TE values were compared against two analytical quality specifications: Total allowable error (TEa) limits defined by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) BV database, and the German guideline (Rili-BAEK) [9], 10].

Results

In our study, all IQC data for each month between May 2022 and April 2023 – evaluated and accepted according to multiple Westgard rules (22 S, 13 S, R4S) – were collected during the first five consecutive working days of each month. The 12-month CV and bias values calculated for 20 tests, along with the number of IQC samples analyzed, are summarized in Table 2. An important point is that, in our laboratory, patient sample analysis on the analyzers begins only After the IQC runs have been accepted using the laboratory-defined multiple Westgard rules. Nevertheless, to eliminate potential inter-analyzer result differences, the approach designed in this study allows the implementation of additional measures – prior to running patient samples on the analyzers – when deemed necessary for certain tests. Such measures may include applying additional Westgard rules, introducing a third-level IQC material at a different concentration, updating calibration and IQC frequencies, or reviewing reagent kit test volumes, among others. The lowest number of IQC measurements was recorded for total cholesterol (n=15) and ethanol (n=17), whereas the highest average numbers of IQC measurements were observed for CRP (n=200), glucose (n=165) and BUN (n=156). The increased number of IQC runs for these tests was attributed to the use of multiple reagent kits within the same analytical unit.

Discussion

In routine practice, accuracy is monitored on a daily basis using IQC with Westgard rules frequently applied for performance evaluation. In our laboratory, the results from each autoanalyzer are routinely evaluated individually through participation in EQA programs. EQA results are generally the most reliable method for detecting bias. However, it often takes up to two weeks for these EQA results to become available to laboratories, leading to a considerable delay in the identification of test deviations.

Random errors increase patient risk and are more challenging to detect. Designing a QC plan to increase precision is crucial for patient safety. This reduces erroneous test results and decreases the need for patient retesting [7].

In high-capacity hospitals, daily IQC evaluation may require further improvement, as inter-device differences should not exceed biological variation. For this purpose, quality goals such as RCV, which includes BV and associated evaluation targets, can be utilized [11]. Additionally, using TE, obtained from various sources, has been widely accepted for evaluating analytical differences [11]. TE can be defined as an analytical quality requirement that sets a limit for both imprecision (random error) and bias (systematic error) that are tolerable in a single measurement or test result.

In our laboratory, due to reagent capacity and test volume, multiple reagent bottles may be used in a single module for tests like AST, ALT, creatinine, glucose, and urea. Given the possibility of reporting results using different devices or, within the same device, different modules and reagents on the same day particularly for hospitalized patients this study aimed to develop a standardized evaluation procedure to ensure reagent compatibility by utilizing IQC data and implementing a practical, Laboratory Information System (LIS)-integrated approach suitable for routine daily use.

Despite standardized calibrators and controls, personnel and equipment differences may influence results.

A Clinical and Laboratory Standards Institute (CLSI) guideline related to this topic (EP31-A-IR) is available [12]. According to the guideline, no standardized “Acceptance Criteria (AC)” covers all tests for comparison experiments. Depending on the variety of tests used in laboratories, different ACs may be appropriate. The Laboratory Director decides which of these criteria is suitable. We aim to establish a workflow suitable for routine use, facilitating our evaluation by defining rules that can be narrowed or expanded for each test.

Another strong AC in the guideline is “goals set by accreditation agencies”. Accreditation organizations and regulatory authorities, such as College of American Pathologists (CAP) or Rili-BAEK, set targets like TEa using Proficiency Testing or EQA results. Our study used TEa values determined by EFLM and Rili-BAEK as the target AC. When calculating bias, we used the difference between the initial value obtained for each test, as recommended in some publications, rather than the manufacturer’s assigned values of IQC [13]. It has been reported that as the number of measurements increases, the average results approach the manufacturer’s values, but the analytical CV increases [6].

In our study, the number of control results used ranged from 15 to 200 so these numbers explain why our CV values are higher than those reported in the literature.

Furthermore, since IQC results were obtained at different times of the day and weekly, our likelihood of encountering bias due to factors such as carry-over and reagent decay is higher than reported in the literature. In this instance, our data demonstrated greater reproducibility than repeatability [14], [15], [16]. Depending on the clinical purpose of the test, the laboratory specialist must make a specific evaluation for each test by narrowing or expanding the evaluation algorithm rules we defined. The technical requirements defined by the ISO 15189 standard emphasize that the comparability of patient results should be ensured when the same test is performed on multiple devices. It is stated that it should be periodically checked at clinically significant intervals, and if a difference is detected, its effect on biological reference ranges and clinical decision limits should be evaluated. Although patient samples are preferred for this purpose, the practical use of only IQC for daily comparison may be feasible.

When reviewing the literature, it is observed that there have been few publications related to device comparison methods in recent years. Until now, many studies have generally accepted the validity of TEa or measurement uncertainty (MU) methods in method validation. However, there still needs to be a consensus on the comparability of multiple devices of the same brand and model. TE, CV and bias, sigma metrics, and MU have been used in different publications [4], 6], 17], 18].

In a study published in 2023, data from two analyzers were compared over four years [18]. Monthly TE calculations were based on IQC CVs, and biases were derived from EQC data. Since we aimed to incorporate our defined workflow into daily routine operations, we intended to use only IQC data. In this study, unlike ours, the analytical performance of clinical chemistry tests performed on different brand autoanalyzers in different years was evaluated. Nevertheless, the results obtained are highly compatible with ours. When assessed according to the EFLM European Biological Variation Study (EuBIVAS) criteria, almost all tests failed; in contrast, nearly all parameters were within the acceptable limits defined by other guidelines, such as the Clinical Laboratory Improvement Amendments (CLIA).

In another study, they used IQC to calculate CV and EQC to calculate bias in an accredited laboratory with similar characteristics to ours, and they evaluated the data with CLIA TEa [5]. They evaluated the CV, bias, and TE goals of 26 tests separately. Tests identified as out of quality specifications (QS) were like our findings, including sodium and albumin. In a similar study, tests performed on two analyzers were grouped according to sigma values using a different perspective [6]. Albumin and sodium have the lowest sigma values ​​in almost all groupings.

In another study with methods most like ours, one analyzer’s IQC results were compared with the results of the other analyzer using only bias calculation [4]. When average biases were examined over 60 months, it was reported that less than 2 % of the samples exceeded the allowable bias for low-level controls of bilirubin, chloride, glucose, potassium, sodium, and urea, but these were deemed clinically insignificant. In our results, bilirubin and urea were successful when compared with EuBIVAS. Sodium, creatinine, glucose, and potassium were found to be unsuccessful in some months, like this study. Evaluations in the mentioned study were based solely on bias, and albumin was not included, unlike in our study.

A recently published study evaluated the effectiveness of the EP31-A-IR guideline for the periodic verification of comparability between results obtained from different analyzers by CLSI [19]. The assessment was made by comparing the maximum observed differences between the IQC results of the devices with the Maximum Allowable Differences. The Maximum Allowable Differences were calculated from the pooled standard deviation data obtained in the short- and long-term studies. The results showed that the differences between the analyzers did not exceed the target values for all parameters evaluated. Because our study aimed to compare devices of the same model in the same environment, we used the TE approach. However, as the variance components increase, it would be more appropriate to use more comprehensive measurement uncertainty approaches.

To our knowledge, no prior studies have simultaneously calculated TE and used such extensive IQC data as in our study The TEa model has gained widespread acceptance in practical laboratory settings due to its ease of use and the inclusion of random and systematic errors in analytical processes. Despite being primarily intended for method validation and standardization, we repurposed it for our study.

When each test was evaluated on a monthly basis, the 12-month TE results were analyzed by comparison with the EUBIVAS criteria, ALT, AST, BUN, CRP, direct and total bilirubin, triglycerides, and uric acid performed well. In contrast, other tests showed varying degrees of poor performance. When evaluated with Rili-BAEK, the variety of failed tests and the number of months decreased, with only sodium failing throughout the year. When tests meeting the EFLM Desirable Total Error (DTE) criteria were examined, they were almost perfectly aligned with the literature. In this case, we can say that comparing TE data with DTE will be sufficient for routine use.

The failure of sodium to meet all ACs, similarly to our study, has been emphasized in all literature. However, given today’s patient load, using multiple devices for sodium testing in laboratories is inevitable. According to the Milan Consensus criteria established in 2014, Analytical Performance Specifications (APS) should primarily be based on clinical output studies or BV data [20]. The BV based APS model is the preferred model for many tests performed in CLs, but this model has limitations in some cases. When applying the TE model is impossible, the “state-of-the-art” approach is recommended. It is known that some measurands, including sodium, fail to meet the APS predicted by BV models for currently used models due to their very low measurement uncertainty (<2 %) and the very narrow BV resulting from tight homeostatic control. When evaluated from a medical perspective according to the Milan hierarchy, the authors concluded that the current analytical quality of sodium meets medical needs.

Ideally, users should select analytical goals according to clinical needs [14]. Oosterhuis et al. also stated that for tests like sodium, which are widely used medically, state-of-the-art studies should be conducted, and APS should be based on medical necessity or outputs [21].

According to the rule-based approach proposed in our study, a TE value below the allowable limit defined by the EFLM indicates that the analytical performance of the test is adequate and that no additional control step is required. When the TE value falls between the TEa thresholds defined by the EFLM and Rili-BAEK, it serves as a warning signal, prompting close monitoring of the test by the laboratory specialist. TE values exceeding the Rili-BAEK threshold, however, necessitate the implementation of test-specific corrective actions, with the final decision to be made by the responsible laboratory expert.

The main limitations of our study include the use of Desirable Performance Specifications for TEa, which may not reflect optimal analytical conditions despite its widespread use in clinical practice, and the omission of preanalytical factors like sample handling, which should be addressed in future research.

Conclusions

In high-volume laboratories with multiple analyzers, ensuring result consistency is essential to prevent discrepancies that may affect patient diagnosis and follow-up. Our model enables continuous performance assessment of identical analyzers using only routine IQC data. Integrating this approach into the LIS may allow early detection and correction of inter-analyzer differences before they impact patient results, thereby improving the reliability of laboratory data, particularly for low-performing tests such as sodium, albumin, creatinine, and potassium.