Two approaches for measurement uncertainty estimation: which role for bias? Complete blood count experience

Introduction

The result of a measurement is only an approximate estimation of the value of the measurand. Therefore, it is completed with an expression of the uncertainty of this estimate [1]. Measurement uncertainty indicates a range of values that cover the measured parameter’s exact value [2], [3]. When the measurement uncertainty is available with the test result, it provides information about the result’s quality [4]. Laboratories should determine their measurement uncertainty values for parameters they analyze as a requirement of ISO 15189.

There is no clear consensus about the calculation of measurement uncertainty and there are different approaches. Calculation methods can be divided into two main approaches, bottom-up and top-down. In the bottom-up approach, all factors and the effect-on rates are included in calculating the uncertainty one by one. On the other hand, top-down approach uses present analytical performance data obtained from quality control materials [5], [6].

The top–down approach is reasonably preferred, as it is easier to apply. But there is also no consensus on how to apply the top–down approach. ISO 15189 does not mandate any particular procedure. On the other hand, ISO Technical Specifications 20914 is based on precision results and calibration uncertainty and recommends the correction of bias or the inclusion bias uncertainty in the uncertainty calculation [7]. There are different views on the inclusion and exclusion of bias [8], [9].

Australasian Association of Clinical Biochemists Uncertainty of Measurement Working Group (AACB) and Handbook For Calculation of Measurement Uncertainty in Environmental Laboratories (Nordtest Guideline) are written according to the top–down approach [10], [11].

AACB Guideline advises at least six months of imprecision of IQC data to calculate measurement uncertainty and does not recommend using bias [10]. On the contrary, the Nordtest guideline equation uses bias, imprecision and the participant number of the EQA and the imprecision of IQC [11]. Both recommend calculating measurement uncertainty separately for different concentration levels of the measurand.

Complete blood count (CBC) is one of the most common laboratory tests. It is preferred for diagnosis and follow-up of a wide range of diseases such as anemia, infectious diseases, certain cancers, bleeding conditions, allergies and immunodeficiencies [12]. Every parameter reported under CBC headline may help the physicians to diagnose different diseases.

Analytical imprecision varies depending upon the concentration of the analyte [11]. Therefore, measurement uncertainty is also expected to have different values at different concentrations. Analytical imprecision, hence measurement uncertainty, is particularly greater at the concentration of lower and the upper limit of measurement [13]. User manuals have no information about measurement uncertainty but includes performance characteristics of autoanalyzers as imprecision.

The study aims to calculate the measurement uncertainty values of certain CBC parameters with two different top–down approaches and compare them via imprecision values of user manuals, as performance characteristics of autoanalyzers. To the best of our knowledge, there is no study in the literature which compares measurement uncertainty guidelines and shows the effect of bias by using CBC parameters and the difference in imprecision values between concentration levels.

Materials and methods

This retrospective study was performed in Karadeniz Technical University Farabi Hospital with the approval of the ethical committee. Quality control data of 2017 (January 2017 to December 2017) were used for calculating measurement uncertainty values of four CBC autoanalysers.

Measurement uncertainties were calculated for each of the two Beckman Coulter LH780 (Miami, USA) (enumerated as R1 and R2) available in the routine laboratory and two Beckman Coulter DXH 800 (Miami, USA) (enumerated as E1 and E2) available in the emergency laboratory. Measurement methods are as follows for both autoanalyzer models: Coulter Principle for white blood cell count (WBC), red blood cell count (RBC) and platelet count (PLT). hemoglobin (HGB) is measured photometrically, mean corpuscular volume (MCV) and red cell distribution width (RDW) are derived from RBC histogram. Mean platelet volume (MPV) is derived from PLT histogram, Hematocrit (HCT), mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC) are calculated parameters [14], [15]. Calibration samples of both autoanalyzer models do not have any uncertainty value for parameters.

Three levels (low, medium, high) of IQC of CBC parameters were carried out by Beckman Coulter 5C and 6C Cell Control for LH780 models and DXH800 models. Quality control samples changed monthly but mean (target) values of samples obtained from the manufacturer were close to each other. 13 and 12 different lot series were used for IQC for LH780 and DXH800 models, respectively. “n” is the number IQC sample, pooled IQC CV calculated as:

For each level separately.

EQA program was BIO-RAD External Quality Assurance Services Hematology Program. WBC, RBC, HGB, HCT, PLT, MCV, MCH, MCHC, RDW, MPV are evaluated by EQA. Therefore, these parameters were included in the study.

Expanded measurement uncertainty found by multiplying pooled imprecision value by coverage factor in the equation for two sided estimation at 95% confidence interval:

In accordance with AACB guide.

We used peer group results of EQA reports for all bias, CV% and participants numbers for Nordtest guide equations. Nordtest recommends using bias values of EQA results as a Root Mean Square-bias (RMS_bias) value. RMS_bias, where the “n” is the number of EQA sample, is found with the equation:

Additionally, EQA samples have their uncertainty, named as u (c _ref). CV% of external quality control and participants number (pn) can be used for calculation u (c _ref) as:

Coverage factor k=2 is chosen for two-sided estimation at a 95% confidence interval. The guide calculates measurement uncertainty in two steps. We made these two steps into one equation as follows:

To calculate measurement uncertainty separately for different concentration levels of the measurand, we classified EQA samples retrospectively according to their mean levels. Thus, we divided the EQA samples into three groups (low, medium, high) in accordance with the three levels of IQC. Averages of mean values of IQCs are given in Table 1. If three or more EQA results close to a certain concentration level, we calculated measurement uncertainty for that level with the Nordtest guideline. If two or less results were available, we accepted insufficient EQA results to calculate measurement uncertainty with the Nordtest guideline.

User manuals of analyzers do not have measurement uncertainty values but show within-run precision values. Although they are different concepts, imprecision values and measurement uncertainty values are parallel due to the equations. Also, the imprecision values reflect the test performance. For the tests that perform good, the measurement uncertainty value is expected to be low. So we use user manuals to evaluate our results.

Results

Measurement uncertainty percentages (%) of WBC, RBC, HGB, HCT, PLT of two Beckman Coulter LH-780 model auto analyzers are shown in Table 2. Table 3 shows the measurement uncertainty of the same parameters in the Beckman Coulter DXH-800 model. Graph 1 and Graph 2 show the measurement uncertainty percentages of PLT and WBC of different levels and guides, respectively.

IQC mean values of all three levels of MCH, MCHC, MCV, MPV, RDW parameters are so close to each other (Table 1). Therefore, we calculate measurement uncertainty values for only one level for these parameters (Table 4).

Imprecision values of IQC for all parameters differs between 0.47 % (Beckman Coulter LH-780, HGB, level of 155 g/L) and 2.41% (Beckman Coulter DXH-800, PLT, level of 70 × 10⁹/L).

The maximum RMS_bias value is 7.29% and belongs to PLT in Beckman Coulter LH-780 at level of 70 × 10⁹/L. The minimum RMS_bias value is 0.37% and belongs to HGB Beckman Coulter LH-780, at the level of 123 g/L. RMS_bias values of EQA for other parameters differ between these percentages.

u (c _ref) values of EQA for all parameters differs between 0.1 % (Beckman Coulter LH-780, HGB, level of 155 g/L) and 0.8 % (Beckman Coulter DXH-800, PLT, level of 70 × 10⁹/L).

Conclusion

Analytical performance of auto analyzers depends upon concentration levels of the measurand and in hematology auto analyzers, CV values increase as the platelet count decreases [16]. There is no consensus for evaluating measurement uncertainty values, so it is hard to estimate which approach gives more confidential results. However, in our study, PLT and WBC results revealed striking data especially when evaluated with the concentration-imprecision relationship.

It is considered that the user manuals have one and only imprecision value for most parameters, while there are different imprecisions at different concentrations of PLT and WBC. For WBC at 0.5–2 × 10⁹/L concentration level CV% ≤5%; at 5–10 × 10⁹/L concentration level CV≤3% in user manual of Beckman Coulter DXH-800. LH-780 user manual gives unique CV for WBC as ≤1.7% at 9–11 × 10⁹/L concentration level. For PLT in the user manual of Beckman Coulter LH-780 CV≤3.3% for the level of 280–320 × 10⁹/L; CV≤6.6% for the level of 90–110 × 10⁹/L; CV≤14% for the level of 10–15 × 10⁹/L. And in user manual Beckman Coulter DXH-800 CV≤3.5% for the level of 200–400 × 10³/μL; CV≤12% for the level of 10–12 × 10⁹/L for PLT [14], [15]. The repeatability of DXH800 is slightly better than LH780 at low concentration levels of PLT. In agreement with the literature, it is seen that the imprecisions of auto analyzers increase in the user manuals as the PLT and WBC levels decrease. Auto analyzers perform better at higher levels of PLT and WBC; measurement uncertainty values are also expected to be compatible with this situation.

The measurement uncertainty value calculated with the AACB guideline can be expected to be more correlated with the imprecision values of the user manuals than those calculated with Nordtest because imprecision is the one and only variable that requires to calculate measurement uncertainty with AACB. However, in our study CV% values of IQC are so close to each other, regardless of concentration level, in both autoanalyzers for PLT. Therefore measurement uncertainty values calculated with AACB guideline were close to each other, contrary to expectations. But the part of our study performed with the Nordtest guideline is compatible with user manuals. Although 10–15 × 10⁹/L level of PLT does not have IQC and EQA, it is obvious that the uncertainty calculated with the Nordtest guideline increases as the concentration of PLT decreases (Figure 1).

Figure 1:

Measurement uncertainty of PLT of four autoanalyzers with two different guideline, with contribution rates of RMS_bias, u (c _ref) and imprecision of IQC. AACB, Australasian Association of Clinical Biochemists Uncertainty of Measurement Working Group Measurement Uncertainty Guideline; Nordtest, Handbook for Calculation of Measurement Uncertainty in Environmental Laboratories Guideline; u (C _ref), uncertainty of external quality assurance sample; RMS_bias, root mean square bias; IQC CV, imprecision of internal quality control; R1 and R2, two Beckman Coulter LH-780 autoanalyzers run in routine laboratory; E1 and E2, two Beckman Coulter DXH-800 autoanalyzers run in emergency laboratory.

PLT and related parameters provide information about bleeding disorders and the risk of thrombosis [17], [18]. Low levels of our IQC (70 × 10⁹/L) may be the most clinically important levels of PLT because the diagnostic threshold of thrombocytopenia is close to that level. Moreover, the indication of platelet transfusion is distinctly lower than that value. Guidelines often recommend a platelet count threshold of 50 × 10⁹/L before major surgery. The prophylactic platelet transfusion threshold is accepted as 10 × 10⁹/L for uncomplicated thrombocytopenic patients [19]. So calculation and reporting of measurement uncertainty of lower levels are more important than higher levels.

IQC imprecision of WBC is pretty lower than imprecision of user manuals. However, it was compatible with user manuals in terms of differences in concentration levels. On the other hand, lower IQC imprecision results in lower measurement uncertainty percentages than expected. In the user manual of DXH-800, CV≤3% for the concentration of medium level of WBC. Thus measurement uncertainty can be expected as ≤6% with AACB. In fact, uncertainties calculated with AACB were nearly half of the expected value, but Nordtest results are more compatible with expected uncertainty results (Figure 2).

Figure 2:

Measurement uncertainty of WBC of four autoanalyzers with two different guideline, with contribution rates of RMS_bias, u (c _ref) and imprecision of IQC. AACB, Australasian Association of Clinical Biochemists Uncertainty of Measurement Working Group Measurement Uncertainty Guideline; Nordtest, Handbook for Calculation of Measurement Uncertainty in Environmental Laboratories Guideline; u (C _ref), uncertainty of external quality assurance sample; RMS_bias, root mean square bias; IQC CV, imprecision of internal quality control; R1 and R2, two Beckman Coulter LH-780 autoanalyzers run in routine laboratory; E1 and E2, two Beckman Coulter DXH-800 autoanalyzers run in emergency laboratory.

Nordtest guideline suggests at least six interlaboratory comparison results within a reasonable time interval, but in the examples of guideline, three results are used to calculate RMS_bias value. It is hard to find six results for each level in one year. Padoan et al. [9] have not calculated measurement uncertainties for certain parameters in a study designed with Nordtest Guideline and indicated this situation with the words of ‘surveys per year are low’ but they did not give exact number of surveys. We accepted that three EQA samples could be used to estimate RMS_bias, based on the examples in the guideline. In our study, the measurement uncertainty values could not be calculated according to the Nordtest guideline since there were not enough EQA samples at the high levels of WBC and PLT and low RBC, HGB, HCT. In order to overcome this problem, it can be attempted to reach a sufficient number of EQA samples for a certain level by increasing the time interval. A single-bias value may not reflect the quality of measurement, so we do not recommend calculating measurement uncertainty with a single bias.

The use of imprecision is common in both guides. However, Nordtest uses it in a Pythagorean equation (Eq. 5), so the importance of CV% of IQC decreases in Nordtest guideline in proportion to AACB. By using the bias values, the uncertainty values calculated with the Nordtest guide are generally higher than those calculated with AACB guide.

Manufacturer within-run precision recommendations for parameters other than PLT were <2.2% in LH780 model, and <5% in DXH800 model. Our imprecision values are convenient to these recommendations.

Although three levels of IQC samples are used, we have observed that only quality control of the values close to the reference range of the MCH, MCHC, MCV, MPV, RDW are appropriately performed. This is the same in terms of EQA and IQC. We do not have sufficient data about the analytical performance of pathological levels of these tests, which have an important role in determining the etiology of anemia. It may also cause suspicion to clinical studies committed with these parameters. This is an issue that the manufacturers of IQC and EQA samples and us as the experts who decide which quality control material to be used in the laboratory should pay attention.

In conclusion, we reached the most striking findings when calculating the measurement uncertainty of the PLT parameter. These findings show the importance of the use of bias in the calculation of measurement uncertainty and representing the novelty of this study.