How to define a significant deviation from the expected internal quality control result

Introduction

ISO 9000 [1] defines quality control (QC) as “part of quality management focused on fulfilling quality requirements”. In clinical laboratory testing, QC includes the procedures intended to monitor the performance of a test procedure to ensure reliable results. Internal quality control (IQC) represents the whole set of activities performed to assure the constant monitoring of the performances of an analytical system with the aim of providing an alarm as soon as the analytical process fails to meet the predefined analytical goals.

ISO 15189 [2] requires that (paragraph 5.6.1) “The laboratory shall design internal quality control procedures that verify the attainment of the intended quality of results”. “Intended quality” is related to the quality goal or quality requirement for the test, which means the level of precision and trueness necessary for the method.

This goal can be accomplished for statistical QC procedures by following the guidance in CLSI C24-A3 [3]. According to CLSI C24-A3 the careful planning of an IQC procedure requires several steps:

• Define the quality specification for the test;

• Select the appropriate control materials;

• Determine the stable (in control) performance characteristics of the measurement procedure;

• Identify candidate quality control strategies;

• Predict the likelihood that candidate quality control strategy will detect out-of-specification performance;

• Specify desirable goals for the QC performance characteristics;

• Select a quality control strategy whose predicted performance meets or exceeds the quality control performance goals.

Once selected the quality specification and the appropriate QC material a possible way to determine the in control performance characteristics of the method is to calculate the measurement uncertainty of the results produced.

The introduction of the concept of measurement uncertainty [4] led us to consider an approach to the identification of a QC strategy different from the Westgard approach and the σ-metric QC selection tool [5]. According to the International Vocabulary of Metrology (VIM) measurement uncertainty is defined as “non-negative parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used” [6]. The “Uncertainty Approach” assumes that the information from measurement only permits assignment of an interval of reasonable values to the measurand, based on the assumption that no mistakes have been made in performing the measurement and that “a measurement result is generally expressed as a single measured quantity value and a measurement uncertainty”.

ISO 15189 [2] at paragraph 5.5.1.4 requires “The laboratory shall determine measurement uncertainty for each measurement procedure in the examination phase …” so any ISO 15189 accredited laboratory should have already available the uncertainty data.

The proposed approach is essentially based on the following documents: JCGM 106:2012 Evaluation of measurement data – The role of measurement uncertainty in conformity assessment [7], ISO 14253-1:2013 [8] and Eurachem/Citac Guide “Use of uncertainty information in compliance assessment” [9] that describe the use of uncertainty data to prove conformity with a defined target.

Theoretical basis

• Estimated measurement uncertainty shall be taken into account to prove the conformity or non-conformity with the given specification.

• The complete measurement result, y′, is represented by the measurement result±measurement uncertainty (U) (Figure 1).

• Conformity with a specification is proved when the complete measurement result, y′, falls within the zone defined by a lower specification limit (LSL) and an upper specification limit (USL) identifying the maximum permissible error (Figure 2A).

  LSL≤y−U and y+U≤USL.

• The same conformity can be proved similarly when the measurement result, y, falls within the zone of maximum permissible error reduced on either side by the expanded measurement uncertainty “acceptance zone” (Figure 2B).

  LSL+U≤y≤USL−U

• Non-conformity of a specification is proved when the complete measurement result, y′, falls outside the maximum permissible error (Figure 3).

  y+U≤LSL or USL≤y−U

• The problem arises when the complete measurement result, y′, includes one of the limits (Figure 4A) [or y falls outside the limits reduced by U (Figure 4B)]. In this case the knowledge of the possible values of the measurand, summarized by giving a best estimate (the measured quantity value) together with an associated measurement uncertainty, can be encoded and conveyed by a probability density function or a numerical approximation of such a function. An assessment of conformity with specified requirements is thus a matter of probability. The probability of conformity falls to 50% when the measurement result falls on the tolerance limit. However when the IQC result falls into the gray zone it implies an alarm and indicates that the probability of respecting the defined quality specification is reduced. To react or not to the alarm depends on the probability of not respecting the quality goals that one accepts.

Figure 1:

Result of a measurement, y, and complete measurement result, y′.

Figure 2:

Conformance with a specification is proved.

(A) The complete measurement result falls within the maximum permissible error. (B) The measurement result falls within the maximum permissible error reduced on either side by expanded measurement uncertainty.

Figure 3:

Non-conformity with a specification is proved (ULS≤y–U).

Figure 4:

(A) and (B). Neither conformity nor non-conformity with specification can be proved.

The model is based on the following assumption:

• The control materials respond to the modifications in the analytical system in a manner similar to the patients’ samples.

• The measurement uncertainty has been defined in a correct manner, taking into consideration all the possible sources of variation, including bias or its correction.

• The model is well applicable only if the defined quality goal is greater than the measurement uncertainty (U) otherwise no “acceptance zone” can be defined. If U is larger than the quality goal the probability of producing a result within the quality specification is lower than 95%, so one has to adopt (and communicate to the customers) a lower level of target quality (or change or modify the analytical method).

• The measurement uncertainty should be periodically verified to assure its stability with time.

Calculation of measurement uncertainty

The GUM, bottom-up approach [5] is in general not applicable to the clinical laboratories. The top-down approach, as proposed by the NordTest Report [10], derives the random component of the uncertainty [intermediate precision – u(Rw)] from the results of the IQC and the systematic component by using a certified reference material (CRM). The trueness control provided by the manufacturer represents the most practical approach. A CRM produced by international organizations like NIST or IRMM might be used only if its commutability for the method in use is demonstrated. To calculate the random component of uncertainty we propose to use the weighted mean of the monthly CV of at least 6 months according to the following equation:

(1)u(Rw)=(nA−1)×CVA2+(nB−1)×CVB2+…+(ni−1)×CVi2(nA+nB+…+ni)−nperiods (1)

The calculation has to be performed at different concentration levels.

The bias can be calculated from the target value of the manufacturer’s trueness control (CRM) according to the following equation [10]:

(2)u(bias)=(bias)2+(Sbiasn)2+u(Cref)2 (2)

Where the bias is the % difference from the nominal or certified value

(Sbiasn) is the standard error of the measurements performed on the CRM

u(Cref) represents the expanded uncertainty around the certified value (but this information is often not available).

u(Cref) is usually expressed as expanded uncertainty, so:

u(Cref)=UCRM2u(Cref)%=u(Cref)CRM×100

The square root of the sum of the squares of the two components of the uncertainty gives the combined uncertainty:

(3)u=u(Rw)2+u(bias)2 (3)

And expanded uncertainty (U) is calculated according to the equation

(4)U=u×k, (4)

where k=2 for a probability of 95.5%.

Materials and methods

IQC results for glucose and creatinine are from two different clinical laboratories.

Analyzers: for glucose, Ortho Vitros 350 (Ortho Clinical Diagnostics, Milano, Italy) (Laboratory 1), for creatinine Siemens Advia 2400, alkaline picrate reagent (Siemens Healthcare Diagnostics, Milano, Italy) (Laboratory 2). Both centers used the same IQC material: Bio-Rad Liquicheck Unassayed Chemistry Control, lot 16670 (Bio-Rad Laboratories srl, Segrate (MI) Italy.

Results

Measurement uncertainty

Glucose: u(R_w)=1.16%, u(bias)=0.68%, Combined uncertainty (u)=1.34%, Expanded uncertainty (U)=2.69%

Creatinine: u(R_w)=4.69%, u(bias)=0.61%, Combined uncertainty (u)=4.83%, Expanded uncertainty (U)=9.66%

In Figure 5A is presented a control chart where the mean is represented by the mean obtained by the laboratory on the specified control material in the previous months, the acceptability limits are represented by the total allowable error (TEa) defined according to the principle of biological variation (BV), desirable limit for total error [11, 12] and the obtained results are represented as complete measurement results (result±U). It appears clearly that almost in all cases the complete measurement result falls within the maximum permissible error limits; Figure 5B shows the same results, but with the more intuitive approach presented in Figure 2B.

Figure 5:

Glucose quality control chart.

(A) Glucose quality control chart. Each quality control result is presented with ± measurement uncertainty. The maximum permissible error limits are defined according to the biological variation theory as ±6.96%. (B) Same glucose quality control chart as in A, but in this case the quality control results are presented without uncertainty while it is indicated the “acceptance zone” (maximum permissible error reduced on either side by expanded uncertainty, U)

Figure 6 has the same structure of Figure 5B and presents the results for creatinine, low concentration control material (laboratory overall mean 70 μmol/L). In this case it has been impossible to adopt the desirable quality limit according to the BV principles (TEa=8.87%) [12] because U was higher (9.66%). However, also applying the minimum quality criterion (TEa=13.3%) only 52% of the data fall in the “acceptance zone”, all the others but one are in the “gray zone” indicating that we cannot guarantee, with the 95% probability, the respect even of the minimum quality specification. On the contrary, at higher creatinine concentration (460 μmol/L), 95% of the data point fall in the acceptance zone (U=6.6%) (data not shown).

Figure 6:

Low level creatinine quality control chart. Same as Figure 5B.

If we compare these three examples with the classical 6σ approach we found that for glucose we are almost in a 6σ situation (imprecision component of U=1.16%, small bias, TEa=6.96%, σ=5.4). For creatinine, low concentration, the σ-value is only 2.8 (even with a very limited bias component, imprecision is 4.69%), while for higher creatinine concentration the σ-value is 4.4 (imprecision=3.0%).

Discussion and conclusions

According to CLSI C24-A3 “For purposes of quality control, the laboratory must consider the stability of the analytical testing process, its susceptibility to problems that may occur, and the risk associated with an undetected error” [3]. In our opinion the proposed approach is very easy and intuitive: if the QC result falls within the “acceptance zone” it means that its complete measurement result (±U) is within the maximum permissible limit (with ≥95.5% probability), so no further actions or rules are required. With this approach the chosen quality specifications are directly indicated in the QC chart and the relationship between desired quality and performance of the method is immediately perceivable. The proposed approach works independently from the criterion applied to set the quality specifications. They can be based on true or false classification [13] as in case of total cholesterol or HbA_1c, biological variation [11] or state-of-the-art [14].

We chose to apply the TEa limits as quality goals because a single measurement (like the one preformed on a control material) is always affected by both random and systematic error. Even if metrological traceability ensures that “the method” is virtually unbiased, this does not mean that its application in a specific laboratory could not introduce a certain amount of bias (calibrations, ageing of the reagents, different reagent lots etc.), so the quality goal for a single measurement remains the total error, not only the imprecision.

When the relationship between maximum permissible error (however defined) and U is >2.5 (as in Figure 5) the situation is optimal and, purporting that the reactions of the control material to possible alterations in the analytical system are similar to those of the patients’ samples, it is very easy to keep the system under control. There will be the guarantee that the vast majority of the results will respect the defined quality limits. Ideally we should strive for working in these conditions for all our tests. When U is close to or even larger than the quality specifications it will be impossible to guarantee the respect of the specifications. In such a situation, like for low level creatinine of Figure 6, the only effective solution is to change the analytical method (e.g., moving to enzymatic creatinine) or even analytical system. In the meanwhile higher frequency of calibration and maintenance could help in reducing the uncertainty level, minimizing bias, thus allowing a higher percentage of samples to respect the quality goals. Also with this approach it is possible to calculate a sort of power function curves based on the ratio between the quality specification and U and the number of IQC replicates, these data will be presented in a further publication.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.