Supplements to a recent proposal for permissible uncertainty of measurements in laboratory medicine

Introduction

The DGKL Working Group Guide Limits (Arbeitsgruppe Richtwerte) recently has published a proposal for deriving permissible uncertainty limits of biological variation data [1]. Reference intervals were used to estimate biological variation.

Biological variation data as the basis for permissible uncertainty limits are generally accepted. These concepts usually apply a fixed factor leading to unrealistic stringent limits for quantities with a relatively small biological variation (e.g. plasma sodium) and to very permissive limits for quantities with relatively large biological variation (e.g. plasma triglycerides). To avoid these drawbacks, the working group has suggested a non-linear relation between biological variation and permissible uncertainty limits [1]. This proposal led to a compromise between the biological variation model and the state-of-the-art concept.

The new approach has been published in detail and exemplified with 84 quantities listed in the RiliBÄK [2]. The algorithms published allow to derive permissible limits for all quantitative measurands in laboratory medicine. The concept was based on intermediate imprecision. After its publication, three supplements appear necessary: 1. additional specifications of standard uncertainty, 2. a discussion on permissible limits for diagnosis and monitoring purposes, and 3. a discussion on circular reasoning in our approach.

Additional specifications of standard uncertainty

Recently, we published a proposal for permissible uncertainty of quantitative measurements in laboratory medicine [1]. One component of the uncertainty can be inferred from the statistical variation of the results of a series of measurements [category A of the Guide to Measurement Uncertainty (GUM) concept] [3]. In terms of laboratory medicine, it means the analytical imprecision and can be expressed by the experimental standard deviation. The term imprecision should be further specified.

Various international standard organizations [3–5] specified various types of standard uncertainty (imprecision) as summarized in Table 1. The maximum time span of repeatability is 1 day. Intermediate imprecision covers measurements from several days. The position of the control material in one run should always be the same, preferably before unknown samples are determined. If several control samples are measured during 1 day, only the first result should be included for calculating the imprecision. Further details have been published by Thompson [6].

The imprecision has several operational sources of uncertainty including, e.g. pipetting and temperature controlling, variations caused by operators (mainly calibration and maintenance) and manufacturer caused variations. The imprecision is expressed as standard deviation s or coefficient of variation CV. Evaluating s and CV, different lengths of series are used. However, both s and CV are estimations with confidence intervals decreasing with an increasing amount of repeats (Figure 1). For n=20 measurements, the additive effect of the confidence interval decreases to about 10% and the effect of additional repeats is small. The choice of n=20 is a practical compromise followed by many medical laboratories worldwide.

Figure 1:

Confidence limits (95%) for an estimated standard deviation (broken lines), obtained by Monte Carlo simulation (20,000 replicates per n value).

Limits were calculated as described in ref. [7], equation 6.102, p. 344). Horizontal axis: Number of values from which the standard deviation is calculated. Drawn line: mean estimated standard deviation.

Our proposal (1) was based on intermediate imprecision which is determined during one control cycle (e.g. 20 d×1=20 days, one measurement per day), with one instrument/analytical system and the same control material as also claimed by CLSI [5]. Several operators may be involved. Variations caused by operators are included, but variations caused by the manufacturer (e.g. between reagent lot variation) are excluded. However, they are tolerated as long as the permissible expanded uncertainty is not exceeded. If calibration and maintenance actions are undertaken only occasionally, the control cycle can be extended to, e.g. a 3-month period to include these effects on s. The longer series result in smaller confidence intervals estimating s. However, for series lengths above 20 this effect is small (Figure 1).

The quantities to be assured should be close to the most appropriate reference limits. That means more than one control material should be applied. The German RiliBÄK requires at least two materials for each measurand [2].

The imprecision as determined by control materials is an estimate of the actual imprecision designed for internal quality assurance purposes. It does not provide an estimate of the overall imprecision of the results received by the requesting physicians for materials taken from patients. For the latter purpose, the critical difference (also called reference change value) concept provides more meaningful information to the requester. Then, the uncertainty may be expressed as 1.64·(s_A*+s_I*)* [s_A analytical standard deviation, s_I intra-individual standard deviation].

If estimating imprecision, it is sometimes appropriate to eliminate suspect results which do not belong to the underlying distribution (outliers). Often, they are identified by subjective expertise. However, it is preferable to apply a statistical test. The literature contains a lot of more or less complicated outlier tests [8]. A simple and often used test was proposed by Youden [9]: elimination of all values outside mean±3s and recalculation of s (s=standard deviation). Probably, more reliable are the r-test [10, 11] or the Tukey Test [12].

Different permissible uncertainty limits for diagnostic and monitoring purposes?

Various concepts are suggested for uncertainty limits in laboratory medicine. Most proposals [13, 14] are based on biological variation (CV_B). If CV_B is multiplied by a fixed factor common to all measurements, quantities with a relatively small CV_B (e.g. plasma sodium) lead to a permissible analytical uncertainty (pU%) which is unrealistic low under the present technology, and quantities with a relatively large CV_B have a pU% which is too permissive (e.g. plasma triglycerides). Therefore, Fraser [14] proposed three different factors and we proposed even five different factors [15]. If the analytical uncertainty obtained by the majority of medical laboratories is presented in relation to CV_B, different factors lead to artificial jumps (as indicated by vertical black lines in Figure 2). An alternative is to use a curve with a continuous relation between CV_B and pU% [1, 16] as demonstrated by blue diamonds in Figure 2.

Figure 2:

The relation between empirical biological variation (CV_E*), the permissible extended combined uncertainty pU% (diamonds) and the permissible RiliBÄK requirements (RMSD, root mean square of measurement deviation, brown rectangles) [1, 16].

The straight blue lines represent the permissible uncertainty limits derived of CV_E* by fixed factors (lower line: 0.25·CV_E, middle line: 0.5·CV_E*, upper line: 0.75·CV_E*). CV_E* means CV_E calculated from ln values of the reference limits. For further explanation see ref. 1. The blue diamonds follow the equation pU%=2.39·(CV_E*–0.25)^0.5 [1].

So far, no consensus exists which biological variation should be applied to derive permissible limits for uncertainty. Different CV_B concepts have been described: CV_I means intra-individual variation, CV_G inter-individual variation, CV_C combined intra- and inter-individual variation, and CV_E empirical biological variation (CV_C and CV_A, analytical variation).

Several authors proposed that CV_I may be applied for monitoring purposes and CV_G for diagnostic purposes [14, 17]. Although this idea can be scientifically justified, it does not appear realistic under routine conditions. Usually most medical laboratories serve diagnostic and monitoring purposes. In most cases, the laboratory does not know for which purpose a test is requested. Therefore, only the monitoring purpose should be chosen because it is most restrictive. Then, the adequate CV_B would be CV_I. The relation between CV_I and pU% is also non-linear. In Figure 3, the circles show a similar curved pattern than the rectangles in Figure 2. The curved behavior is demonstrated by a local regression line in Figure 3. Because the curves in Figures 2 and 3 are oriented on the technical reality (technical feasibility), the permissible U% obtained is similar. In this case, both CV_B (CV_I and CV_E) can be used to derive permissible uncertainty limits.

Figure 3:

Intra-individual variation (CV_I, data taken from ref. 18) vs. the permissible expanded uncertainty (RMSD of the RiliBÄK, Table B1a, column 3 [2]).

The black line is the local regression.

We prefer to use the model based on CV_E for several reasons [1]: it realizes a compromise between technical reality and biological variation, considers imprecision profiles, considers normal and skewed distributions, takes advantage of the fact that laboratories are obliged to use reference limits for all quantities determined either by establishing their own reference limits or to verify the transferability of reference limits chosen from outside sources. According to existing guidelines, all reference limits must be validated and periodically verified [4].

Because permissible limits can be derived from any CV_B concept and should be oriented on the technical reality, the purpose for its application (monitoring or diagnostic) should be of secondary concern in our model.

Discussion of circular reasoning

A frequently raised critical comment on our proposal is that the combination of CV_A and CV_C in CV_E leads to a circular reasoning. Although this argument is correct, it can be neglected if the analytical variation is small in relation to the biological variation. This is the case for most measurands in clinical chemistry (Table 2).

A relative large empirical biological variation was encountered for h-choriongonadotropine (hCG) and C-reactive protein with a CV_E*=51.4 and a pCV_A=7.15 [16]. Assuming a CV_A=7.0, and CV_c =CV_E* and CV_C+pCV_A= (54.4²+7.0²)^0.5=51.9. The corresponding pCV_A value of 7.19 means a negligible increase (+0.5%).

With plasma sodium (a quantity with a relatively small biological variation), the increase of CV_E* is 7.6% due to the non-linear relation mentioned above (Table 2). However, the pCV_A derived of our model is equal to the permissible limit of the RiliBÄK [2].

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Research funding: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.