Equivalence limits of reference intervals for partitioning of population data. Relevant differences of reference limits

Introduction

The distribution of population data may differ between subpopulations, either due to analytical systematic and/or random errors, to biological differences (ethnic, gender, age, nutritional or regional differences) or to both effects. Larger analytical or biological variations always lead to broader reference ranges than smaller variations. Differences between corresponding reference limits (RLs) should be tested to justify partitioning or non-partitioning. Partitioning is a valuable tool to improve the diagnostic efficiency of laboratory examinations. The RLs are either determined by the laboratory (intra-laboratory RLs) or taken from external sources (extra-laboratory RLs). RLs can be determined by direct or indirect approaches [1] and should be verified. RLs can be verified by comparing two RLs with each other determined at two different situations (A and B).

Reference limits need to be compared with each other for two main purposes: to evaluate the clinical relevance of observed differences if reference limits are derived consecutively in time from a presumed identical population or, if subpopulations are compared, to check if the limits can be considered as identical. If not, the total population should be partitioned (stratified) in adequate subgroups (e.g. males and females).

The Clinical Laboratory Standard Institute has recommended relating the difference between means of two subgroups to the biological variation of the subsets in order to decide about partitioning [2]. This approach was based on the concept of Harris and Boyd [3] who suggested calculating the statistical significance of the difference between subgroups means by the standard normal deviate test. This approach, however, requires a normal distribution of the data set.

The proposals of Lahti et al. [4, 5] hitherto are the most advanced procedures for Gaussian and non-Gaussian distributed population sets to investigate the necessity of partitioning. These proposals base on limiting the proportion of the subgroup distributions outside the common reference interval (rate of false positive results, FPR). Usually, the reference limits are defined by the central 95% fraction of a non-diseased subpopulation. In the case of two subpopulations, four FPR values (two at the lower and two at the upper end of the two distributions) are obtained. If the subpopulations have identical distributions, the common RLs are identical with the subgroups RLs and, therefore, all FPR have the value of 2.5%. Otherwise, the FPR differ from this value. Gellerstedt and Hylthoft Petersen modified the proposal of Lahti et al. [4, 5] for several subpopulations by means of hierarchical cluster analysis [6]. In this case, some subpopulations may form a group or cluster that can share a common RL. This proposal appears complicated, depends heavily on the details of the clustering procedure and might lead to keeping two very different sub-populations that happen to be similar in the sense of the cluster analysis.

The approaches mentioned above do not answer a frequent question: are intra-laboratory RLs comparable with RLs determined at an earlier occasion or taken from an external source. Transferability and periodical reviewing is required by international recommendations [2, 7]. In this report, a general approach is proposed for the comparison of two RLs. It is based on the concept of equivalence limits (EL) which finds its application when comparing two regression lines in method comparison studies [8]. ELs are solely based on the permissible analytical standard deviation (ps_A).

Proposal

If two RLs of the same measurand are compared with each other, the crucial question is: can the difference be explained by analytical uncertainty or is it due to additional biological causes? If the difference is explainable by the uncertainty of the analytical procedure (pu), no partitioning is required.

Two sets of lower and upper reference limits, denoted by RL_1A, RL_2A, RL_1B, RL_2B (suffix 1: lower limits, suffix 2: upper limits, suffix A: first subgroup, suffix B: second subgroup) are involved when comparing RLs from two sources. Typically A is the subgroup with established RLs (already used by the laboratory) and B is the subgroup from which an RL is newly determined. Some examples for underlying data distributions are shown in Figure 1. A useful graphical characterization of the RL positions is obtained by joining the points (RL_1A, RL_1B) and (RL_2A, RL_2B) by straight lines in a coordinate system which shows the RLs from subgroup A on the horizontal axis and the RLs from subgroup B on the vertical axis (Figure 2). If RLs from subgroups A and B are identical, then this line lies completely on the diagonal (y=x). Otherwise, the line lies somewhere else, depending on the underlying data distributions in the subgroups. The examples shown in Figure 1 correspond to lines I, II and III in Figure 2. Partitioning criteria must consider all possible positions of these lines. A band around the diagonal limited by an upper and lower equivalence line (red lines in Figure 2) defines the permissible range of the (RL_2A, RL_2B)–(RL_1A, RL_1B) lines.

Figure 1:

Distribution patterns of two population subsets of which the reference limits are to be compared.

Green broken line: lower RL of distribution (procedure) A, RL_1A; blue broken line: lower RL of distribution (procedure) B, RL_1B; violet broken line RL_2A; red broken line: RL_2B. Blue line: distribution A, red line distribution B. The axes are given in arbitrary units.

Figure 2:

Regression lines between the lower and upper reference limits.

Vertical black lines represent the reference limits of procedure A (abszissa). The scales of the two axes are identical with arbitrary units of the two analytical procedures (A and B) or of measured quantities obtained with the same procedure at different time intervals (A and B). The units differ of those in Figure 1. The red lines are the upper and lower equivalence limit lines. The blue, broken regression lines I (Figure 1A), II (Figure 1B) and III (Figure 1C) correspond to the examples shown in Figure 1.

The equivalence lines are based on the permissible analytical uncertainty of procedure A (pCV_A). The multiplication factor for pCV_A was chosen arbitrarily as about 1.3 because it appeared to provide plausible permissible limits. This factor happened to correspond to the 90% quantile of the normal distribution (one-sided confidence interval of about 90%). Then, the upper equivalence line in Figure 2 is defined as

and the lower equivalence line is

In these equations, x_i represents values from subgroup A, y_i represents values from subgroup B, and pu denotes the permissible uncertainty of x_i. In the theoretical example of Figure 2, the permissible analytical imprecision at RL₁ is lower than at RL₂. The equivalence lines are analogous to equivalence limits proposed for method comparison studies [8].

Assuming that RL_A and RL_B are determined without bias or with the same bias, the permissible uncertainty pu can be reduced to the analytical standard deviation ps_A (pu=ps_A). Bias may be considered if RL from external sources are compared with each other and the bias of both analytical procedures is either unknown or presumably different. Then, pu may become 1.22·ps_A as recently proposed [9].

For simplification, instead of equivalence lines, only both ends of a reference interval (RI) are tested independently. If bias can be neglected, the absolute difference D₁(=|RL_1A–RL_1B|) between the two lower RL (RL_1A and RL_1B) should not exceed a permissible difference (pD₁)

and the absolute difference D₂ between the two upper RLs (RL_2A and RL_2B) should not exceed a permissible difference

Figure 2 shows the permissible differences. pD₁ is the distance between the intersection of the vertical RL₁ line with the blue diagonal line and the intersection of the RL₁ line with the upper or the lower equivalence line. Analogously, pD₂ is the distance between the intersection of the vertical RL₂ line with the blue diagonal and the intersection of the RL₂ line with the upper or lower equivalence line.

In the proposed approach, the length of the measurement series for estimating the analytical variation is assumed to be n=20 and that the effect of larger series lengths can be neglected [10]. According to international recommendations [7, 11], laboratories should know their permissible uncertainties. If this is not the case, ps_A values can be calculated (equation 10 in the Appendix) from the corresponding pCV_A values taken from published sources, e.g. by Ricos [12] or by Haeckel et al. [9]. The latter list may be preferred because permissible limits can be calculated for any quantity of a measurand based on the biological variation. A list with pCV_A,Xi values for 84 measurands is available from the home page of the German Society for Clinical Chemistry and Laboratory Medicine [13]. This list is part of an Excel platform for the calculation of permissible uncertainty of any quantitative measurand. The algorithms for calculating pD₁ and pD₂ (equations 3 and 4) are also included. The interested reader finds a summary of the corresponding algorithms in the Appendix.

Results

Some practical examples of permissible differences (pD%) are listed in Table 1. The examples shown in Table 1 are compared with limits suggested by CSLI [2]. The CLSI recommendation is based on a proposal of Harris and Boyd [3]. It is assumed that s₁=s₂ and d=mean₂–mean₁. This approach considers only the distance between mean values, their standard deviations and depends on the number of contributing values (see Appendix). In most cases, the permissible differences were higher than those from our proposal. With sodium, the permissible difference of CLSI (0.70%=1.02 mmol/L at 145 mmol/L) is probably too stringent for most laboratories. Whereas for quantities with a relative large biological variation, e.g. aspartate aminotransferase (AST), triglycerides and thyreotropine (TSH), the pD% of CLSI were much higher than can be achieved by the present technology. At the upper reference limits, the new proposal leads to pD% values of 1.59% for sodium (pD=2.3 mmol/L), 6.7% for triglycerides (pD=0.09 mmol/L) and 7.4% for TSH (pD=0.2 mU/L) which appear plausible under the present technologies.

The applicability of the permissible differences presented in Table 1 is demonstrated by two practical examples: plasma creatinine and AST. An upper RL for creatinine of 104 μmol/L has been found with men and of 88 μmol/L with women [14]. This difference of 16 is above the pD of about 5 μmol/L reported in the table. Therefore, stratification according to gender can be justified. The upper RL of AST for women is 35.0 U/L. If a laboratory finds 37.0 U/L, the difference is not relevant, because the difference of 2.0 U/L is less than the permissible difference of 2.3 U/L.

The calculations behind the results of Table 1 can be performed on the Excel platform. The Excel platform with the algorithms required can be obtained from the authors or from the homepage of the DSRL [13] gratuitously.

Discussion

Differences between two RLs derived of two population subsets can have different components:

1. biological differences of the distribution pattern (differences in shape, in modes, in variation due to age, gender, race or other genetic causes),

2. analytical differences. The two RLs may be determined by analytical procedures with different imprecision and/or with different bias.

If analytical reasons for an identified difference can be found (e.g. bias), the reason may be eliminated and the testing repeated afterwards. If analytical reasons can be excluded and biological causes are suspected (different sex or age in a population examined under the same analytical conditions), stratification is recommended. For the purpose of combining RLs (to create common RLs), only analytical variation may be acceptable (within specified limits). All other factors should be either excluded (if detected) or, if suspected, are reasons for stratification. Only if the difference between RLs is higher than can be explained by analytical causes, RLs should be stratified.

Differentiation between analytical and biological causes may not be easy. Therefore in practice, RLs of one subset should not be modified by using additional data from another subset more than would be allowed for analytical reasons only (in the absence of biological reasons). If an inacceptable change occurs, a significant biological reason is assumed, and/or the random error or the bias inherent in one RL is too large to justify combining the two subgroups. Although the relevance of D in equations 3 and 4 is based on pCV_A, it may also include small differences in biological variation. pCV_A is taken from Ref. [9], as also explained in the Appendix, and is related to the biological variation and preferably to the established RL (procedure A) or to the RL with the lower value.

In equation 1 and 2, a factor of 1.28 was chosen corresponding to the 90% quantile of the normal distribution. This confidence interval could be changed by applying a higher factor, e.g. 1.64. This would increase the confidence interval to e.g. 95% with the disadvantage of higher rates of false negative results. The 90% confidence interval for equivalence limits is analog to the recommendation of IFCC [15] for confidence intervals of reference limits.

Benefits of the present proposal in comparison with other concepts [2, 4, 5]: it

1. considers biological variation;

2. is prevalence independent;

3. does not require approximately equally large sub-populations;

4. can be used for stratification purposes and for comparing two RLs which are suspected to emerge from the same population. The latter purpose fulfills the requirements for periodic reviewing of RLs as, e.g. postulated by the former ISO standard 15189 [7];

5. can be applied if only one (lower or upper) RL of the RI is known. However, the indirect model developed by a working group of the German Society for Clinical chemistry and laboratory medicine (DGKL) always yields both ends of a reference interval and can be used [13].

Conclusions

The model proposed can simply be applied by equations (5) and (6), and by taking permissible CV_A values from already published lists.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Research funding: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.