Separating disease and health for indirect reference intervals

Introduction

Diagnostic testing essentially aims to support the clinician in answering specific clinical questions. While many of these diagnostic queries are usually aimed at answering the question “Is disease present?”, the simplest interpretative tool provided on reports, the reference interval, has only an indirect relationship to identifying disease. The reference interval describes the confidence limits based on the typical values found in apparent health, or the absence of disease, and therefore only directly answers the question “Is health likely to exist?” Therefore, reference intervals identify the apparently healthy population and must be distinguished from ‘clinical decision limits’ that more appropriately identify the values in a specified diseased population. These are fundamental issues which are often poorly appreciated so the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) Committee on Reference Intervals and Decision Limits (C-RIDL) have recently published a position statement around this distinction [1].

Laboratorians may be familiar with receiver operator characteristic (ROC) curves that decrease the specificity for health against the sensitivity for disease (or vice versa), in order to define a cut-off that has intermediate specificity and sensitivity. It is important to acknowledge that ROC cut-offs are not reference intervals, which are, by definition, solely concerned with specificity for an apparently healthy reference population.

The object of this discussion is to address the methodological issues in deriving ‘indirect’ reference intervals. Direct reference interval studies aim to identify a population of healthy individuals ‘a priori’, and then describe the confidence limits of their distribution of results. Indirect reference intervals aim to identify the apparently healthy distribution of results ‘a posteriori’ from a database that includes both healthy and diseased individuals [2]. Whilst indirect approaches are relatively convenient and cheap to do, it is obvious that dealing with a mixed population is far more difficult than dealing with a homogenous healthy population, and therefore any tools that are used to separate healthy and diseased distributions must be understood, and applied with high degree of rigour, if a reliable indirect reference interval is the aim.

Indirect reference interval databases

While all indirect reference intervals are derived from a mixture of healthy and diseased individuals, the prevalence of disease in the database can vary considerably. Indirect reference intervals can be derived from cross sectional population studies, such as NHANES [3], [4], [5], which contain the background community prevalence of all diseases. The community prevalence of disease might be assumed to be low, however common diseases such as diabetes are often present in over 10% of the population in many developing and developed countries, and many studies find it necessary to exclude large proportions of the general population on the basis of supplied medical history and outlier observations. Clinical laboratory result databases from tertiary medical facilities, such as teaching hospitals, are at the other extreme of disease prevalence, including the high prevalence of disease amongst inpatients, as most patients have an illness as the reason for their admission. One of largest and most useful clinical databases for indirect reference intervals is that of primary care clinical laboratories. Ambulant ‘patients’ have will either have milder disease than their inpatient counterparts, or they don’t have disease and are having check-ups to monitor their risk of disease. These very large databases have disease prevalence that is closer to the background disease prevalence of general community.

There is an acknowledged international standard for defining reference intervals which is published by the Clinical Laboratory Standards Institute (CLSI) and endorsed by the IFCC [6]. The standard mentions that the best data for a posteriori studies would be from apparently healthy individuals who would by definition be asymptomatic and having health screening e.g. occupational screening or genetic screening or routine clinical screening before blood donations or minor surgical procedures.

Clearly, the higher the prevalence of diseased individuals in the database, the greater the effort required in separating healthy and diseased individuals. One of the main reasons that this may not be as problematic as it appears is that any particular test is not affected by all diseases. For example, diabetes does not usually increase prostate specific antigen (PSA) [7], nor are glucose levels related to prostate cancer mortality [8]. Thus tests which are unaffected by the disease (or its treatment) in a patient may be considered to represent a healthy state.

Strategies

The strategies for distinguishing healthy individuals from diseased individuals using the indirect approach are fundamentally exactly the same as for direct reference interval determinations. These strategies can either have a clinical or statistical basis.

The CLSI standard [6] states that the designation of good health for a candidate reference individual can include either clinical criteria such as history, physical examinations or laboratory criteria such as the results of clinical laboratory tests. As an example, to formally establish direct reference intervals for testosterone, we took clinical histories and examined 124 young men, including orchidometry, as well as performing semen analysis on all these men to a priori exclude testicular insufficiency [9]. I will discuss (below) the equivalent of taking a clinical history and examination or performing relevant clinical laboratory tests using the a posteriori indirect approach.

As well as seeking the clinical evidence from history, examination and testing, there is also a statistical strategy applied direct reference interval definition which includes applying parametric, or non-parametric, assumptions to the reference distributions and identifying statistical outliers. These are also vitally important considerations when defining indirect reference intervals.

Strategies can be both clinical and statistical and we will discuss how limiting the number of results from each individual reduces statistical bias as well as reducing clinical bias because diseased individuals often have more results.

Strategic pre-requisites for the indirect approach

Farrell and Nguyen [10] have pointed out that it is essential to check for pre-analytical and analytical artifacts prior to using any laboratory data. Pre-analytical factors, such as sampling time, sampling tubes and sample transport and storage prior to analysis must be understood and be consistent over time. Secondly, analytical factors such as assay manufacturer and calibration must also be consistent. This is relatively easy to check for as a review of monthly means [11] (or medians) over the course of the study should highlight any shifts in the data that need to be taken into account. Internal quality control and external quality assurance (EQA) will also give greater confidence in analytical stability as well as supporting the transference of any reference interval determined. Shifts of the result distributions are assumed to be due to analytical and pre-analytical factors, but can occasionally be due to population shifts, if the laboratory has a significant change in patient characteristics. The longitudinal trends in the data also have an extra advantage because circannual variations can often be seen as expected for vitamin D [12] and less expected analytes such as HbA_1c [13] and routine chemistry [14], [15]. It is therefore desirable that the data avoid circannual effects by having a database that includes multiples of a whole year. All laboratories could consider routinely auditing their result distributions, including flag rates, to confirm that their methods are stable as well as confirming that their reference intervals are still appropriate. I am often extracting data in response to clinical ‘complaints’ that they are seeing an increase in abnormalities. Similarly, if EQA reveals a potential shift in patient’s results, a review of result distributions and flag rates will usually reveal any significant concerns.

Clinical strategies for identifying underlying disease

Clinical history (and examination)

Optimal pathology request forms include clinical notes supplied by the clinician that include the clinical reasons for testing. These notes were ultimately derived from the observations elicited in the history and examination. Absence of clinical notes from a request form does not equate with absence of purpose for testing, simply that the purpose wasn’t conveyed to the laboratory. The nature of clinical notes clearly impact the abnormality rate of a test [16], [17]. Figure 1 shows two distributions of serum luteinising hormone (LH) levels measured in 40–45 years old women. One distribution is from women with regular periods indicated by the stated day of their cycle on the request form (e.g. Day 2 or 7 or 21), whereas the other was from women who were indicated to be possibly menopausal e.g. flushing. This demonstrates how data distributions can be selective for unaffected individuals simply using supplied request form clinical notes. Thyroid tests are usually accompanied with an indication of whether the patient is taking thyroid medication, and these patients should be excluded when deriving thyroid test reference limits [18], [19], but ideally linking with a pharmacy database may give greater confidence (than the limited information on the request form) in excluding patients on thyroid medications [20].

Figure 1:

Distribution of luteinising hormone (LH) results for 9,684 women aged 40–45 years, with the day of their period indicated on the request form (black line) compared to distribution of 4,295 women with clinical notes indicating flushing or possible menopause (grey line).

LH was measured on Roche Elecsys over the period 2010–2020 in a community based laboratory.

Some examination findings would be particularly helpful when deriving reference limits especially anthropometric measures that might indicate obesity, or blood pressure results. These are sometimes unavailable in the clinic, let alone in the laboratory, but some laboratories can collect these important measurements [21]. Obesity is a common cause for skewness for numerous common biochemical analytes [22], [23], [24], [25], and this is often a neglected issue for both indirect and direct reference limit estimation [26].

Other forms of clinical exclusion of indirect data include using stored medical and hospital diagnostic information [27]. For example, if trying to derive reference intervals for renal tests, it would be useful to exclude patients from renal wards or patients who were admitted or discharged with a diagnosis of renal disease. It will be difficult to enumerate and exclude all the possible clinical causes of a particular test abnormality [28] unless all that clinical information is captured. The Laboratory Information Mining for Individualised Thresholds (LIMIT) method systematically identifies ICD9 disease codes associated with extreme results and then excludes individuals having these codes in an iterative manner [29]. It is logical that patients with hospital ICD9 codes of anaemia, blood transfusion and surgical procedures are easily identifiable as populations with a large proportion of outlier haemoglobin levels, so they can be excluded.

Clinical laboratory tests

Tests are not only performed for diagnostic purposes but also for screening and monitoring, therefore analysis of laboratory data needs to be conducted with awareness of these separate contexts in order to identify and mitigate potential biases in the data [30]. Patients having multiple tests, are more likely to be monitored for disease, therefore excluding patients with numerous results (e.g. more than three) [31], or including only a single result per patient (typically the first) [26], or even more thoroughly, limiting only to individuals having a single test in the study period [32]. Figure 2 shows the increasing likelihood that adults between the age of 18 and 65 years will have repeat measurements of serum creatinine within 12 months, when the initial creatinine lies outside their respective gender related reference interval. Patients having follow up testing due to previous abnormalities can also cause irregular distributions which is another strong reason to exclude them [33].

Figure 2:

Relative likelihood of repeat testing within 12 months of serum creatinine against the initial creatinine result for 530,000 women (light coloured line) and 430,000 men (dark coloured line).

The creatinine reference interval for women (Roche enzymatic method) is 45–85 μmol/L for women and 65–105 μmol/L for men.

Co-requested tests are a particularly useful strategy for the indirect approach. It is logical to insist that patients selected for potassium reference interval determinations do not have a low estimated glomerular filtration rate (eGFR) indicative of renal failure. Selectivity can be increased by using the results of multiple tests, such as when deriving an indirect reference interval for parathyroid hormone (PTH), where not only renal function (eGFR) should be normal, but also that calcium and vitamin D levels should be normal [34]. Similarly, tests that are affected by inflammation could be selected from patients that have normal co-requested C-reactive protein (CRP) levels. Figure 3 shows that patients with elevated serum CRP skew the results for serum alpha-1-antitrypsin (A1AT) and therefore using only A1AT results from patients with a normal CRP give a more healthy distribution. This may rely on a clinical understanding of the associations of all tests with disease or a learning strategy from the statistical evidence that the analyte of interest is correlated with other measures of disease [35]. A similar learning strategy is to exclude all patients having repeated co-requested tests within a short time interval because repeated testing, regardless of the results, is more likely to be associated with disease [36].

Figure 3:

Gaussian distribution of alpha-1-antitrypsin (A1AT) levels in 1,125 patients with normal C-reactive protein (CRP) (≤5 mg/L, light bars) compared to higher levels of A1AT in 475 patients with raised CRP (dark bars).

CRP and A1AT were measured using a Roche immunoturbidimetric method.

Statistical strategies for identifying underlying disease

The dominant statistical strategy for isolating indirect reference intervals is that all reference distributions are Gaussian, or log Gaussian. I will also address my cautionary view regarding the assumption of other distributions. Non-parametric approaches are uncommon and include Piecewise Regression that operates on non-parametric percentile analysis to identify partitioning ‘break points’ which are assumed to reflect normal physiology, but therefore rely heavily on clinical disease exclusion criteria [5].

The confidence that Gaussian distributions explain reference intervals may seem like a divine belief in the beauty and symmetry of the Gaussian curve, however there is something fundamentally ‘natural’ in the Gaussian curve which reflects the diversity of nature. If we explore the distributions of chance, moving from binomial distribution (e.g. tossing a coin), to multiple determinants of chance (e.g. tossing 20 coins), the distributions gradually become Gaussian [37]. Therefore a Gaussian reference interval reflects the many factors (usually >>20) that cause variation of results between healthy individuals. This is no different to what laboratory scientists intrinsically understand regarding analysis, where measurement uncertainty follows the Gaussian distribution because there are numerous determinants that affect every step in analysis including calibration errors, sampling errors, lot to lot reaction variations and reading errors. Even when measurement uncertainty is very small, it follows a Gaussian distribution.

The Gaussian distribution of results from a reference population includes not only the Gaussian analytical variation in each result, but also the intraindividual and interindividual variations which are, according to the assumptions of biological variability, also Gaussian [38]. Intraindividual changes have a multiple determinants including hydration status, diurnal variation, circannual variations, as well as the various effects of daily diet, activity and stress. Interindividual differences are as varied as our unique physical appearances and height, weight and body mass index (BMI) were some of the historical measures demonstrated by Quetelet to follow Gaussian distributions [39].

Hoffmann vs. Bhattacharya strategies to Gaussian resolution

Hoffmann recognised that plotting a cumulative Gaussian distribution on probit (standard deviation) based graphs results in a straight line [60] and this became a popular way to define that mean (centre of the Hoffmann line) and standard deviation (slope of the Hoffmann line). This method is clearly influenced by the presence of an overlapping population, and the prevalence of an interfering distribution gradually invalidates both the mean and SD estimates of this approach. Figure 4A illustrates two hypothetical overlapping serum creatinine distributions (men and women). Figure 4B shows their corresponding Hoffmann lines for each gender as well as the combined population, and shows that the combined Hoffmann line has lost its relationship to the underlying distributions. Figure 5A shows some real serum creatinine data in young adults (20–29 y/o) where the right skew of the overall distribution is actually due to the identifiable gender based underlying distributions. While the individual Hoffmann lines in Figure 5B do a reasonable job at identifying each gender based distributions, the Hoffmann line for the combined distribution does not reflect the underlying data. Overlapping populations effectively cause the Hoffmann reference interval estimate to be too narrow and must therefore be separated, and this necessity was identified soon after the technique was originally proposed, and solutions using an iterative process were developed [61]. Rather than using probit based graphs, modern applications of the Hoffmann method can be performed by data mining using large databases but these may also include modifications to the original Hoffmann method which introduce sophisticated iterative outlier exclusion algorithms, as well as further sophistication in the iterative assessment if the linear region [62].

Figure 4:

Hypothetical distribution of serum creatinine in men (dark grey), women (light grey) and the combined distribution of men and women (black line).

Panel (A) also shows the Bhattacharya function for each distribution and the combined distribution, while panel (B) shows the Hoffmann function (see text for further explanation).

Figure 5:

Distribution of laboratory serum creatinine in young adults between 20 and 29 years of age; men (dark grey), women (light grey) and the combined distribution of men and women (black line).

Panel (A) also shows the Bhattacharya function for each distribution and the combined distribution, while panel (B) shows the Hoffmann function (see text for further explanation).

The Hoffmann technique is therefore usually simplistic and relies heavily on the initial exclusion of overlapping distributions. This is not always appreciated and modified Hoffmann techniques are often misused [63], and can be demonstrated to be inferior to the alternative common technique of deriving the underlying distribution – the Bhattacharya method [64].

Bhattacharya described a statistical approach of resolving a distribution into its Gaussian components in 1967 [65] by recognising that the slope of a Gaussian distribution (from left to right) rises but soon gradually falls toward zero slope at the zenith, then becomes increasingly negative before it dwindles away. The slope changes logarithmically, so the logarithm of the slope forms a straight line – as long as that distribution is not interfered with by an overlapping distribution. Figure 4A also includes the Bhattacharya line for the hypothetical male and female serum creatinine distributions, and show that the Bhattacharya line is the same for the combined distribution as the individual distribution when there is negligible overlap. Figure 5A shows the Bhattacharya lines for the real serum creatinine data, and shows how the Bhattacharya line for the combined distribution overlaps with the individual distributions where the overlap is negligible. The slope of the Bhattacharya line again correlates with the standard deviation of the slope, but the mean is the point at which the slope function would be zero. The selection of this linear portion of the line can be subjective, however objective statistical approaches can be applied [66].

One of the limitations of the Hoffmann and Bhattacharya methods, to date, is that while the correlation coefficient of the line of best fit provides a statistical measure of confidence in the proposed distribution, a routine method of estimating the confidence in the reference limits has not been published. A related challenge in the Bhattacharya method is the subjective choice of bin size and location for the frequency histogram which may influence the final result. This is particularly an issue when data has been rounded and/or transformed and to minimise both these issues, the data should start with as many significant figures as possible [67]. When large data sets are available, adjustment of the bin locations should not significantly affect the results.

The Bhattacharya technique was employed in clinical biochemistry within three years by Gindler [68] and refined by Baadenhuijsen and Smit in 1985 [69]. Online resources have been available [70], [71]. Because the Bhattacharya line is derived only from the unaffected part of the combined distribution, it is unaffected by outliers including the presence of disease, as long as unaffected linear part of the Bhattacharya line (which defines the reference population), represents at least 40% of the total population [67]. It is therefore useful to apply clinical exclusion techniques before using Bhattacharya for most analytes to increase the proportion of the combined distribution that represents the reference population.

If a distribution is close to symmetrical, it is generally true that the centre (mode) of the observed indirect distribution will be the mean. In any case, the possibility of overlapping populations increase as you move away from either of these central measures. This also depends on whether disease causes an increase or decrease. If disease causes increases e.g. alanine transaminase (ALT), then the lower side of the distribution is largely unaffected and the reference distribution can be estimated using results in the lower side of the distribution. Conversely, if disease generally causes lower results (e.g. Hb), then the upper part of the distribution can be used to estimate the underlying reference distribution. This is essentially the basis of the Pryce method [72]. The more challenging situation is when disease causes both increases and decreases (e.g. serum sodium), so we lose confidence as we move in either direction from the mode. Despite the relatively high frequency of results around the centre of the distribution, an increased reliance on this narrow span of data, is what usually makes it necessary to have large databases.

Advances in indirect reference interval estimation does not necessarily assume that the truncated central part of the distribution is unaffected by disease [73]. The ‘truncated minimum chi-square’ (TMC) approach starts with a Hoffmann estimate from this part of this distribution, then uses an iterative approach to define an assumed power normal distribution (including Box–Cox adjustment) using the data both within and outside the apparently unaffected interval [74].

I’ve been fortunate to work in ‘mega’ laboratories testing over 10,000 patients per day, which have access to very large databases. These private laboratories focus on primary healthcare and have provided the opportunity to define many indirect reference intervals from tumour markers [75], [76], [77], [78], hormones [79] to routine biochemistry [80], [81], [82] and routine haematology [83]. In my experience, using these very large databases, clinical exclusion is not as important as statistical exclusion because I use a statistical technique (Bhattacharya) that is least affected by outliers and overlapping disease populations. Others have also concluded that some indirect methods can work well in unselected populations [84].

Table 1 compares the clinical and statistical approaches to separating health and disease. The clinical approach is attractive to those that are clinically focussed and sceptical about how statistical methods might be able to separate subpopulations. The clinical approach does rely on the availability clinical information as well as an understanding of the effect of disease(s) on each laboratory test. The statistical approach can be applied to laboratory data without clinical information of understanding, however the complex statistical methods usually require very large databases to locate the small span of data unaffected by disease. While I personally prefer a combined approach, this requires both very large databases as well as a demonstrated consideration of available clinical information. I find the extra effort useful as it specifically quantifies the pathological impact of disease (which may not be known) at the same time as considering the relative impact of human physiology. In simpler terms, I can demonstrate that young adults have less disease, at the same time as demonstrating they are physiologically different to the elderly.

Overlap and skewness due to overlap of healthy partitions

Just as importantly, a heterogeneous distribution may comprise overlapping partitions of healthy individuals described by physiological variations. Figure 6 shows that the skewed distribution for serum bilirubin could be interpreted as a logarithmic Gaussian distribution or a series of overlapping normal Gaussian population. The latter is more likely given that overlapping bilirubin distributions more appropriately reflect the common and distinct genetic polymorphisms in the UGT1A1 promoter [85]. It is highly desirable not to broaden reference intervals by ignoring important physiological distinctions as this may undermine the primary purpose (specificity) of reference intervals and in the case of serum bilirubin, this will lead to incorporation of Gilbert’s syndrome into the common reference interval. While it could be argued that this may be appropriate, it should be done consciously rather than accidentally.

Figure 6:

Distribution of serum bilirubin values in young men between 20 and 29 years of age.

Panel (A) shows the log Gaussian solution to the skewed distribution whilst panel (B) shows three overlapping normal Gaussian populations as the solution to the distribution.

Investigating partitioning can be just as relevant as generating the reference interval in the first place [86]. In my experience, overlap and skewness due to physiological heterogeneity is a more pressing issue than overlap or skewness due to disease and indirect reference interval techniques are a convenient tool for exploring physiological variations. Understanding physiology is therefore a pre-requisite to understanding reference interval partitioning [87] and the prevalence of physiological subgroups in the reference population should be known and observed in the calculations for every reference interval study [88]. There are very few (if any) measurands that are the naturally same across childhood and adulthood let alone gender and pregnancy. Concordet et al. [89] have indicated that complex techniques have several weakness, including separating young and old individuals rather than healthy and diseased ones. If indirect reference intervals are determined that do not demonstrate known physiological changes, then be very cautious about accepting that the investigators have accounted for the overlap due to disease.

Sometimes the differences between healthy groups seem small and a decision needs to be made about whether the difference is significant enough to justify a new partition. Clinical justification of partitioning should be related to the potential change in clinical specificity of the derived reference limits. Normally, a 95% reference interval will have a 2.5% false positive rate above the 97.5th centile and 2.5% false positive rate below the 2.5th centile. Harris and Boyd developed a statistical method for identifying partitions but observed that abnormal rates of less than 1.0%, or greater than 4.0%, deviate too much from the ideal value of 2.5% [90]. Lahti et al. [91] have more stringently determined that if more than 4.4% are flagging (rather than 2.5%), this justifies a new partition. Conversely if less than 3.3% are flagging, then partitions should be combined because the flag rate is not confidently different from 2.5%.

Conclusions

While direct reference intervals should be preferred, they have several limitations. They seldom account for all the major pre-analytical factors that affect test results such as age, gender and ethnicity let alone the minor factors such as diet, exercise, smoking, drugs of abuse including alcohol and medications [98]. Furthermore, methods vary between manufacturers and any analytical improvements also require reassessment of the appropriateness of previously defined direct reference limit.

The indirect approach can address the very difficult partitions across paediatrics, pregnancy and the elderly where direct reference intervals have even greater complexity and cost. The indirect approach is not only convenient and cheap but also operates on data from the ‘real world’ of clinical practice. The pre-analytical factors of collection and transport in the reference population match routine processing. Reference intervals should be derived from a reference population as close as possible to the populations being assessed [99] and patient traits including e.g. diet and exercise should be similar to the reference population [100]. A patient’s comorbidities (e.g. obesity) might be present in the reference population [101]. Perhaps hospitalized patients, who are generally stressed and supine, be interpreted against reference intervals developed from healthy ambulatory populations or against unaffected hospitalised patients [28]. These often poorly defined case mix factors are considered to be flaws of the indirect reference interval approach [102], but I would agree with Barth that these are significant advantages [102].

Summary

Indirect reference intervals are cheap and convenient and can address the many aspects of reference interval derivation that are hard to address with expensive direct reference interval studies. The two principles of clinical exclusion of diseased individuals from the reference population, combined with statistical identification of overlapping populations and outliers must always be considered. The use of advanced statistical techniques that focus on the dominant reference population, and are least affected by the presence of overlapping populations, may be more important than clinical exclusion. The value of indirect reference intervals should be demonstrated by their ability to identify the numerous physiological variations already present in laboratory data, that direct reference intervals seldom accommodate.