Quantitative laboratory results: normal or lognormal distribution?

Background

Quantitative laboratory results of healthy individuals usually show either a symmetric or a right-skewed histogram [1], [2]. The former may be described by a normal and the latter by a lognormal distribution, although real data probably never follow exactly ideal distributions in the form of normal, lognormal or other simple types of distributions. Nevertheless, in the spirit of the saying that “all models are wrong, but some are useful” attributed to the famous statistician George Box, it is reasonable to work with such idealized model assumptions. In the context of quantitative laboratory results, one might argue that it is worthwhile to consider more than just two options – normal vs. lognormal – for instance in the form of Box-Cox transformations. However, we demonstrate here that it is extremely difficult – if not impossible – to estimate the correct parameters for a Box-Cox transformation from mixed populations. This is the reason why we focus here on the decision between a normal and lognormal distribution. Evaluating the skewness – and consequently the distribution type – is easy, if the data set represents results of a homogenous cohort of healthy individuals [1]. It is, however, challenging under the conditions of a recent IFCC recommendation [3], where reference intervals are estimated from mixed populations.

The correct selection of the distribution type has an impact on the parametric estimation of reference intervals [1], [3], [4], as well as on other statistical laboratory tasks such as the calculation of permissible uncertainties [5] or the standardization of laboratory results [6].

The decision whether the data of a mixed population should be transformed or not, will always end in a vicious circle: Pathological values can be misleading for the choice of the transformation, whereas removing outliers before transformation can be too strict because in a skewed, e.g. lognormal, healthy population, non-pathological values could be classified as outliers. We belief that deciding on the transformation based on a method which is robust against outliers is a way to break the vicious circle. The focus of this paper is on the decision for the transformation, not the steps after the transformation, i.e. removing outliers and estimation reference limits. Nevertheless, we apply simple methods for reference interval estimation to generally demonstrate the usefulness of our approach but without the intention to discuss and evaluate different methods for reference interval estimation once the data have been transformed properly.

Unknown proportion of abnormal results [3], [4], [7]. To overcome the problem of the unknown proportion of abnormal results, a common suggestion is to apply a power transformation of the form x′=k·x^λ (with x′=log(x) for λ=0) making a skewed distribution more symmetric [3], [7]. The challenge here is, however, choosing the optimal exponent λ [7], [8].

Materials and methods

Results

Figure 1 illustrates the principle of our method, using simulated Na and ALAT concentrations as typical examples [14]: For normally distributed Na data, both skewness measures were about 0, whereas lognormally distributed ALAT data exhibited a Pearson skewness of 1.30 and a Bowley skewness of 0.17.

Figure 1:

Boxplots and histograms of 1000 simulated sodium (μ=140, σ=2.5) and ALAT values (μlog=2.6, σlog=0.49, shift=2.5).

The boxes extend from the 25^th to 75^th percentile (Q1, Q3) and the thick vertical line represents the median (Q2). If the distribution of the central 50% of the values is symmetric, the two halves of the box (Q2−Q1 and Q3−Q2) are equal and their difference – the numerator of the fraction – becomes 0. If the distribution is right-skewed, the right half will be larger than the left one, and the difference will become positive. The denominator Q3−Q1 (interquartile range, IQR) serves to standardize BS to a range from −1 to +1.

In order to demonstrate the robustness of Bowley’s skewness coefficient as compared to the classical Pearson skewness, we added one to 10 pathological outliers of 152 mmol/L to the simulated sodium data (Figure 1). Figure 2 shows that Pearson’s skewness exceeded 0.15 (red line), which is the upper limit of the 95% confidence interval for symmetry after addition of just three such outliers (0.3% of all values), thus leading to the erroneous assumption of a right-skewed distribution. In contrast, the quartile skewness remained at a level near zero and never exceeded the respective confidence interval of 0.08 for Bowley’s skewness.

Figure 2:

Addition of an increasing number of pathological results (152 mmol/L) to the sodium data of Figure 1.

The red line indicates the upper limit of the 95% confidence interval for the classical skewness of a normal distribution at n=1000.

To check the Bowley approach with real data, we applied the algorithm to routine blood counts, electrolyte concentrations and enzyme activities as shown in Table 1, and compared the results with the optimization of λ according to Emerson and Stoto (ES).

Taking a 95% confidence limit of +2.61000≅0.08 for BS as a cut-off for the detection of right-skewed data, we obtained the expected distribution types for Hb, Na (i.e. normal distributions) as well as for white blood cells (WBC) and ALAT (i.e. lognormal distributions). Surprisingly, K and Crea came out “intermediate” with respect to both sexes (Table 1, rows 4 and 5).

Figure 3 shows that the boxplots and density curves actually reflect the results of this analysis quite nicely: Na and Hb appeared symmetric or left-skewed for both sexes, whereas WBC and ALAT were clearly right-skewed. The boxplots for K in males and Crea in females, however, were not quite symmetric, and the corresponding density curves exhibited shoulders on the right side, which obviously raised BS above the limit of 0.08.

Figure 3:

Density and boxplots for routine laboratory results in women (pink) and men (blue).

Looking at these results in more detail, we observed in accordance with a publication of Haeckel and Wosniok [15] that taking logarithms of the original data had only a minor effect on Bowley’s skewness of most analytes in Table 1 except for WBC and ALAT. These latter analytes were the only two, for which we definitely expected a lognormal distribution [14], [16]. Therefore, we would like to introduce a slight modification to the aforementioned algorithm. Bowley’s skewness should be calculated from the original and the log-transformed data, and the difference between both skewness measures should be taken as a criterion with a threshold of 0.05 (Figure 4).

Figure 4:

Improved criterion using the difference of Bowley’s skewness for original and log-transformed data.

We chose a cut-off at 0.05 by visual inspection to ensure that Na, K, Hb and Crea come out as normal distributions across both sexes.

As to the method comparison depicted in Table 1, the proposed exponents of the ES method were close to 0 for WBC in both sexes, and close to 1 for Na and K in females, indicating the expected lognormal and normal distributions, respectively. Some λ values fell surprisingly far outside the interval of 0–1, reaching almost 20 for Na in men (as compared to 1 in women). If we assume that any markedly left-skewed distribution is due to pathological outliers on the left, we may set λ>1 to λ=1, in order to improve the agreement between the two methods. However, differences worth discussing remain for K in males and Crea in both sexes.

To investigate the practical application of our algorithm, we calculated reference intervals from original and log-transformed real data as described in the “Materials and methods” section. In a series of preparatory experiments, we determined the following optimal experimental conditions. Step 2 of the algorithm was repeated until no further outliers were detected. For the QQ plots in step 3, we calculated 39 quantiles with equidistant probabilities between 0.025 and 0.975. Linear regression lines were constructed from the central 27 dots, The intercepts were set to μ and the slopes to σ. From the results of the mclust function [12], we selected μ and σ from the subpopulation that made up the highest proportion of the Gaussian mixture. Figure 5 shows the results for Na (a typical normal distribution) and ALAT (a typical lognormal distribution).

Figure 5:

Comparison of distribution models in the context of reference interval estimation.

Green lines indicate the confirmation of expected results, red lines indicate unexpected density curves and QQ plots. Left column normal distribution,right column lognormal distribution. Upper panel (A) sodium, lower panel (B) ALAT.

The upper half of Figure 5 shows that, with regard to reference interval estimation, normally distributed analytes like sodium yield very robust results, irrespective of the distribution model and the method to fit that model. In contrast, the lower part demonstrates that lognormally distributed analytes like ALAT are susceptible to the distribution model chosen. The choice of the wrong model leads to a curved QQ plot and yields a very narrow reference interval with a too low upper limit.

Discussion

Our study shows that Bowley’s quartile skewness is a simple and robust method to classify normality vs. lognormality in mixed populations (Table 1). In the simplest version, the 95% confidence interval for the skewness of normally distributed data may be used to make a safe classification of normally distributed sodium [2] and hemoglobin [17] as well as lognormally distributed WBC [16] and ALAT [14] test results.

For K and Crea, the basic algorithm predicted different distributions for women and men, which were due to irregularities in the shapes of the density curves (Figure 3). Interesting enough, the distribution of potassium has been a matter of unresolved debate since the 1950s [2], [18]. This finding underscores the statement of Ralph Graesbeck [1] – one of the fathers of the reference value concept – that “laboratory results distribute as ‘nature feels fit’ and that parametric (curve-constructing) statistics only imitate the distribution with a function that allows calculation of desired informative indices”.

Nevertheless, we would like to suggest a slightly more sophisticated approach in order to avoid the aforementioned discrepancies. Our results confirm an earlier observation [15] that the shape of the density curves does not change very much upon log transformation if the data is normally distributed with a relatively small biological variation. Consequently, the Bowley skewness will not change very much either, and thus the differences should be close to 0. Choosing a maximal difference of 0.05 as a cut-off will lead to homogenous classification results for K and Crea (Figure 4).

Power transformations using optimized exponents have been suggested by some authors as alternatives to model such intermediate distributions more correctly [3], [7]. From our experience, these approaches can be helpful but can also be misleading: Especially with regard to the ES method [8], [10] tested here, some results do in fact fit while others are not plausible at all. This observation confirms an earlier finding that the ES method may behave poorly with skewed data [19]. So, given the fact that log transformation may be applied anyway without a great risk of false results (Figure 5 and ref [15]), the question should be asked whether it is worthwhile to determine a λ between 0 and 1 with complex and error-prone methods.

In a final series of experiments, we tested whether choosing an appropriate distribution model had an influence on the indirect estimation of reference limits from routine data. Investigating two representative examples (i.e. ALAT and Na) in detail (Figure 5), we could show that this is indeed the case for the lognormally distributed transaminase ALAT, whereas no difference was observed for the normally distributed electrolyte sodium. This again confirms the suggestion of Haeckel and Wosniok that “unknown distributions of clinical chemical quantities should be considered to be log-normal” [15].

It is noteworthy that the robustness against outliers of Bowley’s quartile skewness is essential to decide for the right data transformation. Because we assume that the decision for the transformation should be carried out before the removal of outliers, classical non-robust hypothesis tests for normality such as the Shapiro-Wilk or the Kolmogorov-Smirnov test will necessarily fail. For our examples, these tests for normality reported very small p-values (all of them except for WBC were much smaller than 0.007) both for the untransformed and the transformed data, leading to the rejection of the null hypotheses of normally and of lognormally distributed data.

As a side observation, our literature research revealed that the statistical distribution of laboratory results was an active research topic in the past century [e.g. 13, 16–18], while it is currently not in the focus of laboratory medicine. In the future, we expect an increasing interest in this topic again, e.g. in the context of big data applications and standardized storage of results in electronic patient records [6], [20], [21]. Our method opens the possibility to easily review the distribution of laboratory values on a large scale.