Automated sex and age partitioning for the estimation of reference intervals using a regression tree model

Introduction

Reference intervals (RI) are an important information for the interpretation of laboratory results. It is the responsibility of medical laboratories to validate the existing reference intervals or to establish their own. According to the CLSI/IFCC standard [1] RI are usually estimated from the central 95 % interval of a healthy population, a method known as the direct RI estimation. Values outside of this interval are classified as decreased or elevated.

An essential issue in determining RI is understanding the factors that influence variations in analyte concentrations [2]. Age, sex and ethnicity have a major impact on biology and are therefore important factors influencing the RI of laboratory parameters. In addition, different analytical methods can lead to deviating RI. For the estimation of RI, a minimum of 120 reference subjects is recommended by the CLSI guidline [1]. However, this was later considered as too low [3]. In principle, the number of individuals is required for each subgroup like sex or age cohort. This results in a considerable number of healthy individuals who must be included in the process to establish direct RI.

One of the major challenges in establishing RI is the partitioning in subgroups. While this is relatively straightforward for sex, it becomes more complex for ethnicity and particularly challenging when dealing with the categorization/binning of numeric variables such as age. Partitioning by age groups is a crucial method for enhancing the diagnostic value of RI [4]. Currently, there are three basic principles for partitioning:

1. Partitioning by social categories (e.g., newborn, infant, adolescence, adult) [5].

2. Partitioning based on data and its biochemical and medical context. This is the most commonly used approach to date [1, 4, 5].

3. Newer and sophisticated approaches with continuous RI [6], [7], [8], [9].

While partitioning by social categories simplifies the classification of values for laypeople, it poorly represents physiological processes, especially in childhood [5]. On the other side, continuous RI have the greatest potential to accurately reflect biological backgrounds. However, the validation and use of continuous RI are currently not covered by normative documents [7]. Therefore the partitioning based on the data with its biological background can be considered as the current standard, which is also used in this work and the CALIPER (Canadian Laboratory Initiative on Pediatric Reference Intervals) study [10].

The aim of this study is to evaluate automated and therefore potentially unbiased tools for age- and sex-dependent segmentation. To that end, the authors chose to utilize decision tree algorithms since they display several properties which are useful when using machine learning-tools in a medical context:

1. they are robust to outliers, which makes the results more reliable,

2. they are easy to visualize and understand even without any specialized knowledge besides the ability to read flowcharts, and

3. they are by default non-opaque, which helps to interpret the results on the basis of medical knowledge (so-called explainable AI).

Materials and methods

Results

Figures 1 to 4 present a visual summary of the results obtained for the four analytes: ALB, GGT, CREA, and AP, respectively. Each Figure is subdivided into three parts. Subfigure A displays the age and sex partitions recommended by CALIPER alongside the RI estimated using reflimR. Subfigure B shows the same data utilizing the partitioning suggested by rpart. The RI are shown as boxes for girls in red, boys in blue, or for the combined sexes in black. RI calculated as percentiles (in the case of small sample sizes) are indicated by dashed lines. Subfigure C illustrates the outcomes of the rpart algorithm in the form of a decision tree. For each split, the response variable value of the model for respective node or leaf is displayed together with the number of observations and the percentage of the total observations. The underlying absolute numbers for reference limits and age partitions can be found in Tables 1 and 2. The results of the comparisons of specific partitions using the Harris & Boyd method are shown in Table 3.

Figure 1:

Analysis of albumin (ALB). (A) RI of partitions suggested by CALIPER for ALB. (B) RI for partitions suggested by rpart for ALB. (C) Flow chart for age- and sex-dependent reference intervals for ALB.

Figure 2:

Analysis of γ-glutamyltransferase (GGT). (A) RI of partitions suggested by CALIPER for GGT. (B) RI for partitions suggested by rpart for GGT. (C) Flow chart for age- and sex-dependent reference intervals for GGT.

Figure 3:

Analysis of creatinine (CREA). (A) RI of partitions suggested by CALIPER for CREA. (B) RI for partitions suggested by rpart for CREA. (C) Flow chart for age- and sex-dependent reference intervals for CREA.

Figure 4:

Analysis of alkaline phosphatase (AP). (A) RI of partitions suggested by CALIPER for AP. (B) RI for partitions suggested by rpart for AP. (C) Flow chart for age- and sex-dependent reference intervals for AP.

For ALB, rpart suggests two very simple partitions, which only consider age (see Figure 2). These the splits occur at approx. one and ten years. In contrast, CALIPER suggests a sex-dependent subdivision from 15 years onwards, with male individuals having a much broader scattering than female individuals. As a result, in older age groups, the CALIPER reference intervals cover more data points than rpart, although when comparing the intervals, the Harris & Boyd method again do not indicate a significant difference between the corresponding rpart partition and the two CALIPER partitions for boys and girls.

The analysis of GGT also reveals very similar results with both approaches. Significant age splits are seen at about one month and one year, and again CALIPER suggests an additional split at an age of 11 years, which is missing in rpart. According to the Harris & Boyd method, there is no statistical need for this subdivision.

For the remaining age progression, rpart assumes a constant reference interval for both sexes, while CALIPER suggests an additional split at 11 years with a particular effect on young males. According to the Harris & Boyd method, there is no significant difference between the partitions suggested by rpart or CALIPER, even if the CALIPER proposal looks somewhat closer to the blue dots on the right-hand side of the graph.

Both approaches suggest quite similar partitions for CREA, albeit with greater complexity of age partitioning. This is due to the fact that creatinine levels (in contrast to ALB and GGT) increase more or less continuously from infancy to adulthood, which is difficult to depict using constant age groupings. In view of this challenge, the position of the age splits proposed by rpart matches that of CALIPER remarkably well. For children older than 13 years, rpart suggests only one partition for girls but two for boys with a further subdivision at 15.5 years. CALIPER, on the other hand, suggests an additional sex-independent partition and then only divides into separate partitions for boys and girls from 15 years onwards. Again, significant differences can be observed between the individual segments within both approaches, with no significant differences between the two approaches in the Harris & Boyd analysis.

AP is the most complex analyte examined here. It is therefore not surprising that the clearest differences between the two approaches can be seen here. The complexity is primarily due to the strong age and sex-dependent kinetics of AP. In the first year of life, there is initially a significant increase in AP with considerable variability, followed by a rapid decline to a relatively stable level, which persists until around the age of 10. Subsequently, there is a noticeable sex differentiation associated with puberty. Both approaches attempt to map this complex process as effectively as possible. In this regard, the solution from CALIPER appears clearer due to largely sex-independent partitions, while the rpart RI cover more data points.

When comparing the partitions within the approaches, we found that there are no significant differences between the successive CALIPER segments from 15 days up to an age of 13 years. Similarly, there are no significant differences in the transition to the first CALIPER segment for boys aged 13 years and older. In contrast, all subdivisions within rpart are significant. When comparing both approaches, there is initially no significant difference between the sex-dependent rpart segments and the unisex CALIPER segments up to the age of 13 years. As expected, the comparison of the male CALIPER and rpart segments shows no significant differences. However, when comparing the female segments, a significant difference is found between the last two segments of CALIPER and rpart.

Discussion

This proof-of-principle study supports our hypothesis that automated tools like the decision tree algorithm implemented in rpart can be used in the generation of reference intervals. The partitioning suggestions of the machine learning algorithm matches the judgment of human expertise surprisingly well, even for complex age progressions, and therefore appear suitable to estimate age ranges for calculating RI.

Instead of visually inspecting the data as in CALIPER [10], rpart relies on a purely mathematical basis to calculate the age and sex partitions. According to the used outcome metric (the Harris & Boyd’s test), all partitions suggested by rpart differed significantly from each other. This is an improvement over the visual and thus more subjective CALIPER results, which according to the Harris & Boyd’s test was sometimes prone to oversegmentation.

For the analytes GGT, ALB and CREA, the automatic suggestions are in good agreement with their biological background and generally agree with the assessments made by human experts. For AP, an increase in the variance of the measured values between 10 and 14 years is also described in the literature [12]. It is noticeable that the rpart partitions reflect this kinetics better than the CALIPER partitions (see Figure 4, fewer data points outside the RI of rpart compared to CALIPER). In addition, there are problems with small sample sizes for the reflimR method within the two dashed CALIPER partitions and the non-significant (non-)differences between several subsequent CALIPER partitions (as described in the results section). Therefore, the partitioning performed by rpart seems to be more stringent than the one done by CALIPER.

A notable difference between the suggested partitions from rpart and CALIPER is that rpart sometimes suggests gender-specific partitions with clearly differing age ranges. This is particularly noticeable for creatinine (CREA) (Figure 3B) and alkaline phosphatase (AP) (Figure 4B). From a mathematical perspective, rpart aims to minimize the number of partitions. Conversely, humans might tend to create sex-specific partitions with identical age intervals during visual partitioning. We observed that some consecutive partitions in CALIPER do not show significant differences. Therefore, we consider partitioning suggested by rpart to be mathematically superior. However, for clinical use, this must be evaluated within the biological and clinical context.

The Harris & Boyd method is currently regarded as an established method for the comparison of reference intervals [1], although some methodological problems have been described [4]. One of them is that Harris & Boyd’s test assumes an approximate normal distribution of the data. The problem with skewed data is briefly mentioned in the original paper, but not discussed further [21]. A critical evaluation of the precise statistical problems of the methodology is beyond the scope of this publication. For several reasons, Haeckel and Wosniok [22] argue that there are almost no normally distributed analytes in clinical chemistry and propose that all quantities should be considered lognormally distributed. To briefly address the main features of the problem, we used the lognormally distributed analyte GGT to check whether the logarithmization of the measured values in our analysis leads to a different interpretation. In that case rpart suggested one additional age segment without any medical relevance, which is why we refrained from further consideration of the problem (data not shown).

With regression tree algorithms, however, there are other parameters influencing the results. Firstly, the sample size can have a major impact. Decision trees are known to easily overfit to their training set [23, 24]. In this case, tuning of the hyperparameters is required to ensure that the individual partitions reach an appropriate size. This process is also known as pruning. In particular, parameters such as the minimum number of data points per partition or the maximum number of branches need to be adapted so that non-relevant and redundant parts of the tree are removed.

It should be noted that the actual boundaries of the partitions in the decision trees of rpart are calculated from the mean value of the limit values between the two partitions. While this leads to more stringent reference interval estimations, the algorithm doesn’t interpolate in case of missing values, which might lead to gaps in the RI range. To illustrate this, we intentionally plotted the rpart boxes in the scatter plots with the minimum and maximum age values of each age partition to reveal any gaps in the calculation. In our case, the data were relatively evenly distributed across age, but when interpreting with rpart, one should be aware of possible biases due to underrepresented age intervals.

In addition, we want to stress that without careful preparation of the real-world data, analyses like this one cannot successfully be performed [2]. After all, the data from the routine laboratory requires a clean structure comparable to that of the CALIPER data in order to obtain meaningful results. On one hand, there are technical challenges such as the extraction of data from the laboratory information system (LIS) with potential subsequent anonymization. On the other hand, there are also medical issues to consider, such as how to deal with metadata like a “sample is hemolytic”-flag or the “measured value is below the detection limit”-information.

Our study demonstrates that rpart can be used for establishing reference intervals in an unbiased and mathematically rigorous fashion. This approach offers a valuable tool and can provide some guidance when trying to establish partitions in reference intervals. Thus, we underscore the importance of understanding the biochemical and statistical context. Furthermore, all derived conclusions should be carefully validated within their specific context.