Assessment of machine-learning techniques on large pathology data sets to address assay redundancy in routine liver function test profiles

Introduction

Traditionally, pathology laboratories have grouped individual tests performed on high throughput biochemistry analysers into profiles associated with different organs [e.g., liver function tests (LFT) and renal function tests (RFT)] to enhance laboratory screening for disease. As well as health care settings, pathology test profiles are central to screening community patients. In the community setting, the lower prevalence of abnormalities contributes to reduced test specificity, leading to many false positives that are expensive to investigate and heighten patient anxiety. A new assessment of blood test profiles, particularly for community patient screening, is therefore required.

In 2012, an Association of Clinical Biochemists (UK) Clinical Practice Section publication suggested an initial standardisation of common biochemistry profiles, and the eventual harmonisation of profiles across the UK [1]. The content of the LFT and other profiles are largely historically dictated. Traditionally, when a new test was developed and found to be abnormal in a disease associated with a particular organ, it was included in the profile without regard to whether it provided any additional diagnostic information. This study presents an evidence-based strategy via mass laboratory data access and sophisticated machine learning algorithms. The strategy presented herein was simply to predict the diagnostic value of an existing routine LFT marker, γ-glutamyl transferase (GGT), via other routine LFT marker results; the implication being that if patterns from other markers accurately predict GGT elevation, GGT is not required for routine screening. After initial screening however, GGT does retain value for laboratory diagnosis, for example in cases of alcohol and drug abuse. GGT has been found to have very good diagnostic value for monitoring alcoholism, the differentiation of drinking level and liver cirrhosis, particularly when considered with serum urate, carbohydrate-deficient transferrin (CDT) and other routine and novel biomarkers [2–4]. When considering only abnormal GGT responses for this study, serum total cholesterol was also identified as having potential diagnostic value in this context; increased serum GGT has been identified as a predictor of cardiovascular disease [5, 6].

The most common routine serum LFT profile consists of GGT, alkaline phosphatase (ALP), alanine transaminase (ALT), aspartate aminotransferase (AST), sometimes lactate dehydrogenase (LD), serum albumin, and total serum bilirubin [7]. This multicomponent biochemical LFT has poor sensitivity and specificity for liver disease, and it has been suggested that it is time to consider whether some of the components should be removed [1]. The Association of Clinical Biochemists has advocated a four-component profile (albumin, bilirubin, ALT and ALP), but there was no supporting evidence for any basis for reducing the profile to this group. There have been in excess of 6000 papers published since 1990 dealing with LFTs [8], however most have been based on hospital practice rather than community practice, and they have been mostly retrospective and concerned with probabilities given a disease state not the predictive probability of disease. The BALLETS study [9] was a prospective study based in primary care practice where patients were fully investigated following at least one abnormal analyte from a full LFT panel. The BALLETS study showed that the traditional LFT profile has low specificity for liver disease and concluded that “the routine LFT profile contain fewer components and provide the same or greater value”, particularly if it is used as a screening procedure [8–10]. Benefits from reducing the number of components of the routine LFT profile while retaining predictive power include reduced likelihood of over investigation due to the low specificity/sensitivity, reduced costs to the health system and less patient anxiety [11].

Understanding the deeper interactions between routine clinical chemistry serum biomarkers (Table 1) for a given disease condition can be achieved today through access to enormous datasets and the application of pattern recognition strategies via machine learning methods. This study explored a retrospective large data set of individual pathology test results (>25,000 cases), which included thousands of cases with deranged liver enzymes elevated above the specific reference interval, obtained from screened community patients after a medical consultation (Table 2). Two machine learning techniques were combined to examine predictions of normal or elevated GGT response, as defined by laboratory reference interval (Table 1); (i) Recursive partitioning based decision models (decision trees), which have been applied to medical knowledge domains [12, 13] and provide advantages of applicability to diverse data regardless of distribution, as well as multiple decision boundaries [14] and (ii) Support vector machines (SVM), which provide powerful classification and regression tools through kernel analyses without high computational cost [15]. SVMs have been successfully developed previously to separate diagnosed nonalcoholic steatohepatitis (NASH) patients from healthy controls, with 3 biomarkers from 17 identified with high predictive value after algorithm training and testing [16].

Table 2

Summary of GGT response categories investigated by machine learning to examine GGT redundancy for routine LFT profiles through tandem recursive partitioning (decision tree) and support vector machine (SVM) prediction.

For the prediction of serum GGT response, single decision trees were used initially to identify individual LFT assay importance for GGT category prediction, and to ascertain the decision thresholds (e.g., ALT >30 U/L) to guide the most accurate GGT category prediction. With the top predictors (ALT and ALP) and predictor thresholds identified by single decision trees, final prediction accuracy for GGT category was achieved via SVM training/testing protocols run as a 10-fold cross validation analyses. An additional aim was to use machine learning to interrogate cases of elevated serum GGT (>50 U/L), to assess the capacity of ALT/ALP with other clinical chemistry biomarkers (e.g., serum urate, cholesterol) to enhance the detection of liver pathology, as reflected by varying degrees of GGT elevation above the reference interval.

Materials and methods

Machine learning and analysis

All decision tree and SVM analyses were performed using the R packages rpart [17] or e1071 [18] respectively. Single decision tree rules based on calculated predictor variable thresholds (e.g., ALT <30.5 U/L) at decision nodes were determined for each GGT response category (0 or 1). Single decision trees were run using default settings, which included a cp value of 0.01 and a minimum bucket size defined as “minbucket=round(minsplit/3)”. Single decision tree analyses also provided screening for predictor variable importance rankings, with the top 2–3 GGT category predictors ultimately applied to tuned SVM models. SVM predictions were based on 10-fold cross comparison of training and testing data to arrive at an overall mean percentage of class (category 0 or 1) prediction (70% data for training, and 30% for testing prediction accuracy). The SVM investigations included a pre-analysis tuning phase where the best cost and γ-coefficients were identified for the combination of the response category (normal or elevated GGT) and the predictor variables of interest [15]. As a result of tuning, all SVM models used a gamma of 0.1 and cost coefficient of 10 or 100. Unpruned decision tree models containing all LFT predictor variables were fitted to identify the best GGT predictors for 10-fold training/testing SVM analysis of prediction accuracy. Four to six predictor variables were tested for initial decision tree models (ALT, AST, ALP, LD, total bilirubin, serum albumin) with only ALT plus ALP applied to subsequent SVM prediction models. Age was initially included in SVM models, but except for one example (Figure 1), did not aid prediction accuracy. Confidence intervals were calculated for each SVM testing phase percentage accuracy rate using a traditional method based on the Central Limit Theorem, which proves a normal distribution for a proportion [19].

Figure 1

Prediction of GGT category (category 0, within laboratory reference interval; category 1, GGT elevated above reference interval) by ALP and ALT for the total case sample provided (male and female cases; Age 0–106 years; n=25,420).

(A) Single decision tree analysis of the combined ALP and ALT rules to predict GGT category, including prediction accuracy and ALP/ALT concentration thresholds. (B) GGT category prediction accuracy calculated from 10-fold training and testing cross validation via support vector machines (70% data training and 30% data for testing). The reported result is the testing phase SVM prediction accuracy.* Adding Age <36.5 years increased the prediction accuracy for Cat 0%–94.5% (794/840 cases correct). Age did not improve category 1 prediction.

Results

Single decision trees, SVM and predictions of elevated GGT responses by ALP and ALT

Preliminary unpruned single decision trees were produced to identify the leading predictors of GGT category for normal versus abnormal responses (Table 2B). To start, the entire sample of 25,420 cases, female and male with an age range from 0 to 106 years, was investigated by including all LFT markers in the decision tree model to predict GGT response category. Of the original 25,420 cases, 18,439 were within the GGT laboratory reference interval (category 0), and 6981 cases were elevated above the upper limit of the GGT reference interval (category 1) (Table 1). The preliminary trees identified only ALP and ALT (and occasionally age) as important to the classification of GGT response (which was confirmed by a multiple tree analysis, the random forest: results not shown). Additional tree analysis utilised only ALP+ALT+Age, or ALP+ALT as predictor variables for GGT response classification.

Figure 1 shows the single decision tree rules identified to predict and differentiate an elevated GGT response (category 1) from a GGT response within the reference interval (category 0). The crucial ALT threshold calculated by the tree was greater or less than (> or <) 30.5 U/L, with an associated ALP threshold of <90.5 U/L for category 0 (85.8% prediction accuracy), and ALP >121.5 U/L for category 1 (90.2% accuracy). The addition of age to the model improved the prediction accuracy for category 0 to 94.5% (<36.5 years), but age did not influence category 1 prediction. Cases <18 years of age were removed and another decision tree analysis conducted on the same sample (Figure 2). Similar prediction accuracies and ALT/ALP thresholds were found, except that the ALP threshold for a category 0 prediction increased to 109.5 U/L.

Figure 2

Prediction of GGT category (category 0, within laboratory reference interval; category 1, GGT elevated above reference interval) by ALP and ALT for the adult case sample provided (male and female cases; Age 18–106 years; n=24,812).

(A) Single decision tree analysis of the combined ALP and ALT rules to predict GGT category, including prediction accuracy and ALP/ALT concentration thresholds. (B) GGT category prediction accuracy calculated from 10-fold training and testing cross validation via support vector machines (70% data training and 30% data for testing). The reported result is the testing phase SVM prediction accuracy.

The influence of sex on the prediction of elevated serum GGT response by ALT and ALP was investigated. For females a prediction of category 0 was similar to the combined sample of 18 years or older (Figure 2) with ALT of <29.5 U/L and ALP of <109.5 U/L required (Figure 3). However, for the prediction of category 1, ALT >29.5 alone provided the best accuracy. Compared to the combined sample including both sexes, female prediction accuracies were low with 73.6% for category 0 and 67.3% for category 1. Male category 0/1 prediction accuracies were >90%, with an ALP decision threshold at 124.5 U/L (Figure 4). ALT was also required for the highest prediction percentage with a category 0 prediction needing a threshold of <30.5 U/L, while a category 1 ALT threshold was lower at >27.5 U/L.

Figure 3

Prediction of GGT category (category 0, within laboratory reference interval; category 1, GGT elevated above reference interval) by ALP and ALT for female adult cases (Age 18–103 years; n=11,154).

(A) Single decision tree analysis of the combined ALP and ALT rules to predict GGT category, including prediction accuracy and ALP/ALT concentration thresholds. (B) GGT category prediction accuracy calculated from 10-fold training and testing cross validation via support vector machines (70% data training and 30% data for testing). The reported result is the testing phase SVM prediction accuracy.

Figure 4

Prediction of GGT category (category 0, within laboratory reference interval; category 1, GGT elevated above reference interval) by ALP and ALT for male adult cases (Age 18–106 years; n=13,658).

(A) Single decision tree analysis of the combined ALP and ALT rules to predict GGT category, including prediction accuracy and ALP/ALT concentration thresholds. (B) GGT category prediction accuracy calculated from 10-fold training and testing cross validation via support vector machines (70% data training and 30% data for testing). The reported result is the testing phase SVM prediction accuracy.

To validate the accuracy of GGT class prediction reported in Figures 1–4 for single decision trees, the same data was used to train and test SVM ensembles (10-fold cross validation, 70/30 train/test data ratio). For the entire sample (Figure 1B) and the combined female/male sample of 18 years or older (Figure 2B), the category prediction accuracies ranged between 75% and 80%, lower than the accuracy percentages estimated by single decision trees. The disparity in GGT category prediction accuracy by ALP and ALT when comparing adult females and males was also detected by SVM ensembles (Figures 3B and 4B), with the overall category prediction trends via SVM the same as those found by single decision trees, albeit marginally lower.

The addition of AST to SVM ensembles did not increase prediction accuracy percentages produced by ALP and ALT, and for some models decreased prediction accuracy by 2%–3%. SVM models using total serum bilirubin, LD and serum albumin as GGT predictor variables achieved category 0 prediction of 78%, but prediction of category 1 by this model was the lowest of any model tested at 63%. The addition of bilirubin, LD or albumin to the ALP/ALT models did not enhance GGT category prediction for any analysis, and often reduced accuracy (results not shown).

Serum cholesterol and ALP interaction for abnormal LFT cases

Abnormal LFT cases (Table 2A) were examined by SVM to further assess the utility of ALT plus ALP for elevated GGT prediction, as well as consider non-LFT clinical chemistry markers (Table 1), for example, serum urate (which has been previously found as a laboratory marker for alcoholism, a cause of elevated serum GGT).

The GGT response categories were both above the upper limit of the GGT reference interval at 40–180 U/L (Cat 0) and 181–2613 U/L (Cat 1) (female and male cases). While serum urate was found as a useful elevated GGT predictor, total serum cholesterol appeared from preliminary analysis as interacting strongly with ALP to produce a clear discrimination between the two elevated GGT categories. Figure 5A shows a SVM plot describing the relationship between ALP and cholesterol, with a serum urate concentration of 0.50 mmol/L included as a constant in the SVM model. The model found an inverse relationship for ALP and cholesterol, with a decrease in ALP associated with an increase of serum cholesterol when predicting elevated GGT category (overall prediction accuracy of 75%). This analysis was repeated with 3 elevated GGT response categories (Figure 5B), with an intermediate category (GGT 119–293 U/L) introduced to examine the ALP – cholesterol relationship further. The introduction of the extra category emphasised a strong ALP – cholesterol relationship for the intermediate GGT category (119–293 U/L), which was also inverse. The decrease of ALP within the GGT response range of 119–293 U/L showed that a serum ALP concentration of 500 U/L correlated with a serum total cholesterol concentration of approximately 3.0 mmol/L. An ALP concentration of approximately 100 U/L was associated with a cholesterol level of 7.5–8.0 mmol/L (overall prediction accuracy of 60%). Without clinical notes or patient history, the significance of cholesterol and its interaction with ALP cannot be ascertained with confidence, but the analysis does highlight that interactions with other serum markers beside the traditional LFT profile assays may add value to the laboratory diagnosis of abnormal liver function.

Figure 5

Support vector machine (SVM) analysis of serum chemistry predictors of elevated GGT response within an abnormal liver function test (LFT) profile (Table 2A). ALP and serum cholesterol were the y- and x-axis variables respectively, with a constant serum urate concentration of 0.50 mmol/L added to the model to define the SVM hyper-plane separating the GGT response categories.

(A) The two response variable categories investigated were elevated GGT category 0 (40–180 U/L: n=344) or 1 (181–2613 U/L: n=347). The replacement of ALP, cholesterol or urate by ALT did not alter the classification plot represented above. (B) The three response variable categories investigated were elevated GGT category 0 (40–118 U/L: n=233), category 1 (297–2613 U/L: n=230) and intermediate category 2 (119–293 U/L: n=228). The replacement of ALP, cholesterol or urate by ALT did not alter the classification plot represented above.

Discussion

The results of this study provides a quantitative, evidence-based strategy to assess the value of routine pathology tests in the context of reducing the complexity of the diagnostic information conduit that laboratories report to requesting clinicians. This approach will be of particular value to community practice where tests such as LFTs are used for screening to identify underlying liver disease. GGT is a serum enzyme marker routinely included in the liver function test profile, within a broader clinical chemistry (Table 1) and routine diagnostic pathology-testing regime [7, 10]. In Australia, a full pathology (blood test) profile also includes a full/complete blood count (FBC/CBC), with special tests also requested if indicated (e.g., immunoassay, drug testing). This study focussed on the assessment of GGT as a routine LFT marker through machine learning prediction by other test biomarker patterns contained in large clinical chemistry datasets. The results reported here showed that normal (not elevated) and abnormal (elevated) GGT responses can be predicted with high accuracy by ALP and ALT; AST was not required (particularly for cases with highly elevated, abnormal GGT responses – Table 2A and C). In comparison to ALP and ALT alone, total serum bilirubin, LD and serum albumin were not effective for the prediction of GGT response, and the addition of these markers individually or together to ALP and ALT did not augment prediction accuracy. This evidence, therefore, suggests that ALP and ALT are sufficient for routine LFT screening of community patients, with their combined activity sufficient to also predict normal or abnormal GGT results. The decision tree results added additional value via the calculation of ALP and ALT concentration thresholds to guide decisions on whether to suspect elevated GGT.

In general, an ALT level >30 U/L combined with an ALP level >100–125 U/L suggested an elevated GGT at accuracies of >80% (with prediction variation associated with sex, but not age). If presenting with an ALT >30 U/L and ALP >125 U/L, and a history that indicates alcohol or drug abuse, GGT could then be considered as a second tier test for future monitoring, rather than a primary marker (and additional discernment would be required if the patient is a 17–19 year old male, since the upper limit of the ALP reference interval is 200 U/L: Table 1). In the current configuration, routine LFTs have sensitivity and specificity limitations, leading to inappropriate investigations and patient anxiety if results are misinterpreted or over interpreted. Reducing this potential confusion is difficult because LFTs have become a cornerstone of community medicine testing. Therefore, robust evidence must be produced that demonstrates that LFT profile modification will not compromise existing diagnostic efficacy; as described here, evidence of test redundancy can be identified and evaluated by machine learning analyses of large pathology data sets.

The BALLETS trial [9] investigated a large number of patients who were identified as having at least one abnormal component of their LFT, where after they were extensively investigated to determine what might have caused that abnormality. The BALLETS study found that 45% of patients with an abnormal component of the LFT did not have an identifiable disease and 52% had non-alcoholic fatty liver disease (NAFLD) or at-risk alcohol intake. Lilford et al. [8] reporting on the findings of the BALLETS trial suggested that the LFT should comprise albumin, bilirubin, ALT and ALP, and this was independently suggested by the Association of Clinical Biochemistry (UK) Clinical Practice Section [1]. The BALLETS conclusions and the recommendations of the Association of Clinical Biochemistry (UK), which did not include GGT, have been partly confirmed here by identifying ALP and ALT as leading predictors of GGT levels, as well as adding value through providing decision boundaries for ALP/ALT via machine learning.

Single decision trees rules produced classification predictions of high accuracy, based on subsets of cases taken from the hundreds – thousands available. To increase sampling and prediction accuracy for this method, decision tree ensembles can also be employed [13, 20] or a tandem decision tree plus SVM approach [21]. This study combined single decision trees with 10-fold cross validation SVM ensembles to identify prediction rules and percentages of prediction accuracy. In this context decision trees are very effective for dimension reduction, with ALP and ALT consistently the two highest ranked predictors of GGT category on variable importance scales. Once leading candidate predictor variables are identified, SVM provides very powerful assessment of classifier performance for the response of interest. Running the 10-fold SVM ensembles were also important to explore the risk of over-fitting the single decision tree prediction models, which was suggested by the generally higher prediction accuracies calculated for decision trees (Figures 1–4).

For the SVM investigation of elevated GGT responses (abnormal LFT profile – Table 2A), a third predictor was added, which like cholesterol had been identified by preliminary decision trees applied to non-LFT markers available in the data set; namely serum urate, which was added to the SVM model as a “slice” at a concentration of 0.50 mmol/L. Urate has been associated with chronic alcohol abuse [22, 23], although this has been debated for a Japanese male population [24]. GGT, AST, mean corpuscular volume (MCV) and carbohydrate-deficient transferrin (CDT), with and without reference to other clinical data, have been nominated for the diagnostic and monitoring utility of alcoholics, with prediction accuracies of 75% (0.75) claimed from this profile [2, 25]. Less expected, as an elevated GGT predictor was serum cholesterol, although GGT has been identified as a predictor of cardiovascular disease, CVD and also of subsequent mortality. This relationship of GGT to CVD included, as part of the risk profile, a role for dyslipidaemia and positive associations with high and low density lipoprotein [HDL/LDL] cholesterol [5]. In addition, studies on cardiac disease risk and GGT also have observed associations to arterial stiffness and coronary artery disease (CAD) [26], as well as coronary artery calcification (CAC) and elevated GGT [27].

Whether GGT elevation is due to drug/alcohol abuse, heart disease or extra-hepatic problems, of course, requires direct interaction with the patient and additional testing. However, the models and predictions reported here address an imperative for laboratory diagnostics, namely an evidence-based method for enquiry into LFT redundancy, and possible removal of traditional LFT markers like GGT without loss of efficacy. The findings reported here support the expert advice given by the Association of Clinical Biochemists (Clinical Practice group) [1] and the outcomes of the BALLETS study [9].