Assessment of 17 clinically available renal biomarkers to predict acute kidney injury in critically ill patients

Study design and participants

This prospective observational study was conducted in the general ICU of Guangdong Provincial People’s Hospital from January 2017 to December 2017. All the patients in the ICU were eligible for inclusion. Reasons for exclusion were as follows: age under 18 years, preexisting AKI before ICU admission, a history of nephrectomy or end-stage renal disease (ESRD) or renal transplantation, preexisting dialysis before ICU admission, or readmission to ICU. The primary outcome was the development of AKI, defined as detecting any stage of AKI occurring no more than 1 week after ICU admission. The secondary outcome was the development of severe AKI, defined as the combined endpoints of AKI stage 2 or 3 according to the Kidney Disease: Improving Global Outcomes (KDIGO) criteria,^[28] no more than 1 week after ICU admission. The study protocol complied with the Strengthening the Reporting of Observational Studies in Epidemiology^[29] and the Standards for Reporting Diagnostic Accuracy criteria.^[30] Written informed consent was obtained from all participants before their enrollment. The study protocol was approved by the ethics committee of Guangdong Provincial People’s Hospital and adhered to the ethical guidelines stated in the Declaration of Helsinki.

Definitions

Within 1 week after ICU admission, AKI was diagnosed according to the KDIGO criteria,^[28] while severe AKI was defined as KDIGO stage 2 or stage 3. The baseline estimated glomerular filtration rate (eGFR) was estimated by the simplified Modification of Diet in Renal Disease formula.^[31] CKD was defined as baseline eGFR <60 ml/ min/1.73 m². For baseline sCr, we used the last available sCr within the latest 3 months before this hospitalization or the sCr at hospital admission.^[32–34] We adopted sCr rather than UO to diagnose AKI, since the UO criteria could be affected by administering diuretics or obesity.

The predictive performances of biomarkers for AKI were stratified into useless, poor, fair, good, and excellent according to the area under the receiver operator characteristic curve (AUC) located in the ranges of <0.60, 0.60–0.69, 0.70–0.79, 0.80–0.89, and ≥ 0.90, respectively.^[35] Biomarkers with a cutoff value higher than the upper limit of normal range (ULN) were also classified as useless because the present study aims to explore the biomarkers’ abilities in predicting the development (onset) rather than the severity, progression, persistence, or survival outcomes of AKI. When the optimal cutoff value is at a higher level than the ULN, even a physician not specializing in renal diseases can intuitively be aware that renal dysfunction (AKI) has already occurred, independent of any prediction.

Sample measurements and data collection

The blood and urine samples were collected simultaneously within 1 h after ICU admission. All the candidate biomarkers were tested using a standard protocol by experienced specialists in the central laboratory of Guangdong Provincial People’s Hospital. All test methods should not be formally used until quality control. Multiplex assays and enzyme-linked immunosorbent assays were used to test urinary or blood concentrations of biomarkers according to the manufacturer’s instructions. Serum biomarkers included blood urea nitrogen (BUN), sCr, CysC, IL-6, lactic acid (LAC), lactate dehydrogenase (LDH), phosphate, procalcitonin (PCT), and superoxide dismutase (SOD). Urinary biomarkers tested included ACR, ALB, alkaline phosphatase (ALP), gamma-glutamyl transpeptidase (GGT), NAG/Cr, total protein (TP), transferrin (TRF), α1-microglobulin (α1M), and β2-microglobulin (β2M). Urinary creatinine concentration was used to normalize urinary biomarker measurements to account for urinary concentration on urinary biomarker concentrations. Except for sCr testing, all biomarkers were tested once on ICU admission. In addition, sCr testing was performed on ICU admission and at least once daily after admission as a portion of routine clinical care during an ICU stay.

In the meantime, clinical and demographic characteristics were collected immediately once these patients were enrolled in this research. The variables evaluated were: age, sex, body mass index (BMI), preexisting clinical conditions, admission type, baseline sCr, admission sCr, baseline eGFR, admission eGFR, Acute Physiology and Chronic Health Evaluation (APACHE) II score, Multiple Organ Dysfunction Score (MODS), Sequential Organ Failure Assessment (SOFA) score, urine production (UP) for the first 24 h after admission, and sCr percent change and the clinical outcomes. We also prospectively recorded the hourly UO from enrollment to ICU discharge.

Statistical analysis

The statistical analysis was conducted with the software programs SAS (SAS v9.4; SAS Institute, Cary, NC, USA) and R v3.3.3 (R Foundation for Statistical Computing, Vienna, Austria) using RStudio v1.0.136 (RStudio Inc, Boston, MA, USA). For AKI and non-AKI, we used frequencies and percentages for categorical variables and medians and ranges for continuous variables to summarize demographic and clinical baseline characteristics numerically. Using Fisher’s exact test or the Wilcoxon rank-sum test, comparisons of patient characteristics between AKI and non-AKI were performed. What is more, we inspected the pairwise associations between biomarkers based on Spearman’s rank correlation coefficient. By conducting a Euclidean distance-based unsupervised clustering analysis and constructing a dendrogram to display identified clusters, groups of similar biomarkers were identified.^[36]

We measured the concentration of 17 urinary and blood analytes to evaluate the capability of the biomarker to predict the occurrence of AKI alone or in combination. Biomarkers were divided into four mechanistic groups (inflammation, oxidative stress, glomerular dysfunction, and tubular dysfunction) according to the presumed pathophysiologic mechanism they reflected. The predictive performance of markers was assessed using simple logistic regression (SLR) models. SLR results were obtained with AUCs and corresponding 95% confidence intervals (CIs). The analysis was achieved via AUC with a leave-one-out cross-validation approach. A leave-one-out cross-validation method was used to determine the model’s generalization ability without an independent test data set. Therefore, three relevant indicators were generated: the proportion of AKI misclassification, the balance of non-AKI misclassification, and the mean square error (MSE). MSE was an average calculation of the squared difference between observations’ event status (0 or 1) and the predicted probability of being an event based on the SLR model. Models with highly discriminating markers yield high possibilities for events and low chances for nonevents, resulting in minor differences and small MSE values. For each leave-one-out cross-validated model, AUC was further used to evaluate performance. When the values of AUC were different, markers to predict AKI were prioritized according to the values of AUC. However, when the values of AUC were the same, MSE was compared because lower MSE indicated better fit of the model. The difference between AUCs for detecting a biomarker for AKI and severe AKI was found, and the performance of this biomarker was identified according to the lower AUC if no difference existed.

Patient characteristics and outcomes

Figure 1 demonstrates the protocol and flow diagram of the screening process. Among 1761 critically ill adult patients enrolled in the study, 263 were excluded, and therefore, 1498 patients were involved in the study. Twenty-five percent (376/1498) of these patients developed AKI according to the KDIGO criteria. Among them, with the aggravation of AKI severity, the mortality of patients was increasing (Figure 1). There were no differences between the AKI and non-AKI groups based on BMI or preexisting clinical conditions of cancer and thyroid disease. Apart from these, other variables of baseline characteristics and outcomes were different between the AKI and the non-AKI groups (Table 1).

Figure 1:

Study flowchart. AKI: acute kidney injury; ICU: intensive care units; KDIGO: Kidney Disease: Improving Global Outcomes.

Biomarker characteristics

Before the prediction analysis, we made two analyses, cluster analysis and correlation analysis, to identify the relationship of individual biomarkers with each other. Foremost, we did an unsupervised cluster analysis to evaluate which biomarkers were similar to each other (Supplementary Figure 1). We discovered several resemblances to our prior grouping of biomarkers. Interestingly, portions of the proteins that we assumed would be analogous to each other were clustered contrarily, such as CysC and BUN. We then compared the correlation coefficients for each of the biomarkers using Spearman’s rank correlation coefficient (Supplementary Table 1).

Biomarker detection abilities for AKI and severe AKI

Table 2 displays the predictive features for AKI for each of the 17 analytes classified by the mechanistic biomarker category. Since higher AUC indicated better predictive ability and lower MSE indicated better fit of the model with real world, biomarkers were evaluated for their predictive performance with AUC and MSE. In contrast to all of the biomarkers and clinical markers, CysC notably had the highest predictive values for both endpoints (Tables 2 and 3). Among biomarkers, the highest AUC value of 0.785 was seen for CysC and BUN to predict AKI (Table 2). Used in the prediction of AKI, LDH had an AUC of 0.764, and ACR, ALB, IL-6, and NAG/Cr had AUCs of 0.759, 0.727, 0.710, and 0.708, respectively. For comparison of clinical markers, the APACHE II score, MODS, SOFA score, and eGFR at admission had AUC values of 0.844, 0.802, 0.798, and 0.703, respectively.

Of the 17 renal biomarkers, 7 biomarkers (serum biomarkers of PCT, SOD, and phosphate and urinary biomarkers of TP, ALP, GGT, and β2M) with AUC less than 0.6 and 2 biomarkers (serum BUN and urinary ALB) with a fair-to-good apparent AUC value but a cutoff value higher than the ULN were useless for both AKI and severe AKI prediction (Tables 2 and 3, Supplementary Table 2). Three biomarkers (serum LAC, urinary TRF, and urinary α1M) were poor for the prediction of both outcomes. Although urinary TRF had a nominal AUC of 0.764 for severe AKI prediction, there was no statistically significant difference with an AUC of 0.693 for AKI prediction; so, TRF was poor for the prediction of both AKI and severe AKI. The same was true for urinary α1M (Tables 2 and 3). Three biomarkers (serum IL-6, serum LDH, and urinary NAG/ Cr) had a fair performance for predicting both outcomes. Two biomarkers (serum CysC and urinary ACR) performed fairly for predicting AKI and showed good performance for severe AKI prediction. These data demonstrated that the currently available biomarkers and clinical markers are better predictors for severe AKI than for AKI. The prediction performance of all 17 biomarkers for AKI and severe AKI prediction is summarized in Table 4.

We subsequently compared the abilities of the biomarkers to predict severe AKI. CysC, LDH, APACHE II score, MODS, and SOFA score showed marked improvements in prediction, whereas the predictive ability of many biomarkers was obviously improved (Table 3). CysC was the best predictor of severe AKI, with the highest AUC of 0.883 and the smallest MSE of 0.060.

Prediction abilities of combinations of biomarkers or clinical markers for AKI and severe AKI

We evaluated the combinations of biomarkers and clinical markers to predict the two outcomes by ranking groups of biomarkers according to AUC and MSE. The combination of CysC and APACHE II score had the highest AUC and the lowest MSE to predict AKI, while CysC and MODS panel had the highest AUC and a low MSE to predict severe AKI (Table 5, Figure 2). Furthermore, the panel of CysC and NAG/Cr had higher AUC and lower MSE to predict both outcomes. As suggested by our correlation analysis, the correlation between CysC and LAC was relatively weak, with Spearman’s rank correlation coefficient of only 0.0975. Consequently, the combination of CysC and LAC was a very good predictor for severe AKI, with an AUC of 0.907, a positive predictive value (PPV) of 0.86, and a negative predictive value (NPV) of 0.931 (Table 5). The values of AUC in predicting AKI and severe AKI by combining CysC or biomarkers or clinical markers are shown in Figure 2. Moreover, boxplots for each of the eventual outcomes of the AKI stage were created for CysC and APACHE II score. We also determined that combinations of biomarkers or clinical markers markedly improved the prediction abilities of a single biomarker (Table 5).

Figure 2:

The best panel of combining CysC or biomarkers or clinical markers to predict the outcomes of AKI and severe AKI. (a) The AUCs for the panels of CysC, CysC and NAG/Cr, and CysC and APACHE II score to predict AKI were 0.785, 0.856, and 0.868, respectively. (b) The AUCs for the panels of CysC, CysC and LAC, CysC and MODS to predict severe AKI were 0.883, 0.907, and 0.912, respectively. (c) Box and whisker plot for patients’ CysC concentration value to predict AKI stage 1, 2, or 3 and death. (d) Box and whisker plot for patients’ APACHE II score value to predict AKI stage 1, 2, or 3 and death. Boxes show the median value and 25th and 75th percentiles. Whiskers represent the range of values.

AUC: area under the receiver operating characteristic curve; AKI: acute kidney injury; APACHE II: Acute Physiology and Chronic Health Evaluation II; CysC: cystatin C; LAC: lactic acid; MODS: Multiple Organ Dysfunction Score; NAG: N-acetyl-β-d-glucosaminidase; NAG/Cr: NAG-to-creatinine ratio.

Ethics approval and consent to participate

The ethics committee of the Guangdong Provincial People’s Hospital approved the protocol. Written informed consent was obtained from each patient or from appropriate surrogates for patients unable to consent.