Artificial intelligence in the NICU to predict extubation success in prematurely born infants

Introduction

Mechanical ventilation can be life saving but has complications including bronchopulmonary dysplasia (BPD) and its sequelae [1]. The duration of mechanical ventilation is a predictor for BPD development with each additional increase in week of mechanical ventilation increasing the risk for BPD development [2]. Reducing the duration of invasive ventilation is dependent on successful extubation, however, this can be difficult to predict with certainty. Weaning a patient too early exposes them to the risks of respiratory decompenation and reintubation. Weaning a patient too late exposes them to prolonged volutrauma and possible ventilator associated infection.

In clinical practice, extubation readiness is often assessed by a spontaneous breathing trial (SBT) in which an infant undergoes a brief period of spontaneous breathing via the endotracheal tube while on continuous positive airway pressure [3]. A systematic review of extubation readiness tests using a SBT in prematurely born infants, however, found a lack of strong evidence to support the use of SBT in clinical practice [4]. Combining an SBT with measurement of the electrical activity of the diaphragm activity (electromyography) pre extubation improved the predictive ability, but there was only moderate accuracy with an area under the receiver operator characteristic curve (AUC) of 0.74 [5].

The neonatal intensive care unit (NICU) is considered a big data environment as patients are under constant respiratory, cardiovascular, neurological and biochemical monitoring [6]. Growing computational methodologies have resulted in a recent focus on artificial intelligence (AI) in healthcare to harness those data and improve detection of clinical deterioration, develop clinical decision-making algorithms and predictive tools [6, 7]. AI is a field of computer science which is focused on the development of systems which mimic human intelligence [8]. AI systems have the ability to learn, process, reason, assess and predict, but in a manner which is probably more powerful, complex and time efficient than the individual or collective capability of clinicians [9].

While there has been some practical successes influencing our everyday lives (e.g. voice recognition and self-driving cars), at present there is no widespread use of AI in healthcare [10, 11]. There has, however, been an exponential increase in citations in the past 20 years exploring the use of AI in adults, but a smaller rise in publications exploring the use of AI in neonates [12, 13].

Machine learning (ML) and deep learning (DL) are subfields of AI. Machine learning uses pattern recognition and computational theories to uncover high complexity patterns that exist among independent and dependent variables [14, 15]. Machine learning aims to process and reason those data via statistical analysis and computational technologies to predict future events and outcomes [12]. Machine learning methods include logistic regression, decision tress, random forest classifiers, support vector machines and gradient boosting [Table 1]. Deep learning (DL) is an area of ML that focuses on neural networks with many hidden layers. Hanson et al. described neural networks as being like the human brain with multiple layers fully connected next to each other, allowing data to flow and pass on information [8]. Deep learning automates much of the feature extraction part of the process, eliminating some of the manual human intervention required in ML and enabling use of larger data sets. Whereas in ML there are challenges with data that are from different sources, DL allows for larger data sets in high volume and multiple sources [16]. An example of DL methods is artificial neural networks (ANN) [Table 1].

The aim of this narrative review is to determine whether AI might improve extubation success in prematurely born infants.

Methods

We conducted a literature search in Medline for articles related to the use of AI models as extubation prediction tools in infants born prematurely or very low birthweight. The search strategy was produced from a combination of search terms (airway extubation; artificial intelligence; deep learning; extubation; infant; premature; machine learning; model*; neonat*; neural networks; newborn; predict*; premat*; premature birth; preterm; ventilator weaning; weaning).

The performance of each tool was assessed using area under receiver operator characteristic curves (AUCs) [20].

Results

Eleven studies were identified (Table 2). There was a heterogenous group of variables identified through feature selection in each study (Table 3).

The ML techniques had AUCs between 0.7 and 0.87. The best reported outcome was for a multiple logistic regression (MLR) technique which used the Multiparameter Intelligent Monitoring in Intensive Care II database (MIMIC-II) which is a freely available database intended to support epidemiologic research as a resource to evaluate new clinical support and monitoring algorithms [34]. Following the reported success of an extubation prediction model in adults using the MIMIC-II [35], Mikhno et al. developed a ML model using MLR for extubation prediction in prematurely born infants [28]. Within the database of 7,800 patient records, 242 patients met the inclusion criteria (being born between 23 and 31 weeks of gestation). A total of 58,520 models were produced. The group then examined the performance of each model combining the two top models with six variables (monocyte cell count, rapid shallow breathing index, fraction of inspired oxygen [FiO₂], heart rate, partial pressure of arterial oxygen [PaO₂]:FiO₂ ratio and work of breathing index) to produce a combined model of extubation prediction with an AUC of 0.87. External validation was not performed.

Mueller et al. [20, 29] compared logistic regression (LR) prediction models with more modern ML techniques which included support vector machine (SVM), naïve Bayesian classifier (NBC) and boosted decision trees (BDT) [29]. Feature selection variables included birth weight, APGAR score at 5 min, maternal betamethasone, FiO₂, peak inspiratory pressure, inspiratory time, tidal volume, pH, partial pressure of arterial carbon dioxide (PaCO₂), PaO₂, oxygen saturation (SpO₂), heart rate, blood pressure, minute volume, surfactant and caffeine administration. Logistic regression performance (AUC 0.77) was greater than that of the more modern ML methods (AUCs: BDT 0.5; NBC 0.6; SVM 0.5) [Table 2] [29]. External validation was not performed.

Song et al. [22] also compared LR algorithms to more modern ML methods including decision trees, random forest classifiers and gradient boosting (extreme gradient boosting [XGB] and gradient boosting machine [GBM]). Features selected included birth weight, gestational age, SpO₂, FiO₂, blood pressure (mean, diastolic and systolic), respiratory rate and positive end expiratory pressure. In their external validation cohort, LR performance (AUC 0.76) was greater than that of other ML methods (AUC: XGB 0.71; GBM 0.70; RF 0.71; DT 0.70). A decision curve analysis suggested that applying their model to clinical predictors could increase extubation success. Of note, the external validation cohort used was the MIMIC-II database in which results were collected between 2001 and 2008.

Mueller et al. was the first group to report an extubation prediction tool in preterm infants using automated neural network (ANN) [20]. They developed a fully cross-validated neural network from an initial 51 variables identified by clinicians as potentially predictive. The resulting ANN identified 13 variables as useful for extubation prediction (gestational age, pulse, blood pressure, pH, arterial oxygen saturation, mode, peak inspiratory pressure, peak end expiratory pressure, mean airway pressure, inspiratory time, ratio of inspiratory to expiratory time, tidal volume, partial arterial pressure of CO₂). When compared to MLR, the ANN performed favourably (AUC: ANN 0.87; MLR 0.75), while also giving a higher AUC than clinician predictors (reported as a accuracy of 78 % in the training data set and 70 % in the validation data set) [20]. No external validation was performed. In a subsequent study by those authors, using a different dataset, the ANN model gave an AUC of 0.68 but found that clinician predictors had an accuracy of 88 % [29].

Discussion

We have demonstrated that the performance of current AI models is not greater than clinical predictors of extubation success in prematurely born infants. Furthermore, very few of the models had undergone external validation. Although a predictive model may exhibit strong performance within its training set, its effectiveness may diminish when applied to an external cohort [31]. External validation tests original prediction models in a new set of patients to determine the generalisability and applicability of the model [36]. Only two studies reported external validation. Song et al. [22] showed a reduced accuracy when compared to their training set, while Chen et al. [23] showed a slight improvement in the AUC (Table 2). While Gupta et al. [26] did not report an external validation cohort in their original report, a subsequent study by the same authors reported an external validation of their model with an AUC of 0.72 which was less than that of the training cohort [26, 33].

Logistic regression models outperformed other machine learning methods in this context. Logistic regression is a simple and interpretable model and is thus advantageous in a clinical settings where healthcare providers need to understand the factors contributing to the prediction [37]. In contrast, complex machine learning models like gradient boosting may be harder to understand. The success of logistic regression could be due to the specific characteristics of the data related to prematurely born infants. In particular, logistic regression works well when the relationship between predictors and outcomes is approximately linear and when there are clear decision boundaries. In addition, logistic regression is designed for binary classification tasks, which is the case when predicting extubation success as ‘success’ or ‘failure’.

There was a heterogenous group of variables identified through feature selection in each study (Table 3). This may reflect that each group used distinct datasets, which themselves displayed heterogeneity in terms of demographic, clinical characteristics and treatment strategies. It may also reflect the different machine learning models studied. Further research is needed to better understand the factors that drive the selection of specific features in the clinical scenarios that relate to extubation.

The studies varied with regard to the gestational ages included in model development (Table 2). While four studies included gestational age as a feature in their final prediction models [20, 22, 26, 30], four other studies included birthweight [21, 23, 25, 29] which correlates significantly with gestational age. The only group to look at subsets of patients based on birthweights did not find that performance was enhanced when there was more homogenous groups [29].

In addition, studies varied in terms of their definition of extubation failure, although the majority selected reintubation with 72 h (Table 2). While there is no consensus on the optimal observation window post extubation [32], 77 % of respiratory reintubations in extremely preterm infants occur within 7 days [38, 39]. As for AI and extubation prediction, future work requires more clear consensus on the definition of extubation failure. In addition, a narrow focus to more specific gestational age subsets may improve accuracy. A multicentre study involving a larger population than has been studied to date is warranted to develop a more robust predictive model for assessing extubation success. Such a study could enhance model performance and broaden its applicability, provided it is subject to external validation.

Conclusions

There is as yet no model of artificial intelligence which predicts extubation success better than clinical predictors. Furthermore, few of the models have undergone external validation limiting their application in the clinical setting.