Development and validation of a nomogram for predicting the disease progression of nonsevere coronavirus disease 2019

INTRODUCTION

Since the global spread of severe acute respiratory syndrome coronavirus 2 disease (SARS-CoV-2, previously temporarily named 2019 novel coronavirus, or COVID-19), it has been designated the sixth public health emergency of international concern.^{[1, 2]} To date, the severity of most cases of COVID-19 has been nonsevere,^[3] and most patients with nonsevere COVID-19 have a good prognosis.^{[4, 5]} However, due to the widespread transmission of the virus, a relatively large number of deaths have been reported due to acute respiratory distress syndrome and multiple organ failure.^{[4, 5]} Medical resources required to treat patients with nonsevere COVID-19 are relatively limited, while those required by severe and critical patients have increased significantly.^[6]

According to the China data in 2020, 81% of cases of COVID-19 were classified as nonsevere, 14% were classified as severe, and 5% were classified as critical illness.^[7] Based on these characteristics and situations, Fangcang shelter hospitals were developed and used for the first time in China to tackle the coronavirus disease 2019 (COVID-19) outbreak.^[8] However, inevitably, some patients with COVID-19 were initially diagnosed with nonsevere cases that eventually progressed. Thus, identifying the progression in the early stage would be significant.

By using a predictive model based on the characteristics of hospitalized patients with nonsevere COVID-19 at the time of first medical evaluation to predict disease progression of COVID-19 from a nonsevere type into a severe type, it might be feasible to identify high-risk patients at the very beginning of the disease and helpful to support the early allocation of medical resources and evidence-based decision-making. However, many prediction models or risk factors have been identified based on the characteristics and outcome of COVID-19.^{[4, 7, 9,10,11,12,13,14,15]} The results varied, which might contribute to the complexity and the different variants and regional distribution of COVID-19. However, screening the possibility of the progression of nonsevere COVID-19 into severe COVID-19 based on the simplified index available in most medical institutions independent of the regions would be helpful for the early treatment and better prognosis of COVID-19. Such predictive tools would be especially useful in Fangcang shelter hospitals and general practitioners’ offices. Nomograms are easy to use and can facilitate management-related decision-making.^[13]

In this study, we aimed to build a nomogram with very simplified variables obtained at the time of the first evaluation of patients with COVID-19 (including symptoms, comorbidities, routine blood tests, C-reactive protein [CRP] levels, and chest radiography) that can provide an individualized, evidence-based, highly accurate risk estimation,^[16] and these variables were available for most temporary medical institutions, especially in Fangcang shelter hospitals. To the best of our knowledge, we have established the first nomogram for predicting the risk of the progression of COVID-19 from a nonsevere to a severe type based on the data extracted from Fangcang shelter hospitals.

METHODS

Study design and participants

This was a multicenter retrospective cohort study. This study was performed to develop and validate a prediction tool as a nomogram for evaluating the progression of nonsevere COVID-19 cases into severe COVID-19 based on simplified data from the patients’ first visit to a medical institution due to COVID-19, which was limited to history, physical examination, routine blood tests, CRP, and chest radiology. The study was approved by the research ethics committees of the First Hospital of China Medical University and followed the Declaration of Helsinki. The primary outcome of this study was defined as the progression of COVID-19 from a nonsevere type to a severe type during the follow-up.^{[17, 18]}

For diagnosis of severe COVID-19 group, at least one of the following conditions should be met according to WHO guidelines, complemented by the COVID-19 Diagnosis and Treatment Guidance (2020) of China (version 6.0)^[18]: (1) respiratory rate (RR) ≥ 30 times/min; (2) arterial oxygen saturation (resting status) ≤ 93% in the resting state; or (3) the ratio of partial pressure of oxygen to fraction of inspiration O₂ (PaO₂/FiO₂) ≤ 300 mmHg (1 mm Hg = 0.133 kPa), as the other studies did.^[19]

The inclusion criteria were as follows: (1) diagnosis of COVID-19 confirmed by positive specific RT-PCR for SARS-CoV-2; (2) diagnosis of nonsevere COVID-19 according to the 7th edition of the medical guidelines from the National Health Commission of the People's Republic of China; (3) age ≥ 18 years; (4) patients with detailed clinical records at the first medical evaluation^[18]; and (5) patients with at least a 28-day follow-up period.

The exclusion criteria were as follows: (1) incomplete data from the first visit to the medical institutions; (2) worsening of the clinical condition mainly due to other diseases but not COVID-19; and (3) classification of the severe or critically severe type of COVID-19 according to the data achieved at the first medical evaluation. If missing data for a certain feature or sample is more than 5%, then we leave that features or sample out.

We collected data from a total of 495 patients with COVID-19 from different designated hospitals: Wuchang Fangcang Shelter Hospital, Tongji Hospital of Tongji Medical College and Union Hospital of Tongji Medical College in Wuhan, Hubei, China, between February 10, 2020 and March 8, 2020 (Figure 1). Data were extracted from electronic medical records.

Figure 1

Study flow chart.

The development and validation of the nomogram adhered to the guidelines in the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (Supplementary Table S1).^[16]

RESULTS

Development of the nomogram

All the risk factors were included in the univariate analysis to analyze their association with the progression of nonsevere COVID-19 (Table 3). The results of the univariate analysis with P < 0.10 were included in the multivariate analysis (Table 3). The nomogram was developed with the nine factors included in the final model (Figure 2 and Supplementary Figure S1). The formula for the total risk point was as follows:

1.12 × Age (years) − 13.33 × Sex (Male 0 or Female 1) + 21.19 × Diabetes mellitus (No 0 or Yes 1) + 32.1 × Dyspnea (No 0 or Yes 1) + 20 × Tachycardia (No 0 or Yes 1) − 39.62 × (Lymphocyte count − 3.5) + 18.33 × Anemia (No 0 or Yes 1) + 0.1 × CRP (mg/L) + 35.9 × Multilobar involvement (No 0 or Yes 1).

Table 3

Univariate and multivariate analyses for association with progression of COVID-19 from a nonsevere type to a severe type

	Univariate analysis		Multivariate analysis
	OR (95% CI)	P value	OR (95% CI)	P value
Sex (male/female)	0.71 (0.45–1.12)	0.143	1.06 (1.03–1.08)	< 0.001
Age (years)	1.06 (1.04–1.08)	< 0.001	0.54 (0.29–1)	0.049
Hypertension	1.74 (0.96–3.15)	0.068	0.78 (0.35–1.71)	0.534
Diabetes mellitus	2.82 (1.26–6.31)	0.011	2.92 (0.96–8.85)	0.059
Cough	1.45 (0.86–2.46)	0.163	..	..
Dyspnea	4.57 (2.6–8.04)	< 0.001	4.33 (2.06–9.09)	< 0.001
Fever	2.08 (1.19–3.62)	0.01	1.38 (0.64–2.98)	0.414
Tachycardia	4.33 (2.18–8.61)	< 0.001	2.58 (1.03–6.46)	0.044
Tachypnea	6.53 (1.33–31.97)	0.021	1.28 (0.18–9)	0.803
Leukocyte count (×109)	0.99 (0.89–1.1)	0.823	..	..
Lymphocyte count (×109)	0.19 (0.11–0.33)	< 0.001	0.26 (0.13–0.52)	< 0.001
Anemia	3.24 (1.37–7.65)	0.007	2.38 (0.76–7.44)	0.134
Thrombocytopenia	0.88 (0.32–2.41)	0.804	1.01 (1–1.01)	0.211
CRP level ⩾ 20 mg/L	1.01 (1–1.02)	< 0.001	5.43 (2.72–10.83)	< 0.001
Multilobar involvement	8.15 (4.45–14.93)	< 0.001	0.26 (0.13–0.52)	< 0.001

1. COVID-19: Coronavirus disease 2019; CRP: C-reactive protein.

Figure 2

A nomogram to predict the risk of progression nonsevere coronavirus disease 2019. To use the nomogram, draw a vertical line to identify the corresponding points of each variable according to their actual status. Then, add the points for all variables and find the position on the total point axis. With the same line mentioned above, you can determine the risk of progression nonsevere COVID-19 with the initial medical evaluation results at the lower line of the nomogram. Tachycardia is defined as a heart rate ≥ 100 beats per minute. Tachypnea is defined as a respiration rate ≥ 24 breaths per minute. Anemia is defined as a hemoglobin level < 120 g/L for males and < 110 g/L for females. Multilobar involvement is defined as the involvement ≥ 3 lobes on a CT scan. Using the cutoff score of 129.9, the sensitivity and specificity for discriminating between those with a high and low risk of developing the progression of COVID-19 in the validation cohort were 65.6% and 85.7%, respectively.

The risk of each individual was assessed according to the above equation. The probability of a case with no disease progression transforming into a case with disease progression could be calculated as follows: 1/(1 + e–z) (e = 2.718 and z = −7.439).

Validating performance of the nomogram

Based on the nomogram built in our study, a score was calculated for each patient, and these scores were used for the following analyses. In the development cohort, the area under the curve (AUC) of the nomogram scoring system for predicting the progression of nonsevere COVID-19 was 0.875 (95% CI 0.836–0.913) (Figure 3A).

Figure 3

Receiver operating characteristic curve of the prediction nomogram. (A) In the development cohort, the AUC of the nomogram scoring system for predicting the progression of COVID-19 was 0.893 (95% CI 0.858–0.928); (B) In the internal validation cohort, the AUC of the nomogram scoring system for predicting the progression of COVID-19 was 0.847 (95% CI 0.787–0.906).

In the validation cohort, the AUC of the nomogram scoring system for predicting the progression of nonsevere COVID-19 was 0.821 (95% CI 0.754–0.888) (Figure 3B). Using the threshold score of 178, the sensitivity and specificity for discriminating between those at high and low risk of disease progression were 85.1% and 65.6%, respectively. The Youden index J (J = Sens + Spec−1) was 0.506. PPV and NPV were 62.65% and 85.37%, respectively. MRS [MRS = 2 * (Sens + Spec–1) * π * (1-π), where π = prevalence of nonsevere to severe progression, which is 37.9%] was 24.0%. The nomogram shows a high-risk stratification ability for COVID-19, which has a higher disease progression prevalence.

Incorporating these variables, the nomogram achieved a good concordance index for predicting the progression of nonsevere COVID-19 cases in the development and validation cohorts (concordance index of 0.875 in the development cohort and 0.821 in the validation cohort) and had well-fitted calibration curves showing good agreement between the estimates based on the nomogram and the actual endpoint events (Figure 4A and B). To evaluate the clinical applicability of our risk prediction nomogram, clinical impact curve analysis (CICA) and decision curve analysis (DCA) were performed. The CICA (Figure 5A) and DCA (Figure 5B) visually showed that the nomogram had a superior overall net benefit within the wide and practical ranges of threshold probabilities and impacted patient outcomes.

Figure 4

Calibration plot showing the predicted probability of the risk of progression nonsevere Coronavirus Disease 2019. Bootstrapping was used to obtain bias-corrected (overfitting-corrected) estimates of the predicted versus observed values based on nonparametric smoothers. The three lines represented the ideal accuracy, the apparent accuracy, and the bias-corrected estimate of predictive accuracy. The bias was estimated due to overfitting or the “optimism” in the final model fit. After the optimism was estimated, it can be subtracted from the index of accuracy derived from the original sample to obtain a bias-corrected or overfitting-corrected estimate of predictive accuracy. (A) Development cohort; (B) Validation cohort.

Figure 5

Clinical impact curve analysis (CICA) and decision curve analysis (DCA) of the prediction nomogram. (A) Clinical impact curve analysis (CICA). The clinical impact curve of the nomogram plots the number of COVID-19 patients classified as high risk, and the number of cases classified high risk with severe NCAP at each high-risk threshold. (B) Decision curve analysis (DCA). DCA compares the net clinical benefits of three scenarios in predicting the severe COVID-19 probability: a perfect prediction model (gray line), screen none (horizontal solid black line), and screen based on the nomogram (red line).

DISCUSSION

In this study, we built a nomogram that showed good performance in predicting the progression of nonsevere COVID-19 and good agreement between the prediction and observations in identifying the probability of progression of nonsevere COVID-19. As the manifestations and severity of COVID-19 vary from asymptomatic cases to severe cases^{[4, 25]}, and if the progression of the nonsevere type into the severe type could be predicted earlier just based on the simplified risk factors, the medical resources could be allocated more reasonably, the identified high-risk patients could be given critical care more immediately, and a better balance could be achieved by epidemic control, disease severity evaluation, and the allocation of medical resources. Because this method is simple and easily practicable based on physical evaluation and the limited associated examinations, such as routine blood tests and radiological changes, which are readily available, it is suitable and helpful for use in temporary hospitals such as Fangcang shelter hospitals and primary health-care centers during the pandemic.

The median time from illness onset of COVID-19 to the first recorded medical evaluation was 5–6 days, while the median time from illness onset to the progression of the disease was 11 days in our study. In a previous study, the median time from illness onset to acute respiratory distress syndrome (ARDS) was 12 days.^[26] Thus, if the nomogram was used, there would be approximately 5 days in advance to predict progression and intervene in patients likely to experience progression of the disease.

Many prediction models have been built for the severity determination or the progression risks of COVID-19,^{[26, 27]} but the predictors and methods differ based on the index involved in different studies based on different clinical settings with different sample sizes.^[28] In addition, all the prediction models were appraised to have a high risk of bias owing to a combination of poor reporting and poor methodological conduct for participant selection, predictor description, and statistical methods used, and 16.67% of the 66 reported prediction models had external validation, and calibration was rarely assessed.^[28] Thus, in our study, we enrolled COVID-19 patients in the development and external validation cohorts to develop a nomogram for the evaluation of the progression of nonsevere COVID-19 into severe COVID-19. In addition, we only employed the readily available index acquired in the Fangcang shelter hospitals based on the first simplified evaluation of the patients. Thus, only disease history, symptoms, vital signs, routine blood tests, CRP, and radiological images were involved.

In total, seven predictors in multivariate analyses were independently associated with the progression of COVID-19 from a nonsevere COVID-19 to a severe type in our study. Based on a backward step-down selection process, nine variables were included in the nomogram in our study according to the complex evaluation of the prediction risk factors. Diabetes mellitus and anemia, which were not fully significant in the initial multivariate analyses, were also included in the final model. Comorbidities are risk factors for a poor outcome.^{[15, 26, 29]} Regarding comorbidities, hypertension (17.5%) and diabetes mellitus (9.1%) were the most common^[29] and may be risk factors for patients with severe disease compared with those with the nonsevere disease,^[30] which is in accordance with the comorbidity predictors from the univariate analysis associated with the progression COVID-19. However, in our study, in the multivariate logistic regression analysis performed to develop the nomogram, hypertension was not selected, which might be because of an overlapping association of these comorbidities with elderly age. Diabetes outweighed hypertension, possibly because diabetes mellitus not only influences the immune status of patients but also overlaps with older age.

In our study, we built a nomogram with age and sex, and male sex increased the weight of the probability of disease progression. Age is reported to be one of the significant risk factors for a poor outcome,^{[15, 26, 29]} while the sex impact varies. Based on a systematic review with a pool of 31 studies, age and sex were involved in 12 and 6 prediction models, respectively, targeting the prediction of the risk of hospital admission, diagnosis, and prognosis of COVID-19.^[28] In one study, it was found that the optimal cutoff of age for predicting imaging progression on chest CT was 51 years, with a sensitivity, specificity, PPV, and NPV of 0.65, 0.58, 0.35, and 0.83, respectively, and the ROC curve revealed that the AUC of age in the prediction model was 0.6.^[31] Earlier studies indicated that males might be more common in COVID-19 patients due to occupational exposure, but subsequent studies revealed sex equivalence.^[21] Furthermore, androgen-regulated gene, transmembrane protease, and serine 2 (TMPRSS2) expression in lung tissue expressed mainly in the adult prostate may explain the increased susceptibility of males to severe COVID-19 complications. Moreover, angiotensin-converting enzyme 2 (ACE-2) acts as a functional receptor for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and male hormones are effective in the ACE-2 passageway and simplify SARS-CoV-2 entry into host cells.^[32] Thus, we included both age and sex in the nomogram for the prediction of the progression of nonsevere COVID-19 to severe COVID-19, with older age and male sex as the weighing factors for the progression of COVID-19.

In many studies, dyspnea was not the most common symptom of COVID-19 due to the various clinical spectra of COVID-19 ranging widely from nonsevere illness to ARDS. On admission, the proportion of patients with symptoms of dyspnea (70.6% vs. 24.7% in the nonsurvivor group and recovered groups, respectively) has been reported to vary.^[33] However, it was suggested that patients with dyspnea should be closely monitored, especially 1–2 weeks after symptom onset.^[34] Based on the univariate and multivariate analyses in our study, dyspnea was selected into the nomogram for the evaluation of the risk for the progression of COVID-19, especially in the early stage, and it is important to evaluate the symptoms of dyspnea.

Tachycardia was identified as one of the predictors in the nomogram. Surely, tachycardia is an index of and response to stress and hypoxia. Furthermore, the cardiac injury was identified as a common condition (19.7%) in hospitalized patients with COVID-19 that was associated with worse severity, mortality, and multiple organ dysfunction, and greater proportions of patients with cardiac injury required noninvasive mechanical ventilation or invasive mechanical ventilation than those without cardiac injury.^[35]Additionally, abnormally higher hypersensitive troponin I (>0.04 pg/mL; OR = 4.388) was associated with unfavorable clinical outcomes.^[36] Myocarditis is depicted as another cause of morbidity among COVID-19 patients.^[37] In addition, drugs currently used to treat COVID-19 are known to prolong the QT interval and can have a proarrhythmic propensity.^[38] Thus, the monitoring of tachycardia might be helpful for the prediction of the severity of the disease and cardiac complications.

The weight of lymphopenia agrees with the findings of a previous study that showed that 85% of critically ill patients with COVID-19 presented with lymphopenia.^{[4, 26, 39]} The longitudinal hematologic variation of the progression of COVID-19 showed that initial levels of absolute lymphocyte count were significantly lower in nonsurvivors regardless of the initial disease severity.^[40] Endothelial dysfunction has been shown to induce disassembly of intercellular junctions, endothelial cell death, and blood–tissue barrier disruption, along with enhanced leukocyte adhesion and extravasation, which could contribute to the lymphopenia observed in patients with severe COVID-19.^[41] It has been hypothesized that successful survival may be related to the adequate replenishment of lymphocytes that are killed by SARS-CoV-2.^[42] In addition, anemia, which was one of the predictors in our nomogram, has been distinguished as one of the risk factors for mortality in inpatients in Wuhan, although hemoglobin has not.^[26] Hemoglobin levels were lower with older age, a higher percentage of subjects with diabetes, hypertension and overall comorbidities, and admission to intensive care,^[43] which is in accordance with the risk factors in our study to evaluate the progression of the disease. Compared to moderate cases, severe COVID-19 cases had lower hemoglobin and red blood cell counts and higher ferritin and red cell distribution widths.^[43] Interestingly, thrombocytopenia is associated with an increased risk of severe disease and mortality in patients with COVID-19 with a greater than fivefold increased risk;^{[26, 44]} however, only lymphopenia and anemia were included in the nomogram in this study, which outweighs the platelet count.

It is common for COVID-19 patients to have bilateral lung involvement and multilobar involvement on CT scans from the very beginning (28%), in the intermediate stage (76%), and in the late stage (88%).^[45] Thirteen prediction models of the 66 prediction models reviewed were proposed to support the diagnosis of COVID-19 or COVID-19 pneumonia (and monitor progression) based on CT images.^[28] The predictive performance varied widely, with estimated C-index values ranging from 0.81 to nearly 1. Importantly, the severity and area of opacification assessed on initial CT scans were significantly related to the progression of opacification on follow-up CT scans.^[46] The ROC curve showed that the sensitivity and specificity of the CT score were 80.0% and 82.8%, respectively, for discriminating the nonsevere type and severe type.^[47] Thus, the initial involvement of the segments on the CT scan indicated the risk of disease progression, which corresponded with our results.

A nomogram could supply a quantitative tool for risk stratification instead of categorized classification. In some studies, the progression or severity of COVID-19 patients was categorized into high risk or low risk, while some continuous variables, such as CRP, were divided by cutoff values, such as CRP >41.8 mg/L and were more likely to have severe complications.^[26] We did not classify the risks or continuous variables into categories such as high risk or low risk because it was suggested that risks calculated by the accumulative points of the risk factors would be more flexible for individual decision-making in different clinical settings with various case volumes and medical resources.^[14] In another nomogram building study, a nomogram was found to have good performance for the early prediction of severe COVID-19 (AUC 0.912/0.853 in the training and validation cohorts, respectively),^[48] but the parameters varied from ours without the enrollment of radiography in the analysis and a higher requirement for laboratory tests such as direct bilirubin, albumin, and lactate dehydrogenase, which are not quite readily available in some primary medical settings.

Some limitations of this study should also be acknowledged. First, this was a retrospective study in which selection biases were unavoidable, although we included a large number of patients from multiple sites. The study was designed as a prospective study. However, the rapid control of COVID-19 in 2020 left the COVID-19 patients scattered in different regions and restricted the prospective external validation; thus, we reregulated the design as a retrospective validation. Nevertheless, the validation performance of the nomogram revealed a superior overall net benefit within the wide and practical ranges of threshold probabilities and impacted patient outcomes. However, if possible, a prospective study is needed to validate the reliability and generalizability of the nomogram. Second, the model was developed and validated using data from patients in Wuhan, China. There might be significant regional differences in the mortality rate and percentage of severe cases in different regions. However, based on one meta-analysis, with a wide range of severe types of COVID-19 (7.2–63.3%), with respect to the severity cohort, nonsurvivors had more significant decreases in lymphocyte count and reduced hemoglobin values than survivors, with no significant difference in mean difference between groups.^[49] To ensure simplicity and practicability, we did not take all the laboratory indices into account, but instead, we employed the simplified indices available in the primary medical institution, and some might be relatively more important with regard to the severity of the disease. In addition, the symptoms and histories were mainly from patients’ self-report at admission, which might lead to recall bias. Finally, the exclusion criteria might lead to bias that originated from those of even sicker patients with severe complications and was classified as progression due to other diseases but not COVID-19 and those of the nonsevere type without the minimum requirement of the follow-up period. However, our study used bootstrap resampling and an internal validation cohort and showed an ideal C-index and well-fitted calibration curves, indicating good consistency between the prediction and observation.

Herein, a nomogram was established to predict the progression of nonsevere COVID-19 built with a simplified index of age ≥ 65 years, comorbid diabetes, dyspnea, tachycardia, absolute lymphocyte count, anemia, CRP level ≥ 20 mg/L, and multilobar involvement (≥ 3 lobes) on chest CT. The proposed nomogram might aid in predicting the progression of nonsevere COVID-19; thus, COVID-19 with a high risk of disease progression could be identified in time, allowing an appropriate stratification therapeutic choice according to the potential disease severity. Future studies should be performed to confirm or explore the nomogram with a higher sample size.