Development of a disease diagnostic model to predict the occurrence of central precocious puberty of female

Subjects and group stratification

This is a retrospective study. A total of 1,102 young girls, ranging from 6 to 9 years of age, who presented at the outpatient department of Beijing Children’s Hospital between January 2017 and October 2022 with the primary concern of “breast development,” were incorporated into the research sample. Comprehensive clinical information and the progress of follow-up were meticulously evaluated. The participants experiencing “breast development” underwent thorough examinations, including breast and pelvic ultrasounds, bone age assessments, and laboratory tests for hormones such as sex hormones and thyroid hormones. Following the monitoring period, the participants were divided into two distinct groups: the PD cohort and the PT cohort, with the diagnostic criteria for these outcome groups outlined as follows.

Due to the retrospective nature of the study, committees of Beijing Children’s Hospital waived the need of obtaining informed consent. The study protocol conformed to the ethical guidelines of the Declaration of Helsinki. The study was approved by the Ethics Committee of Beijing Children’s Hospital, Capital Medical University (No.2020-k-180).

Diagnostic criteria

The diagnostic benchmarks for PD with the manifestation of secondary sexual traits in females under the age of 9, are outlined in the 2022 Chinese clinical guidelines [1]. For the diagnosis of PT, the girl exhibited premature breast development before reaching 8 years of age, details of these criteria have been previously documented in my personal published work [2].

Observations

Each participant was thoroughly questioned regarding their health records followed by a comprehensive physical assessment, which involved measurements of height, weight, and calculation of body mass index (BMI). The BMI was derived using the standard equation [BMI (kg/m²)=weight (kg)/height² (m²)]. The maturation of breast tissue was evaluated based on the Tanner scale and tracked at quarterly intervals until the age of 9. Concentrations of sex hormones such as follicle-stimulating hormone (FSH), luteinizing hormone (LH), estradiol (E2), and progesterone were assayed via immunochemiluminescence at three-month intervals. Pelvic imaging via ultrasound was conducted quarterly by a skilled endocrine sonographer. The assessment of bone age (BA) was carried out every half-year using the G-P technique by a consistent pediatric endocrinologist throughout the study.

Potential predictive variables

Potential predictive variables included the following patient characteristics: anthropometry, clinical signs and symptoms, laboratory findings, and imaging results. Anthropometry variables included chronological age (CA), height, height SDS, weight, weight SDS, BMI and BMI SDS. Clinical signs and symptoms: Tanner staging of breast development. Laboratory findings included basal FSH, LH, and estradiol (E2). Imaging results included BA, uterine diameter, ovarian volume.

Variable selection and model construction

In the developmental cohort, 1,102 participants with breast development were incorporated into the variable selection phase and the subsequent model creation. As previously mentioned, a total of 14 factors were initialized into the variable selection mechanism. The strategy for processing missing data is to conduct multiple imputations using the mice package. Since the missing variables in this study are predominantly continuous variables, the predictive mean matching (ppm) method was selected for imputation.

The proportion of positive endings in the training set of this study was 36.9 %. To avoid building a prediction model in which the model gives higher weights to a larger proportion of the population, thus reducing the prediction effect on the positive samples, a relatively novel model training method was used in this study. First, the training set population was SMOTE oversampled to equalize the proportion of positive and negative samples. Then, variable selection and model training were carried out in the oversampled samples. This study uses support vector machine for modeling. To avoid overfitting when training the model, the study uses ten-fold cross-validation to train the model. At this point, the trained model has equal importance for positive and negative samples and there is no risk of overfitting. This model is applied to the original training and validation sets as the predictive effect of the model in the training and validation sets.

For variable selection, this study employed both logistic regression, LASSO regression, and random forest for variable screening. In logistic regression, variables with p<0.05 were selected. In LASSO regression, to select more streamlined variables and simplify the model, the study utilized a larger λ (λ 1se) as the criterion for variable selection. For the random forest model, the study selected variables within the top 50 % of the importance ranking. Eventually, the intersection and union of the variables screened by the three methods were taken. The SVM was fitted using the intersection of the variables screened by the three methods as the final prediction model, and the prediction effect was explored on the original training set and the validation set. In the sensitivity analysis, this study used the union of the selected variables of the three methods to fit the model and compared it with the previous model to verify the influence of the variable selection strategy on the results.

Assessment of predictive performance

The predictive performance of the model was assessed using sensitivity, specificity, accuracy and area under the receiver–operator characteristic curve (AUC).

Characteristics of the development cohort

Overall, the mean (SD) age of patients in the cohort was 7.69 (0.74) years. The comparative analysis of the physical status of the children in the two groups revealed that the PD group had significantly higher height, weight, and BMI as compared with the PT group. Children in the PD group had advanced BA, sex hormone levels, and pelvic ultrasound parameters. Their anthropometrics and laboratory values are listed in Table 1.

Predictor selection

Table 2 presents the results of modeling the screening of important variables related to CPP using each of the three methods. The details of the variable screening process for the three methods can be found in the Supplementary Material. LASSO screened the largest number of variables, with a count of 9. Logistic regression screened a slightly smaller number of variables, totaling 8. Random forests screened the least number of variables, precisely six. The intersection of the results of the three methods led to the following predictors of CPP: BA, LH, FSH, E2, uterine diameter, and ovarian volume.

Frisch [6] introduced the “critical weight hypothesis” in the 1970s, suggesting that a particular level of body fat reserve is essential for the onset of normal reproductive activity. In line with this, our research team [2] observed that children diagnosed with CPP exhibited an average BMI of 17.0 kg/m², which corresponds to the BMI of a normally developing girl at 10.5 years of age. Considering the vast population in China, incorporating BMI into the diagnostic model for the disease can help decrease the incidence of misdiagnosis to a certain degree.

So, the seven variables were included in the disease diagnostic model: BMI, BA, LH, FSH, E2, uterine diameter, and ovarian volume.

Construction of the disease diagnostic model

Based on the selected variables as mentioned above, this study employs 10 % cross-validation to construct a support vector machine model for the population following SMOTE oversampling. In this model, the radial basis function method is utilized. This method proves highly effective in handling nonlinear classification problems, as it can map data into a high-dimensional space to identify a more appropriate classification hyperplane.

The performance of CPP disease diagnostic model

When the developed model was applied to the development cohort, the corresponding prediction results are presented in Table 3. From the confusion matrix, regarding PT, the model correctly predicted 579 cases as PT and misclassified 53 cases as PD, with a total of 632 cases. For PD, the model misclassified 57 cases as PT and accurately predicted 312 cases as PD, amounting to 369 cases. The overall total sample size was 1,001 cases. The performance of the model was further evaluated using the relevant indexes. Its Accuracy was 0.8901, Sensitivity was 0.9104, Specificity was 0.8548, and AUC was 0.9457 (Figure 2), suggesting that the model performed well in differentiating between PT and PD.

Figure 2:

The receiver–operator characteristic (ROC) curve of CPP in girls with breast development was predicted using a model based on the variables included in the intersection set. CPP, central precocious puberty; ROC, receiver–operator characteristic; AUC, area under the receiver–operator characteristic curve.

Verification of the CPP disease diagnostic model

We assembled an external verification group comprising 226 young girls, ranging from 6 to 9 years of age, who were patients at either Luhe Hospital or the Pediatric Traditional Chinese Medicine Unit of Beijing Children’s Hospital. Within this group, 149 individuals (constituting 66 %) experienced premature isolated breast development, while 77 (accounting for 34 %) underwent precocious puberty. The characteristics of each variable are shown in Table 4.

The results of applying the model to the external validation group are presented in Table 5. In the prediction results, within the PT population, the model correctly identified PT in 119 cases and incorrectly identified PD in 30 cases, with a total of 149 cases. For the PD population, there were 19 cases where the model incorrectly identified PT and 58 cases where it accurately identified PD, totaling 77 cases, and the total sample size was 226 cases. The performance of the model was further evaluated using the relevant indexes. Its Accuracy was 0.7832, Sensitivity was 0.8623, Specificity was 0.6591, and AUC was 0.8357 (Figure 2). This value is slightly lower than the AUC value in the development cohort, yet it still exhibits a relatively good diagnostic performance in the external validation group.

Sensitivity analysis

To verify whether the variable strategy in this paper affects the model performance, a predictive model using the variables included in the union variables was employed in the sensitivity analysis. The performance of this model was tested in the development cohort and the external verification group, respectively, and then compared with that of the simplified model. It was found that the performances were close to each other see Supplementary Material SFigure 3.

Limitations

The current investigation has certain constraints, notably the employment of a narrow sample population when developing the model. Additionally, the data utilized for this research were exclusively sourced from China, potentially limiting the model’s universality across different global locations. It is imperative to gather additional data from diverse regions outside of China to substantiate the model’s efficacy.