A clinically-guided unsupervised clustering approach to recommend symptoms of disease associated with diagnostic opportunities

Temporal visit trends prior to diagnosis

A common observation in studies of diagnostic opportunities is an increasing trend of visits with symptoms of a given disease (e.g., chest pain before AMI) prior to the index diagnosis/visit. This increasing trend has been found for multiple diseases including stroke [8, 27], AMI [7, 22, 27], tuberculosis [9, 10], endemic fungal infections [16, 18, 19], and many others [11, 17, 20, 21]. Figure 1 depicts an example of this trend for tuberculosis, where visits for cough and fever increase prior to an index diagnosis. When nearing the index date of a given disease diagnosis, one expects to observe more healthcare encounters where patients present with signs and symptoms of the disease. Such symptomatic visits ultimately lead to diagnosis, and many of these encounters represent diagnostic opportunities.

Figure 1:

Symptomatic visits for cough and fever prior to tuberculosis diagnosis. The number of patients with a healthcare visit for a given symptom are presented each day prior to the index tuberculosis diagnosis. Similar increasing patterns of symptomatic visits have been found across a wide range of diseases.

The goal of our methodological approach is to identify the set of antecedent conditions suggesting a given disease is present (i.e., diagnosis codes for symptoms or other diseases with similar symptoms) and where encounters for such conditions may represent diagnostic opportunities. Thus, our objective is to find conditions that satisfy the increasing trends depicted in Figure 1. We do so in a clinically-guided fashion where a single focal condition known to be associated with the disease of interest (e.g., cough prior to tuberculosis) is selected by an expert, then other conditions exhibiting similar trends recommended by the clustering approach are evaluated by the expert.

Data processing and curve fitting

For this study, we utilized data from the IBM MarketScan Research Databases from 2001 to 2015. These data represent longitudinal health insurance claims in the United States covering inpatient and outpatient settings. We identified all patients diagnosed with the diseases of interest below. We computed the number of healthcare visits every day for each of the top 500 most common ICD-9-CM diagnosis codes during the year before the index diagnosis. To remove unrelated observation effects and create comparable model fits across codes we applied two data transformations. First, we converted daily counts of healthcare visits to relative frequencies by dividing the daily counts by the number of patients with a healthcare visit each day prior to diagnosis. Supplemental Figure 1 depicts how this can eliminate unrelated conditions appearing to increase due to patient observation. Second, we normalized the relative frequencies by subtracting the minimum and dividing by the range for each code.

Next, we estimated the temporal trends in the daily relative frequencies leading up to diagnosis using a piecewise linear regression model, where separate linear trends are estimated before and after a certain number of days prior to diagnosis (i.e., change-point). We enforced a continuity requirement on the model so the fitted trend is continuous before and after the change-point. We used the Akaike Information Criterion to select the optimal change-point. We apply this procedure to fit separate models for each of the top ICD-9-CM codes. The estimated parameters of these models are used in the clustering process. Each resulting model contains 4 parameters: a change-point, intercept, and two slope parameters (before and after the change-point).

Guided unsupervised clustering

Unsupervised clustering is a category of machine learning techniques that can be used to identify natural clusters of data that share similar characteristics and a relatively common approach is the k-means algorithm. Figure 2 provides a visualization that demonstrates the theoretical basis for how k-means clustering is applied to visit trends prior to diagnosis.

Figure 2:

Visual depiction of the theoretical reasoning behind the clustering algorithm. Figure A depicts the empirical pattern of healthcare visits for related symptoms before the index disease diagnosis (data pictured correspond to tuberculosis). There is an increase in symptomatic healthcare visits before diagnosis. The trend is estimated with two curves: The first segment (flatter) captures the period where clinical disease is unlikely to be present, the second segment (steeper) captures symptoms of clinical disease and potential missed opportunities. Figure B depicts how the k-means clustering algorithm is applied to the trends for each potential antecedent condition (note, this is an oversimplified depiction where only the two slope parameters are used to identify k=3 clusters). The central plot depicts examples of clusters of conditions identified based on the slope parameters. The plots on either side of the clustering graph depict examples of trends that might fit the patterns of conditions shown in the corresponding cluster colors. The cluster containing the focal condition, such as cough (highlighted in red) has a slope near zero in the first period, and a large positive slope in the period right before diagnosis. The other two clusters contain slopes that are near zero in both periods (blue) or have a lesser slope in the second period (green) compared to those in the focal cluster.

For each of the top ICD-9-CM codes in consideration, we applied the k-means clustering algorithm to the estimated parameters from the fitted models. Thus, we use the clustering algorithm to find clusters of ICD-9-CM codes that have trend parameters most similar to one another. We use the kmeans function as part of the stats package in the R programming language, with the default Hartigan and Wong algorithm [28]. We set an initial number of random cluster centers of 50 and a maximum number of iterations of 100. After identifying clusters for a given value k, we isolate the cluster containing the focal symptom of each disease, described below. Finally, we use both expert review and prior literature to determine which codes in the focal cluster are clinically plausible signs the disease may be present.

In order to provide a reproducible example, we have developed scripts for the R programming language that can be used to replicate all of the general approaches described in this analysis. This code can be found at https://github.com/aarmiller/diagnosis_cluster, along with synthetic data to demonstrate each stage of this analysis.

Example applications

To demonstrate the feasibility of our clustering approach, we selected two diseases with differing characteristics in trends of diagnostic opportunities – one where diagnostic delays occur over multiple weeks (tuberculosis) and the other where delays last multiple days (appendicitis). For each disease, we select a single well-known and common symptom to initiate the search process and select the focal cluster. We validated each case study populations by requiring one inpatient or >2 outpatient diagnoses to identify the index diagnosis.

Tuberculosis: Tuberculosis is a highly infectious communicable disease that is a major cause of morbidity and mortality, and is associated with over 1.4 million annual deaths worldwide [29]. Missed opportunities to diagnose tuberculosis are relatively common, often lasting up to a few months [9, 10, 30, 31], and have important public health implications [32]. Supplemental Table 1 provides the list of ICD-codes used to identify index cases of tuberculosis. The primary symptom associated with pulmonary tuberculosis selected as our focal condition is cough, ICD-9-CM code 786.2. Because the number of index cases of tuberculosis were relatively low and the delay duration can last multiple months, we aggregated daily counts of each ICD-9 code at a weekly level. We consider potential missed opportunities based on antecedent conditions within 90 days prior to the index diagnosis.

Appendicitis: Acute appendicitis is an inflammation of the appendix and one of the most common causes of emergency abdominal surgery [33, 34]. Most diagnostic delays occur within days, or at most a few weeks, of the index diagnosis [23, 35]. Delays in the diagnosis and treatment of appendicitis can be costly, potentially leading to perforated appendicitis [36, 37]. The primary symptom associated with appendicitis is abdominal pain, for which we used the ICD-9-CM code 789.00 (abdominal pain, unspecified site) as the focal condition. We consider potential missed opportunities based on antecedent conditions within 21 days prior to the index diagnosis.

Performance evaluation

To evaluate our approach, we compare different values for k from 2 to 25. For each of the resulting focal clusters we describe [1]: the size of the cluster [2]; the number of conditions that were clinically plausible, and [3]; the clinically-plausible precision, defined as the percent of conditions in a given cluster considered to be clinically plausible for diagnostic opportunities. Conditions were labelled as clinically plausible if they were either [1] a known sign or symptom of the disease or [2] a disease with similar symptoms (e.g., pneumonia instead of TB). To evaluate the stability of clusters for different values of k, we repeated our clustering approach 10,000 times for each k and report the 0.05 to 0.95 quantiles of resulting cluster measures. In addition, we compute the number of potential missed opportunities that may be captured using the cluster-based list of antecedent-conditions vs. the single focal symptom suggested by expert review (note, we refer to these as “potential” since the presence of symptoms during visits prior to diagnosis does not necessarily imply a missed opportunity).

Example 1: Tuberculosis

We identified 19,423 patients with an index diagnosis of tuberculosis between 2001 and 2015. In total, these patients had 325,039 healthcare visit dates in the year prior to the index tuberculosis diagnosis. Figure 3 depicts the raw counts and relative frequency of visits for the focal symptom of cough prior to the index diagnosis. Figure 3 also depicts the fitted curve using the piecewise linear model, for which the objective of the clustering approach was to identify antecedent conditions with similar model parameters.

Figure 3:

Visits for focal condition of cough prior to tuberculosis. The left figure depicts the raw counts. The right figure depicts counts converted to relative visit frequency along with the linear change-point model used to fit the trend and derive parameter estimates used for the cluster analysis.

When applying the clustering approach, it is necessary to select a number of clusters k from which to obtain recommendations from the focal cluster. Supplemental Figure 2 provides a summary of results in terms of the overall size, number of clinically-plausible conditions, and clinically-plausible precision of the resulting focal clusters, across different values of k and clustering trials. We selected k=11 as a type of kink-point in the tradeoff between cluster size and clinically-plausible precision. For k=11, the level of clinically-plausible precision achieves a near maximum at 70.8%, suggesting a potential reviewer would have few unrelated conditions to exclude. This focal cluster also results in a reasonably large number of clinically-plausible conditions (on average 34.1 conditions) recommended to a reviewer.

In total 88 conditions were identified as being clinically plausible across all clustering trials. The top 25 most frequently identified conditions are outlined in Table 1. This table also provides the frequency of each condition appearing in the focal cluster for k=11. As seen in Table 1, many of the commonly associated antecedent conditions, such as fever, hemoptysis, pneumonia or other pulmonary symptoms (pneumothorax, lung abscess, mass or neoplasm), were consistently identified across most focal clusters. Each of these top 25 conditions was selected in every focal cluster for k=11. Figure 4 depicts visit count trends for 9 of the top 25 antecedent conditions in the focal cluster.

Figure 4:

Examples of trends in top antecedent conditions selected in the “cough” focal cluster prior to tuberculosis. The black dots depict 7-day average counts of visits with the given diagnosis relative to visit frequency. The linear piecewise model used to fit the trend and derive parameter estimates for the cluster analysis is depicted by the red line (see Supplemental Figure 3 for the remaining top 25 conditions).

To evaluate the potential effectiveness of our approach for suggesting criteria to identify potential diagnostic delays, we evaluated the number of visits and patients that would be captured by different sets of antecedent conditions within 90 days of the index diagnosis. Supplemental Figure 4 depicts the range in potential missed opportunities and patients identified using the sets of clinically plausible antecedent conditions recommended by each trial. The number of potential missed opportunities using the focal symptom of “cough” was 4,382 healthcare visits from 2,842 patients. Expanding to the set of conditions identified in the kink-point cluster (k=11) resulted in 31,162 visits from 9,078 patients representing a potential missed opportunity. Using the entire set of 88 plausible antecedent conditions recovered from our cluster-based approach resulted in 49,063 visits from 11,386 patients representing a potential missed opportunity. Thus, the prevalence of potential diagnostic delays identified ranged from 14.6% of patients, using cough alone, to 58.6% using all clinically plausible conditions suggested by the clustering algorithm.

Example 2: Appendicitis

We identified 572,836 patients with an index diagnosis of appendicitis between 2001 and 2015 that had over 4.5 million healthcare visit days in the year prior to the index diagnosis. Figure 5 depicts the raw count, relative frequency and fitted curves of visits for the focal symptom of “unspecified abdominal pain” prior to the index appendicitis diagnosis.

Figure 5:

Visits for focal condition of unspecified abdominal pain prior to appendicitis. The left figure depicts the raw counts. The right figure depicts counts relative to visit frequency along with the linear change-point model fit to the data. Note: Counts are only depicted over the 180 days prior to diagnosis to better illustrate the trend. The fitted curves rely on all 365 days prior to diagnosis.

Supplemental Figure 5 depicts the results of our cluster analysis across different values of k and clustering trials. For values k≥4 there was a consistently high level of clinically-plausible precision >90%. We selected the value k=4 as the kink-point in the tradeoff between clinically-plausible precision (92.6%) and cluster size (54 total conditions and 50 that were clinically plausible).

A total of 63 clinically plausible conditions were identified across trials; Table 2 presents the top 25 conditions. The clustering approach identified many diagnoses known to be associated with symptoms of appendicitis, including specific sites for abdominal pain, vomiting, fever, nausea, leukocytosis, intestinal infection, or symptomatically-similar diseases such as gastritis, and pancreatitis. All of the top 43 clinically plausible conditions appeared in every focal cluster for k=4. Figure 6 depicts visit count trends for 9 of the antecedent conditions selected from the 43 conditions in the focal cluster.

Figure 6:

Examples of selected trends in antecedent conditions contained in the “abdominal pain” focal cluster prior to appendicitis.

The black dots depict 7-day average counts of visits with the given diagnosis relative to visit frequency. The linear piecewise model used to fit the trend and derive parameter estimates for the cluster analysis is depicted by the red line (see Supplemental Figure 6 for the remaining top 25 conditions).

We computed the number of potential missed opportunities within 21 days prior to the index appendicitis diagnosis using different sets of antecedent conditions. Supplemental Figure 7 depicts the resulting number of potential missed opportunities and patients identified using the antecedent conditions for different values of k. The focal symptom of unspecified abdominal pain identified 49,371 potential missed opportunities from 41,596 patients. Expanding to the additional set of 50 antecedent conditions identified in the cluster k=4 resulted in 137,003 visits from 98,111 patients representing a potential missed opportunity. Using the entire set of plausible antecedent conditions recovered from our cluster-based approach resulted in 142,359 visits from 101,013 patients representing a potential missed opportunity. Thus, the potential prevalence of identified diagnostic delays among our study population ranged from 7.3% of patients, using abdominal pain alone, to 17.6%, using all clinically plausible conditions suggested by the clustering algorithm.