Diagnostic scope: the AI can’t see what the mind doesn’t know

Introduction

When a patient presents with an uncertain constellation of signs and symptoms, a thoughtful clinician considers a broad range of potential etiologies in the differential diagnosis. To achieve an accurate and timely explanation for the underlying problem, the cause of these symptoms is hopefully contained within that list. Appropriately, most efforts to promote diagnostic excellence have focused on the diagnostic reasoning process at the level of a single clinical encounter. By contrast, the diagnostic scope is the aggregation of diagnoses over all encounters at the level of a geographic region, practice setting, or clinical specialty. However, relatively less attention has been paid to this critical feature of the diagnostic process. Additionally, no universally agreed-upon definition of diagnostic scope exists. Yet, the importance of diagnostic scope is actively growing as the potential for artificial intelligence (AI) and machine learning (ML) tools are being used in diagnostic clinical decision support systems (CDSSs). The diagnostic scope informs the development and training of diagnostic AI/ML CDSSs and therefore must be accurately specified in order for those systems to be useful in their desired context.

The importance of diagnostic scope for AI/ML CDSS development arises because of how such systems are trained. Predictive CDSSs typically require humans to specify the outcomes for which the computational models will learn patterns in the training data. For example, a data scientist building an AI/ML diagnostic CDSS for an infectious disease clinic would have to know ahead of time every diagnosis that might be encountered in a given practice setting. But characterizing this broad range of potential diagnoses is not straightforward. Humans, let alone computers, have yet to devise a system of diagnostic categories that are both readily analyzable in large clinical datasets and also useful for clinical decision making at the bedside. Thus, optimal human-AI collaboration for diagnostic excellence will require developing a shared language to describe the diagnostic scope. Patients, too, have a critical role to play in defining the diagnostic scope and setting priorities for its development.

Accurate assessments of diagnostic scope will foster efforts to enhance patient safety, optimize staffing models and clinic organization, educate clinicians in diagnostic reasoning, and improve the diagnostic process for patients and clinicians. In this Viewpoint, we review the current literature on diagnostic scope, propose a standardized definition, consider nuances in its measurement, and highlight critical challenges and areas for further research to realize the potential of AI/ML CDSSs to promote diagnostic excellence.

Defining diagnostic scope

We propose that the true diagnostic scope should be defined as a theoretical construct that includes all possible diagnoses in a given setting (Figure 1). The considered scopes reflect those diagnoses that clinicians and patients entertain during the diagnostic journey. And the observed scope reflects those diagnoses, correct or not, that are documented as present. Each of these scopes may overlap with and diverge from each other. For example, there may be a gap between the considered and observed scopes if some diagnoses are frequently included in the differential diagnosis or tested for but not actually made. Conversely, there may be some diagnoses that are not considered, but should have been, and are therefore still present in a population but not observed. This latter category includes over 10,000 rare diseases, or “zebras” that are frequently missed during the diagnostic process [1], 2]. Rare conditions such as inherited metabolic disorders or certain neoplasms constitute an important element of the true scope that may not be considered by clinicians. These diagnoses reflect false negative diagnostic errors. Each distinct and overlapping area of clinician-considered, patient-considered, observed, and true diagnostic scope offers unique opportunities to improve patient safety and the diagnostic process (Figure 1).

Figure 1:

The true diagnostic scope includes the full range of diagnoses present in a particular clinical setting. Patients and clinicians in a clinical setting each may consider a distinct range of diagnoses. The observed diagnostic scope reflects all documented diagnoses, right or wrong, in a specific setting. Each aspect of the diagnostic scope both overlaps and diverges from the others. Importantly, the Figure is not drawn to any particular scale.

Similarly, there is a distinction between the range of diagnoses that are possible and those that are likely. Therefore, a useful AI/ML CDSS will not only account for the range of diagnoses, but also the probability of each one. Thus, an accurate weighting of the true scope will support the diagnostic process because not every diagnosis in the true scope is equally likely to present during any individual encounter. However, this probability distribution will also likely vary by the practice setting due to heterogeneity in geographic or population-based disease prevalence.

The diagnostic scope has been described to varying extents in previous studies. Prior work has quantified geographic and specialty-specific variation in the content and breadth of documented diagnoses but has not accounted for weighting of the diagnostic scope by disease prevalence [3], [4], [5], [6], [7]. Nor have any previous studies distinguished between true, considered, and observed scopes. In a distinct but related sense of the term, the diagnostic scope has also referred to the range of conditions that a particular test might identify [8], 9]. In this sense, each AI/ML diagnostic CDSS system might analogously have its own diagnostic scope, aligned or not, with the true scope of the clinical setting in which it is used.

Dependence on local factors

The diagnostic scope in any given setting is dependent on local features of the population, including the age distribution, socioeconomic factors, environmental exposures, and endemicity of diseases. For example, one study of Medicare claims identified strong similarities between diagnostic scopes in rural and urban practices but also found large differences between specialty-specific settings, such as between obstetrician-gynecology and general surgery practices [4].

Access to care and referral patterns related to local healthcare infrastructure may also shape the observed diagnostic scope since clinicians may not identify conditions in patients who do not present to care. This presents a critical risk of systemic bias in data. Similarly, clinicians may disproportionately encounter early, presenting symptoms of a disease process in one setting (or among a particular population that is more likely to seek early care) that only later manifest fully with a diagnosis made in a different setting. This information gap is well known for clinicians who often lack diagnostic feedback about their decisions [10] and poses a related threat to accurately measuring the diagnostic scope. Moreover, the availability of resources to detect diseases may also contribute to the extent to which the considered and observed scopes diverge.

For example, lack of access to molecular testing is a key driver of the “diagnostic gap” in identifying tuberculosis [11].

Local differences also underscore the need for AI/ML systems that are trained and validated with data relevant to the populations for which they will be used. For example, while gallstones and alcohol use are the most common causes of pancreatitis worldwide, parasitic infection has been identified as the etiology in nearly a quarter of cases in some parts of the world with ascaris endemicity [12], and should therefore be considered in a useful diagnostic CDS in these settings. However, it likely falls outside the considered scope in most parts of the US and may be absent entirely from the observed scope, and thus also from the training data used for a US-based AI/ML CDSS. Even if the scope is perfectly defined in a given setting, and an AI/ML CDSS is trained with accurate, relevant data to predict that scope, the performance of the model will depend on where and when it is used. Thus, the diagnostic scope should be assessed for external validity or spectrum bias, analogous concept from the literature on predictive modeling and diagnostic testing, in which a tool’s accuracy may change in a population different from the one used for its development [13], 14]. Finally, only prospective evaluations of clinical effectiveness [15] will reliably estimate the accuracy of an AI/ML model’s scope in a given context. By contrast, in silico evaluations, or those that only measure a model’s predictive performance against outcomes in a retrospectively collected dataset [16], are insufficient to identify an AI/ML model trained on a mis-specified diagnostic scope.

Clinician-AI diagnostic reasoning

Clinicians often rely on a range of heuristics in decision making when faced with uncertainty [17], 18] and AI/ML CDSSs show promise to support such decisions. However, there is a circular relationship between the clinicians’ work of observing and documenting diagnosis and the training data that AI/ML models will use to provide decision support back to clinicians, potentially amplifying systemic bias and inaccurate diagnoses. Clinicians may miss important diagnoses if they are not frequently observed [19], leading AI/ML CDSSs to do the same.

Conversely, false positive diagnoses may occur if some diseases are considered in scope due to saliency or availability bias but infrequently observed. Again, AI/ML CDSSs will learn to make the same diagnostic errors. This risk may be further exacerbated by clinicians who insufficiently review the basis for recommendations from an AI/ML CDSS and follow them anyway. This phenomenon, known as “automation bias” is a key safety threat when using AI/ML CDSSs [20], 21].

Diagnostic AI/ML CDSSs, if trained on an accurately weighted diagnostic scope, are especially promising adjuncts to human clinicians during the diagnostic reasoning process. For example, a binary assessment of in/out of scope along with a quantitative assessments of a disease’s prevalence may guide decision making. This information could help to reduce cognitive errors due to the base rate fallacy (the tendency for a clinician to ignore the known prevalence of a disease in a population when estimating the probability of that disease in a patient) [22]. A nuanced framing of diagnostic scope and the interactions between types of scope (Figure 1) may prove useful in medical education to facilitate diagnostic reasoning and history elicitation. As AI/ML technologies are increasingly evaluated for their diagnostic capabilities and integrated into the clinical curriculum [23], [24], [25], the next generation of clinicians will require a sophisticated understanding of these concepts and the risk of circular relationships to maximize human-AI cooperation and accuracy in the diagnostic process.

For example, an AI/ML CDSS may not be useful during a pandemic in which a new disease becomes common if the CDSS was trained prior to the pandemic. In this case, the disease may not even have been in the observed scope reflected in the training data. Or it may have been in scope but much less prevalent. In either case, following the suggestions of this AI/ML CDSS during a pandemic would likely lead to false negative errors, or missed diagnoses. Automation bias would exacerbate this pattern and lead to further missed diagnoses. Even if the underlying AI/ML model were updated to reflect pandemic conditions, it may still lead to under-diagnosis by learning, and then recapitulating, the patterns of missed diagnoses early in the pandemic. For such AI/ML CDSSs to be effective at the bedside, they must reflect not only geographic but also temporal changes in the diagnostic scope [26]. And clinicians must remain vigilant to maintain their own independent cognition during the diagnostic process while simultaneously accounting for the limitations of a particular AI/ML system.

Learning from patients

While generally considered the purview of clinicians, development and assessment of diagnostic scope should also incorporate the unique knowledge and experiences of patients. Currently, a diagnostic error is defined as either failing to identify the correct diagnosis in a timely fashion or failing to communicate that diagnosis to a patient [27]. But this definition hinges only on the final documented diagnosis and does not account for any diagnoses that were considered during the diagnostic process, on any contributions of the patient to that process, or on patients’ knowledge of inaccurate diagnoses that were subsequently corrected at another healthcare center. Patients themselves may contribute important experiences and insights during the diagnostic journey that can inform not only clinician learning, but also organizational culture and practices that minimize communication breakdowns and promote patient- and family-centered care [28].

The model of diagnosis as “wayfinding” incorporates iterative information gathering, deliberation, and integration of insights from both patients and clinicians to eventually arrive at a correct diagnosis [29]. Listening closely to patients and their stories [30] can help identify elements of the true scope not otherwise captured in the clinician’s considered scope. A distributed cognition model that accounts for regions of the patient-considered scope that are outside of the clinician-considered scope can help to avoid diagnostic blind spots [31] (Figure 1). Taking the next step to proactively elicit, aggregate, and learn from patient experiences of diagnostic breakdowns can also enrich clinicians’ considered or observed scope, with the potential to reduce systemic bias and improve diagnostic accuracy and safety, especially if feedback systems “close the loop” with clinicians involved earlier in the diagnostic process [32], [33], [34], [35].

Symptom-focused internet searches are increasingly common and lead patients to consider their own differential diagnoses. In aggregate, and coupled with the patients’ own lived experiences, these may inform the patient-considered diagnostic scope (Figure 1). Patients are increasingly using AI for self-diagnosis. If patient and clinician AI/ML tools are trained on data with differing diagnostic scope, the tools may generate misalignments between patient and clinician diagnostic considerations. While a patient may be more likely to trust their clinician than an AI/ML CDSS, this preference may shift over time or with clinicians that patients do not know well or do not trust, potentially undermining the patient-clinician relationship [36]. Similar to the guidance for clinicians to share trustworthy internet sites with patients, organizations and clinicians may consider guiding patients to context-specific AI/ML resources. Healthcare organizations and individual clinicians will likely view the transparent development and reporting of the diagnostic scope of AI/ML CDSSs as assets in considering these recommendations.

Patient preferences should also inform prioritization of diagnostic scope research and quality improvement efforts. For example, benign, self-limited rashes may have high diagnostic uncertainty and yet may not be frequently represented in the observed scope because a true diagnosis is not definitively made. But such diagnoses likely produce little direct patient harm. By contrast, other diagnoses that are frequently missed or delayed despite their representation in the diagnostic scope, such as heart failure, urinary tract infections, and cancer, can result in significant harm and should receive priority [37], [38], [39], [40].

Measuring diagnostic scope

One reason that diagnostic scope may be understudied is that it is challenging to measure. A lack of open-source and clinically relevant ontologies has so far limited empiric evaluation. For example, a large study of primary care encounters across three countries used a proprietary mapping of diagnostic codes to clinically relevant clusters [5]. Such proprietary systems have limited reproducibility and transparency which preclude their widespread evaluation and validation in clinical practice. On the other hand, several freely available diagnostic ontologies are widely studied and easy to integrate into data analysis pipelines. These include the 61,048 Medical Subject Headings (MeSH) unique concepts as of 2024 [41] and the 71,932 International Classification of Diseases (ICD) - 10 codes [42]. However, such ontologies are so broad and specific as to be clunky at the bedside for clinicians and patients engaged in the diagnostic process. Thus, for purposes of measuring diagnostic scope for building AI/ML models, quality improvement efforts, or medical education, the optimal level of granularity and which diagnostic categories are most useful remain unknown. Finally, the validity of any reported diagnostic scope in practice will depend, to some extent, on both the methods and data sources used to measure it.

Promise and pitfalls of AI/ML tools

Once the relevant diagnostic scope is identified and measured for a given clinical setting, an AI/ML CDSS can be trained to identify patient factors that predict individual diagnoses. While AI/ML CDSSs offer promise to improve the diagnostic process through the ability to learn patterns in massive datasets and generate personalized recommendations, there are several ways in which an AI/ML model trained on an incorrectly specified diagnostic scope may exacerbate existing health disparities. Because AI/ML systems are built for scalability, they risk amplifying the errors in diagnostic reasoning discussed above. For example, acute coronary syndromes are much more frequently misdiagnosed in women compared to men. If an erroneous inclusion or weighting of such diseases in the diagnostic scope is used to train an AI/ML CDSS, that system will invariably reinforce the same diagnostic errors in practice that led to misdiagnosis in the first place [43], 44]. An AI/ML CDSS thus create an opportunity for the errors made by clinicians whose actions were captured in the training data to propagate to new settings and encounters. AI/ML approaches increase this risk, especially when the observed diagnostic scope overlooks sickle cell disease [45], malaria [46], cystic fibrosis [47], and other rare diseases [48].

At the same time, such systems also offer opportunities to mitigate disparities and the potential harmful effects of interpersonal bias, prejudice, or incomplete medical knowledge. Because most diagnostic errors are due to suboptimal history taking or physical examination [37], a properly trained AI/ML system could help to facilitate the diagnostic process and improve diagnostic accuracy by filling cognitive gaps or mitigating biases. These systems may be further strengthened by focused research to identify missed or delayed diagnoses in marginalized populations, considering access to care and delayed presentations and other intersectional factors such as English language proficiency and socio-economic disparities alongside clinician bias. In each of these cases, the diagnostic AI/ML system will only be as good as the data on which it is trained, which in turn, is created, curated, and chosen by humans, requiring principled approaches and multidisciplinary oversight [49], 50].

Tensions, challenges, and opportunities for diagnostic scope

The importance of diagnostic scope is increasingly recognized, especially in the context of developing AI/ML CDSSs to improve the diagnostic process. But several obstacles must be overcome before it becomes directly useful to researchers, clinicians, and patients. First, a pragmatic application of the true scope, weighted by both prevalence and clinical importance, must be identified for each setting. While almost all diagnoses could be present in most clinical contexts, those with infinitesimal prevalence are unlikely to be useful in developing diagnostic AI/ML CDSSs unless their clinical importance outweighs their rarity. For example, Ebola virus disease has temporally sporadic and geographically heterogeneous prevalence – even approaching zero in some locations – but its early diagnosis remains a key global health priority [51]. Further work is needed to better characterize how the diagnostic scope is sensitive to local factors and how it may vary in both content and breadth. Second, more work is needed to develop clinically meaningful diagnostic ontologies that will be useful at the bedside. A useful ontology will balance the simultaneous needs for generalizability and specificity. Third, improved measurement tools and data sharing policies will be needed to ensure accurate assessments of true, considered, and observed scopes. Transparency, interoperability, and protection of patient privacy are all necessary for successful measurement and analysis of diagnostic scope. Fourth, evolving governance frameworks for AI/ML CDSSs will need to ensure clinical effectiveness, safety, and equity. Only in such a regulatory environment will the benefits of future research efforts into diagnostic scope be realized.

In summary, the diagnostic scope is a key feature of the diagnostic process and will play an increasingly vital role with rising uptake of AI/ML CDSSs in healthcare, but it remains largely under-explored. More research is needed to better refine and quantify its role in clinical, cognitive, policy, and analytic contexts to improve patient care and outcomes through diagnostic excellence. Just as “the eye can’t see what the mind doesn’t know,” so too an AI/ML model can’t predict what the humans don’t tell it.