Root cause analysis of cases involving diagnosis

Step 1: Deciding is this a diagnostic error?

Some diagnostic errors are easy to recognize. These are cases where the correct diagnosis became clear at some point, and looking back there is agreement that the correct diagnosis was missed or could have been made much earlier.

In other cases, it is more difficult to say that a diagnostic error occurred. For instance, there are very few clear guidelines about the timeliness of diagnosis. How long SHOULD it take to diagnose a particular infection, cancer, or cardiovascular condition? In many cases, the initial presentation is non-specific, and the condition and diagnostic process evolves over time.

There are now four definitions of diagnostic error in active use (Table 2). The Graber definition is foundational but can only be used in retrospect; the Schiff definition focuses on identifying the steps in the diagnostic process where errors occurred; Singh defines diagnostic error as a missed opportunity to have made the correct diagnosis, a definition that is now widely used in prospective research studies because it focuses on the diagnostic process, where the ultimate diagnosis is not yet known. The NAM definition focuses specifically on timeliness and accuracy, and adds the all-important patient viewpoint because the diagnostic process is not complete until it has been successfully communicated to the patient. If in doubt, a useful tool to help determine whether a case reflects a diagnostic error is the 12-question Revised Safer Diagnosis Checklist developed by Hardeep Singh and colleagues (Table 3) [12].

Table 3:

The revised safer Dx checklist.

Extensive debates on whether a particular case does or does not reflect diagnostic error are inappropriate; healthcare organizations should focus instead on understanding and improving the diagnostic process.

Step 2: Case triage: peer review or RCA?

In cases of medical error, organizations need to decide whether to evaluate the case from the perspective of improving quality and safety, or from a disciplinary perspective that is concerned with clinical competence. Use of a standardized approach such as the United Kingdom’s National Patient Safety Agency (NPSA) Incident Decision Tree, based on Reason’s “Culpability Tree” or Marx’s “Just Culture” model, can help organizations determine which incidents can be routed to safety analysis and which will require peer review/disciplinary action [16]. Marx’s “Just Culture” model can be summarized most simply as follows:

1. Console human error

2. Coach at-risk behavior

3. Punish reckless behavior

Triage decisions should be based on evaluating the individual decisions made in the case and the actions taken, not on the outcome of the particular case.

The bottom line is that in the absence of incapacity, deliberate harm, or a pattern of reckless behavior, most cases of diagnostic error should be prioritized for RCA not for peer or competency review.

Step 3: Convening the RCA team

The team responsible for conducting the RCA must have sufficient knowledge and experience to understand what transpired in a given case and be able to propose relevant solutions. As in system-related RCA practices, the team should comprise roughly 4–6 core members and invite ad hoc staff members or consultants as needed to understand key issues. A physician who understands the diagnostic process and is knowledgeable about the role of cognition in clinical reasoning should be part of the team, along with a nurse familiar with the care setting. A medical librarian should be included in cases involving knowledge-related issues [17].

The involved clinicians. The clinical staff who were involved in the incident should routinely be included as members of the RCA review team. These individuals have first-hand knowledge of what happened and are most invested in ensuring a comprehensive and detailed safety review that reaches appropriate conclusions and proposes actions to reduce future risk [18].

Physicians may be reluctant to discuss their role in diagnostic error cases and will need reassurance that these discussions are concerned with learning and practice improvement, not with criticizing and assigning blame. A guide for involving physicians and nurses in RCAs is included as Supplemental Material, Appendix A.

Patients and families. In system-focused RCAs, the patients or families who have been harmed by the safety breakdown may be interviewed but they are generally excluded from the RCA team. In diagnosis-related cases, involving patients or families in some fashion is a must. The patient or family can provide critical details on the events regarding the case and what was said, providing key contextual information that is typically missing from electronic medical record notes [19, 20]. There is also substantial value in restoring a relationship of trust, as the patient and family see that the organization is doing its best to understand what happened and how care can be improved going forward. Interviews with families after safety-related deaths have found that many want to participate in safety reviews and believe that their perspective is highly relevant to the analysis [21, 22]. A guide for involving patient/family members in the RCA process is available as Supplemental Material, Appendix B.

…assume that patients and families will be partners in investigation and where possible engage them fully from the beginning…

At a minimum, the patient or family should be interviewed. In some cases, depending on the patient or family and their interest, it may also be appropriate to include them on the RCA team. Including patients or families poses challenges [23], and in some states in the US, legal issues may preclude their involvement. The foremost challenge is handling the emotions and expectations that inevitably arise when the injured parties first meet with representatives of the organization. If the patient or family is willing to participate in the RCA, they will benefit from an orientation to the RCA process and what their role in this will involve.

An alternative to their direct participation is to have a patient advocate sit in for (or with) the patient or family during the RCA meetings. Experienced advocates will be familiar with medical language, standard medical care processes, and safety analyses, and can report back to (or explain things to) the patient or family as needed. Someone from the organization’s patient-family advisory council (PFAC) may be available to serve in this role or to help identify a suitable advocate.

Finally, it is very appropriate to check back with the patient or family as the analysis and action plans unfold and to share the RCA findings and recommendations; do they believe that the proposed interventions will solve the problem?

Step 4: Identify root causes and contributing factors

The steps for conducting RCA’s of diagnosis-related safety events are essentially the same as those in a system-focused RCA, although the particular details of some steps will differ.

The initial interviews and fact-finding. One key difference is the timing of the initial safety event review: The review should begin immediately to better capture the cognitive and human factors elements that might have contributed to the case. The involved staff and the patient and family should be interviewed as soon as possible because accurate recall of what happened and what else was happening at the time will fade within days.

A neutral party, ideally a peer, or someone with experience in these situations and knowledgeable about cognitive error, should conduct the interviews in a private setting, free from interruptions. The interviews should start with open-ended questions with little interruption, and the interviewees should be encouraged to freely associate about the event and try to recall their thoughts and feelings. Structured questions may be helpful, and questions should also probe the context of care [24]. The ASHRM Root Cause Analysis Playbook contains a detailed introduction to RCA interviews [5], and a starter set of questions are listed in the box below.

Using formal cognitive interviewing techniques (Supplemental Material, Appendix C) with the involved clinicians will help them recall more facts and insights about the case, and more details about specific issues. Organizations with sufficient resources should invest in training their safety staff in cognitive interviewing.

Where did things go wrong in the diagnostic process?

A good starting point for extracting lessons from diagnosis-related safety events is considering which steps of the diagnostic process worked well and which did not (Figure 2).

Figure 2:

Steps of the diagnostic process.

The best resource for this task is the ‘DEER’ taxonomy from Schiff and colleagues’ Diagnostic Error and Evaluation and Research project, which breaks out the steps of the diagnostic process (Table 4) [13]. Isolating where in the diagnostic process problems developed will be valuable to the safety review, allowing the RCA team to compare the actual event to best practices. Involving subspecialists can be helpful in this regard, and they may also be aware of other cases that involved similar breakdowns in a particular step. Aggregating data from multiple RCAs can help identify specific issues that are ripe for performance improvement initiatives.

Table 4:

The ‘DEER’ taxonomy.

Identifying root causes: The Fishbone Diagram. Once the RCA team has an initial understanding of what happened, they need to consider what factors contributed to the event. There are many different approaches that can be used to identify causal and contributing factors [25], [26], [27], [28], and a collection of different taxonomies is presented in Supplemental Material, Appendix D. Teams can use whatever approach they are most familiar with or that seems most appropriate for the case.

We recommend using an analysis framework that recognizes four dimensions: The context of care, the clinician’s reasoning process, the patient and family perspective, and the particulars of the case itself. Fishbone (Ishikawa) diagrams provide a convenient way to visualize at a glance the four domains, and this approach can be used for both cases of diagnostic success and error [29]. A generic version is illustrated in Figure 3, and the key domains and subdomains can and should be modified depending on the details of the case.

Figure 3:

A generic Fishbone diagram for considering root causes of diagnostic error.

The case . Each case is unique. The particulars of how the patient’s condition presents and evolves, how they describe it, and when they seek care can all determine whether the diagnosis will be easy and correct, or problematic. The exact same disease can present in uncommon ways in different patients. In some cases the speed of diagnosis is not important, and in others it can be critical, including cases of stroke, aortic dissection, and sepsis where delays can be disastrous. Cases that present in the classic, textbook manner will usually be recognized and diagnosed quickly and accurately. Conversely, diagnosis in cases that present in atypical fashion, or involve unusual or rare conditions will often be delayed.

The patient . Characteristics of the patient or the family can play a role in contributing to diagnostic error. Some of these reflect patient characteristics that evoke affective bias, where the clinician is put off by the patient’s age, sex, personality, or ethnic background, or perhaps by a coexisting mental health condition or a rude comment. Patients who are angry, drunk, or confrontational often evoke reactions and emotions on the part of the clinician that detract from clear clinical reasoning.

A common concern in cases of diagnostic error is whether the patient clearly communicated their symptoms. Communication failures can be encountered with infants, patients who don’t speak English, and patients who are intoxicated, intubated, or unconscious, among others. Communication problems can even occur in awake and alert patients if they misunderstand a question, or do not accurately explain their symptoms or course of illness. Patients missing scheduled appointments or tests can sometimes be a factor contributing to delayed diagnoses.

The context of care. The context of care includes a very wide set of factors that can support or sometimes degrade diagnosis. System-related issues are identified in most cases of diagnostic error [10, 30]. Safety officers typically have extensive experience exploring system-related aspects of safety events, and these same dimensions apply to diagnostic errors. An overview of system factors relevant to diagnosis is illustrated in Figure 4 [25] and a detailed approach is presented in Supplemental Material, Appendix E.

Figure 4:

A general framework for considering root causes of safety events.

Some of the most critical contextual elements include considering:

1. Access to care; Communication; Care coordination;

2. Access to expertise and second opinions; Access to appropriate imaging and tests;

3. Health informatics systems and resources;

4. Culture, especially teamwork; Human factors issues; and

5. Diagnostic setting and circumstances.

It is especially important to consider whether human factors, sometimes referred to as ‘error promoting conditions’ may have derailed the diagnostic process. Fatigue, stress, illness, production pressure, cognitive overload, burnout, distractions and a host of other human factor considerations are often identified as contributing factors in cases of diagnostic error. Identifying the role these factors may have played, and understanding how they arose may be the most important findings in a given case and may provide key insights for optimizing the diagnostic process in the future.

Clinical reasoning. Clinical reasoning is the clinician’s ability to synthesize all the available information of the case to arrive at the most likely diagnostic possibilities, based on his\her knowledge and experience. Faulty clinical reasoning is a factor in most cases of diagnostic error. This domain involves an exploration of the cognitive aspects of diagnosis, the part of the RCA process that differs most from reviews of system-related cases. The key elements of the clinical reasoning process are illustrated in Figure 5.

Figure 5:

The elements of clinical reasoning.

Knowledge: The ability to make a timely, accurate diagnosis depends on the ability to recognize or identify the condition based on knowledge acquired during training or experience. There are over 10,000 known conditions, but medical training, and most textbooks, typically cover only those that are common, less than 1,000 conditions. Although every clinician has probably seen the most common conditions, not every clinician will have learned about or seen unusual presentations of these conditions, or the many rare diseases that inevitably present at some point in time. Only 3 % of faulty diagnoses in one study were due to faulty knowledge [10].

Case information: Diagnosis requires obtaining a complete and detailed medical history, conducting an appropriate physical examination, understanding the diagnostic test results, and reviewing available consult reports. In that same study, 14 % of diagnostic errors involved situations where key data was either not available, not sought, or was available but misinterpreted [31]. Cases with handoffs predispose to problems in this domain because information is often lost or distorted passing from one person or care site to the next.

Synthesis: Synthesis represents the cognitive tasks involved in considering the diagnostic possibilities. Errors in this step may reflect either breakdowns in critical thinking (System 2) or in the subconscious, intuitive aspects of diagnosis (System 1) [24, 32, 33]. Faulty syntheses is by far the leading cause of diagnostic error, encountered in 83 % of cases [34].

System 1 and System 2: Fast and slow thinking. The dual processing paradigm, thinking fast vs. thinking slow [35], is the best framework for understanding the nature of diagnosis and the cognitive aspects of diagnostic error [36] (Figure 6). In this framework, diagnosis starts with whether the clinician recognizes (Yes/No) a symptom or sign or a collection of findings.

1. System 1, the intuitive system: If the symptoms and findings are recognized, as they most often are, the diagnosis emerges within milliseconds using a subconscious automatic process referred to as System 1. This is an intuitive process that is often successful in reaching the correct answer but can go astray due to cognitive bias or other factors, leading to diagnostic error.

2. System 2, the rational system: If the findings are not recognized, the clinician needs to stop and think. This is System 2, the purposeful, deliberate, and hopefully rational process of reviewing what is known and consciously considering what the answer might be. System 2 is much slower than System 1. System 2 is considered a more reliable approach to finding the correct answer, but it is also occasionally wrong. In practice, diagnosis typically involves some mix of the intuitive and rational systems.

Figure 6:

System 1 vs. System 2 Cognition. Reproduced with permission from Vanderbyl [31], which was derived with permission from a Neurofied figure (https://neurofied.com) by Philip Jordanov.

The dual process paradigm describes not only diagnosis but how we process most information in our daily lives, where most things and situations are recognized and we know automatically how to respond or what to do. However, sometimes we encounter a novelty that requires conscious thought. Think about how you learned to ride a bike or play a musical instrument – these were stressful and difficult tasks early on, but with time became effortless and automatic. Similarly, first year students process most clinical problems using System 2, and as they acquire knowledge and familiarity, they gradually transition to the intuition and reflexivity of System 1 to handle things.

Many breakdowns in clinical reasoning reflect inappropriate shortcuts or assumptions, and many involve the subconscious tendencies we all have that detract from optimal cognition. The cognitive biases encountered in diagnosis and diagnostic error are the same ones found in everyday life and are simply part of our human nature. Over 175 biases are catalogued in Wikipedia [37]. A few of the biases that are most commonly encountered in cases involving diagnosis include these (see Supplemental Material, Appendix F for others):

Premature closure (also known as ‘search satisficing’).

This bias is our human tendency to be too quickly satisfied with the first diagnosis that comes to mind that explains most of the key findings in the case. Most everyone with a dog is familiar with this tendency – we just fall in love with the first puppy/dog and don’t go searching other litters or shelters. Herbert Simon received the Nobel prize for describing this concept in the field of economics. He called it ‘satisficing’ [38]. In diagnosis, satisficing is the opposite of optimizing, constructing a differential diagnosis of the likely possibilities.

Context errors. If you see a patient with a chief complaint of vomiting, you automatically start thinking of gastrointestinal (GI) causes. The diagnosis may well be a GI problem, but if you focus strictly on the GI context, you may not consider other ‘don’t miss’ causes, like poisoning, sepsis, or intracranial hypertension, among others. If you are looking in the wrong context, you will never make the correct diagnosis.

Anchoring is our tendency to be satisfied with a new or pre-established diagnosis without rethinking the case. In support of our initial belief, we tend to favor evidence consistent with it and discount evidence against it.

Affective bias reflects our subconscious tendency to favor certain people and disfavor others, whether because of their age, sex, socio-economic status, ethnicity, appearance, or behavior. Their medical conditions may be yet another factor that influences cognition. For example, there is good evidence that patients with mental health disorders or a history of drug abuse are treated differently in diagnostic settings [39, 40].

Will cognitive bias derail diagnosis? The situation, patient, clinician, and particulars of the case are all relevant factors in determining whether or not cognitive bias will be a factor in a given case [41]. In RCAs of diagnostic error, it may be important to review each of these dimensions (Figure 7, modified with permission from Dror [41]).

Figure 7:

Factors influencing whether a cognitive bias will be encountered.

There is considerable variability in clinical decision making, and the likelihood that a given clinician will be adversely influenced by bias [42]. Each clinician has had different exposures and experiences during their education and training, and as a result, diagnosis in practice is idiosyncratic. Two clinicians presented with the identical patient story and set of medical facts may come up with very different impressions of what the diagnosis might be.

Gender differences provide an interesting example of this variability [42]. Female physicians, for example, tend to be more effective in encouraging patient engagement and questioning than male physicians, and are more likely to explore psychosocial issues. They are less comfortable with uncertainty, order more tests and consults and are more compliant with clinical guidelines. They also exhibit less implicit racial bias than their male counterparts.

A significant body of evidence has now made it clear that cognitive biases manifest themselves automatically and unconsciously over a wide range of human decision making. Besides their psychology and sociology origins, they are now acknowledged in business, marketing, the judicial system and many other domains. Events on the world stage are influenced by them. It is important for everyone to recognise just how pervasive biases are and the need to mitigate them.

Pat Croskerry: Our better angels and black boxes [43].

Step 5: Crafting interventions

Given that any type of diagnostic error is likely to recur, and perhaps repeatedly in both the same organization and more broadly, the goal of each RCA is to consider interventions that will minimize this possibility. The RCA review is incomplete if it does not include at least one high-priority recommendation.

…(An RCA investigation is) meaningless to patients if it did not lead to action and change……A properly crafted process or outcome measure should be specific, quantifiable, and provide a timeline on when it is going to be assessed.

Charles et al. How to perform a root cause analysis for workup and future prevention of medical errors: a review. [44]

Specific errors point to specific solutions, but the most important interventions for improving diagnosis will center around leadership and culture. If leadership strongly endorses the goal of achieving diagnostic excellence, if champions for diagnostic safety are visible amongst their colleagues, if teamwork is the norm, and if errors and solutions are openly discussed, the prognosis for improving diagnostic outcomes is bright. Until these foundational elements are established, they should be included as recommendations in every RCA.

System-related factors are identified in most cases of diagnostic error. However, because organizations have extensive experience reviewing the problems and interventions in this domain, we will not consider them in detail except to point out the important progress made in ‘catching’ diagnosis-related safety breakdowns before they lead to patient harm. The Kaiser-Permanente – Southern California health care organization has pioneered the use of ‘safety net’ systems to catch potential delays in diagnosis that could lead to harm. Examples include electronic monitoring to ensure that patients with a positive test for fecal occult blood receive endoscopic evaluation, and patients with escalating PSA values are evaluated in Urology. The safety net concept is well established in the United Kingdom as a primary care intervention and is effective in reducing delays in cancer diagnosis.

Another area of active research concerns interventions to tackle lapses in follow-up care. These represent ‘low-hanging fruit’ in efforts to improve the reliability of diagnosis, including failures to follow-up on incidental findings, abnormal screening tests, alertable test results, tests pending at discharge, and patients with concerning but non-specific symptoms.

Cognitive errors. Addressing cognitive errors is likely to be a new challenge for healthcare organizations. “Hardwired” solutions, such as forcing functions, top the list of possible interventions whereas education and training are viewed as weaker choices [45], [46], [47]. It is worthwhile noting that these ‘strength’ hierarchies are passed down as wisdom from the sages more than evidence-based conclusions. Education and training interventions may actually have greater impact on cognitive reasoning skills and may be perfectly reasonable solutions in certain cases for the following reasons:

1. There is no course on diagnosis in medical education today. Doctors and the many other clinicians involved in the diagnostic process have never received formal training on clinical reasoning or on critical thinking in general. Most have only passing familiarity with decision support resources.

2. Clinicians are not generally aware of the many ways that human factor elements can impact diagnosis, for better or for worse.

3. Clinicians generally have never learned about heuristics and biases, and that human decision-making is beset by universal subconscious tendencies that can detract from best judgment. Inappropriately, many or most view themselves as unbiased and immune from affective influence.

4. Finally, clinicians tend to think of themselves as excellent decision-makers. Overconfidence is the rule, and though the concept of calibration may be appreciated at a subconscious level, it is not something clinicians think about as a critical determinant of their skill in diagnosis.

All of these issues may potentially be addressed productively through education. Simulation training is more likely to engender retained knowledge and skills than book learning, and many of the interventions proposed to address cognitive issues are ripe for simulation-based training. Several authorities make the point that education that is content and even case-specific is likely to be a more effective intervention than general education on clinical reasoning and bias. Examples of focused education include practice on differentiating diseases with similar presentations, and practice expanding a differential diagnosis list.

For cases where clinical reasoning is a key issue, Croskerry divides cognitive interventions into those focusing specifically on steps the individual clinician can take to avoid error, and those that use system-based approaches. Table 5 presents options for improving diagnosis. The relative impact of these suggestions has not yet been evaluated.

Table 5:

Interventions to improve diagnosis.

Interventions focused on the individual. These are the ‘cognitive pills for cognitive ills’ that primarily involve metacognitive skills, meaning, being able to consider the actions and decisions one is making and reflect on whether these can be improved. “Stop and Think” captures the essence of these interventions. If clinicians could routinely adopt this approach, it would provide them the opportunity to employ effective forcing functions that would improve diagnostic decision-making [48, 49]. Asking, “what else could this be?”; and “what emotions could be affecting my judgment in this case?” are worthwhile questions to ask in every case involving diagnosis.

After metacognition, the most consistent advice to address cognitive error is to promote awareness of the cognitive and affective biases leading to diagnostic error. A cardinal principle of cognitive psychology is that many cognitive biases are indeed hardwired and cannot be unlearned [35], others are acquired [50]. The key in either case is to know that biases can be mitigated by learning to recognize them in one’s own thinking, or in the diagnostic decisions made by others. This, in turn, provides the opportunity to reconsider decisions before there is harm [51, 52]. Clinicians involved in diagnosis should know about the dual-processing framework and how biases can detract from optimal decision-making.

In cases where affective bias played a role, implicit bias training may be helpful for clinician education [53], along with organizational efforts to promote equity in access to care and services [39].

There is early evidence that cognitive interventions may be effective in addressing diagnostic error [54], [55], [56], but organizations should keep abreast of new research likely to emerge in this area in the future. It seems likely that interventions that are more specific and case-focused may have more impact over those that are more general, like “Stop and Think”.

Interventions focused on the system. Diagnosis is especially dependent on the context of care. When developing interventions to address diagnostic error, interventions should target these contextual connections. Diagnostic decision-making is inherently error-prone, and interventions that either help the clinician make these decisions, or involve others in this process, will likely be beneficial. Below are several recommendations:

Improve teamwork. This was the top recommendation in the National Academy of Medicine report on Improving Diagnosis in Health Care [15]. If group-think problems can be avoided, team members can provide fresh perspectives on a case and help catch cognitive errors. Involving the patient, family, and nurse colleagues are positive steps that can improve diagnosis organization-wide.

Get second opinions and consults. There is strong evidence that second opinions improve diagnosis in pathology and radiology. Second opinions result in important changes in a diagnosis for between 2 and 5 % of cases [57], [58], [59]. It is highly likely that even greater benefit will be seen in frontline diagnostic settings. A study of second opinions in the ED found that consulting a colleague about active cases reduced diagnostic errors by one-third [60]. Improving access to expert consultants is another avenue likely to improve diagnosis.

Group-based (collective) diagnosis is an emerging area with substantial potential to reduce the likelihood of error [61, 62]. Crowd-based decisions may be much more accurate than the ‘stop and think’ approach of reconsidering a case on your own.

Provide decision support. Checklists, mnemonics, and various other decision aides are available to help with diagnosis but are underutilized [63]. These can be helpful tools for metacognition by helping the clinician think of conditions and organ systems they had not considered. More sophisticated, web-based tools to aid in differential diagnosis have been available for some time, and have demonstrated value [64]. It is likely that emerging AI-based systems will be even better, and integrations in which suggestions are ‘pushed’ to clinicians instead of them having to search for potential choices will be especially helpful.

Make it easier: Reduce any error-promoting factors; improve access to knowledge sources. Many clinicians believe they would do a better job with diagnosis if they just had adequate time to think. Production pressures and distractions should be minimized, and it is worthwhile offloading time-consuming tasks that detract clinicians from patient care (e.g. negotiating with insurance companies on the patient’s behalf, many prescription renewals).

Provide feedback [65] to improve calibration. The best diagnosticians are those with profound expertise and experience in a given area, and those with modest expertise who are well calibrated, meaning they have a good sense of what they know, what they don’t know, and when they need to slow down and seek help. Providing feedback to clinicians is an effective way to improve calibration, and organizations should consider ways for clinicians to learn whether their initial diagnostic impressions are correct or incorrect [66, 67].