Challenges in mitigating context specificity in clinical reasoning: a report and reflection

Introduction

Clinical reasoning is fundamental to every physician in practice [1]. Clinical reasoning involves a number of activities such as information gathering, formulating a differential diagnosis, providing diagnostic justification and making diagnostic and therapeutic plans [2], [3], [4]. Mistakes in clinical reasoning undoubtedly contribute significantly to diagnostic errors, which are hypothesized to account for approximately 15% of the errors in primary care [5]. There is mounting evidence that these errors cannot solely be attributed to gaps in physician knowledge or training [6], [7]. Indeed, recent research has identified the phenomenon of context specificity, defined as a physician seeing two identical patient presentations from a content perspective (e.g., identical histories, physical exams, labs and the same diagnosis) yet arriving at two different diagnostic decisions [6], [8]. In other words, in these situations, something other than case content is driving the physician’s decisions leading to unwanted variation in physician performance.

The primary theoretical framework for this article is situated cognition. From this theoretical perspective clinical reasoning is situated (or located) within the specifics of the encounter (e.g., the patient, the physician and the environment) and clinical reasoning processes and outcomes dynamically emerge from and are shaped by these specifics of the situation [9]. In situated cognition there are a host of interactions between individuals and their physical, social, and cultural systems. From this perspective, context specificity is associated with contextual factors, elements arising from patients, physicians, clinical environment and the interactions among all three [6]. When these contextual factors are distracting (e.g., a patient suggesting an incorrect diagnosis, an electronic health record not functioning optimally, compressed time for completing an encounter), this can negatively affect clinical reasoning performance and ultimately lead to diagnostic error [6], [8].

Our recent work investigating context specificity within a situated cognition framework has shown that contextual factors, particularly diagnostic suggestion, can have a negative effect on clinical reasoning performance and can significantly shift the process of clinical reasoning [10], [11], changing, for instance, what participants focus on in think-aloud reflections. That work, however, did not explore, within a situated cognition theoretical framework, strategies that could potentially mitigate the negative impacts of contextual factors on the clinical reasoning process [6], [8]. In this study, we draw from this work to test the mitigating effects of several strategies.

One proposed strategy for reducing diagnostic error is the use of reflective practice [12], [13], [14]. In reflective practice, there are a number of behaviors and reasoning processes that occur in response to complex clinical problems. These include (1) a search for alternative explanations of the problem, (2) exploration of the consequences of such alternative explanations that leads to predictions to be tested by the acquisition of new data, (3) a testing of these predictions against data and a reframing of the problem, and (4) a critical review of one’s own assumptions and conclusions about the problem (meta-reasoning) [12]. Part of reflective practice for the reduction of error can also be related to metacognition, stepping back from the immediate situation to reflect on the thinking process [15]. However, these educational strategies have not yet been employed in medical education settings to determine if they can mitigate the negative effects of context specificity in an authentic (e.g., simulated) environment. Such strategies may enhance physicians’ ability to recognize and possibly reduce the adverse effects of distracting contextual factors on clinical reasoning performance and reduce diagnostic errors. The development and exploration of effective educational strategies that address specific aspects of the clinical context are important for a future implementation of new instructional tools in the training of medical students, residents and physicians.

An optimal way to test out these strategies is through a simulated scenario, which creates a complex, highly interactive context. Simulated environments can help us disentangle the potential effects of distracting contextual factors on clinical reasoning and error and assess the outcome of potential strategies designed to ameliorate context specificity [16], [17]. Scenario-based simulations provide an environment that is similar to an authentic clinical setting, allowing for different degrees of contextual and cognitive complexity in the clinical encounter while controlling for specific leading and differential diagnoses [17], [18], [19]. Further, there is evidence that scenario-based simulations provide effective environments for assessing clinicians’ performance in activities associated with clinical reasoning [18], [20].

According to situated cognition, thinking and reasoning are conceived as occurring within the specifics of the situation. Through this lens, the development of an environment of increasing complexity, where the various interactions between patient, physician and context can be captured, is essential to better understand the intricacies of context specificity in clinical reasoning and the potential effect of mitigating strategies. The purpose of this study was to assess clinical reasoning performance during a simulated encounter, comparing physicians who participated in a computer-based tutorial on contextual factors and their potential link to diagnostic errors and a think-aloud exercise, with those who did not. Our hypothesis was that the use of these dual strategies – tutorial and think-aloud – would enhance clinical reasoning performance measured by a post-encounter form (PEF), leading to better performance by those participants when compared to the control group.

Materials and methods

This comparative experimental study examined whether strategies (consisting of an interactive computer training and a think-aloud reflection) designed to support physicians’ clinical reasoning in the presence of distracting contextual factors improved clinical reasoning performance, as measured by a PEF, an open-ended series of diagnostic and management questions for which we have previously gathered validity evidence [6], [21], [22].

Study design

The study included two groups of participants. The control group began with the simulated encounter, followed by the PEF (the outcome measure for this study) which closely mirrors typical practice [21]. The experimental group participated in an interactive computer-based training module, completed a simulated encounter, did the think-aloud reflection, and then completed the PEF (Control participants also completed the think-aloud exercise, but only after completion of the outcome measure; see Figure 1).

Figure 1:

Study design.

Results

Participants in the control and experimental groups were equally distributed in terms of gender (10 females out of 19 for control; 10 out of 20 for experimental), but in the control group the range of ages was wider and they had more years of experience (i.e., control participants were older and had been practicing longer; see Table 2 for demographic details).

Means for the four outcome variables ranged from 65 to 80% for control group participants and 70 to 87% for experimental group participants, with higher mean scores on all four variables for the experimental group. See Table 3 for means and ranges of outcome variables by group.

MANCOVA results revealed no significant differences between experimental and control groups (Pillai’s Trace=0.20, F=1.9, df=[4, 29], p=0.15, ƞ_p²=0.2; see Table 4 for details). Box's M test did not indicate a significant violation of the assumption of homogeneity of variances (F=1.8, p=0.06). Levene’s test, however, only indicated equal variances for supporting evidence; the other three variables violated the assumption.

When the analysis was run without the family medicine and surgery physicians, means remained higher for the experimental group and MANCOVA results again revealed no statistically significant differences (Pillai’s Trace=0.31, F=2.5, df=[4, 22], p=0.07, ƞ_p²=0.17).

Discussion

We developed strategies to attempt to mitigate the negative effects of context specificity on clinical reasoning performance that included a computer-based training module to raise awareness of these effects and a think-aloud exercise on participation in a simulated clinical encounter. We hypothesized that this computer-based training and think-aloud exercise would lessen the negative impact of distracting contextual factors on diagnostic and therapeutic accuracy of participants in the experimental group compared to the control group. However, we found no statistically significant difference between the two groups. Nonetheless, we believe this work is an important first step exploring how we might mitigate the effects of context specificity on clinical reasoning performance and patient care and should represent the first in a series of studies. [27], [28] We discuss below the importance of – and challenges with – continuing research on context specificity using a situated cognition framework.

Situated cognition anticipates the potentially negative effects of distracting contextual factors, since an individual’s clinical reasoning is interconnected with the social and environmental elements of a patient encounter [9], [29]. Yet the environment created by distracting contextual factors also creates opportunities; Ng and colleagues refer to “indeterminate zones of practice – uncertain, unstable, unique or value-conflicted practice situations” that provide opportunities for the development of clinical practice (p. 463) [30]. Contextual factors may move clinicians into these zones. Explicit education around and reflective practice upon distracting contextual factors in these zones may help clinicians develop better situation awareness to trigger reflection in future encounters with contextual factors [31]. Explicit discussion of biases and “diagnostic timeouts” to reflect on these biases have been suggested more broadly as error reducing techniques [32] and reflection strategies (e.g., using a think aloud) could help build on those insights.

Additionally, strategies like the one employed in this study could offer a somewhat different approach to reflective practice from other scholars [12], [13], [14], [28], asking participants to “think aloud” without explanation, judgment, or structure [24], [26]. While this approach may sacrifice some of the benefits of specific reasoning instructions [28], it may offer a more comfortable space for reflective practice than other, more directed approaches. Boud and Walker point to the importance of the “micro-contexts” instructors create for reflection (defined slightly differently by these authors, but still relevant to our context), particularly in terms of instructors’ potential power over the learner [33]. By removing the instructor from the room altogether and with minimal interruption (beyond cuing learners to continue when they fall silent), think-aloud exercises offer an opportunity to reflect across a full encounter that (in concert with other reflective practices) “permit[s] the making of meaning” (p. 10) [33]. Even if this act of meaning-making does not change the decision made in a particular case, it may serve to “promote adaptive expertise and practical wisdom” (p. 1,048) as physicians develop over the course of a career [34].

There were several limitations to this study. First the sample size did not give us enough power to adequately determine group differences. The experimental group was younger (in fact, the age difference between groups was statistically significant) and less experienced than the control group, which may have dampened the impact of the strategies as experience would be expected to improve performance [35]. Future studies should include a greater number of participants with similar or at least equivalent baseline characteristics. Second our approach was comprised of two parts, the computer-based training and the think-aloud exercise, separated by the encounter; the sequence of these two sets of strategies may have affected the overall effects while hampering our ability to disentangle individual effects. Moreover, the experimental group had additional time to review the case (i.e., the think aloud), which could also explain some of the trends. Future studies with access to more participants might explore these additional conditions. Finally, while our PEF measure has a considerable amount of validity evidence [6], [21], [22], [23], it is one tool and as such provides a limited view on clinical reasoning performance.

In conclusion, this study highlights the difficulty in mitigating and measuring context specificity. Yet, guided by situated cognition, the strategies we employ should be explored in future research, as they required that physicians develop increased situation awareness that moves beyond the self’s actions to the broader range of interactions and systemic influences that comprise a clinical encounter. Future work could explore what cues trigger this kind of awareness in physicians and how sensitivity to those cues shifts over time, helping us to more fully understand not only context specificity, but also the adaptive expertise that characterizes truly excellent physicians.