Sapere aude in the diagnostic process

The challenge of research into context

Typically, the process of investigation of any phenomenon begins with observation and description of the dependent variable in its natural setting. Various models may then be generated to provide an overall understanding of the main factors that appear to be relevant and how they interact with each other. Several models of human decision making have emerged over the last 50 years, a period in which cognitive psychology has flourished. A leading contender emerged in the form of dual process theory (DPT), [13], [14], [15] (though an excellent earlier description of DPT can be found in Thomas Paine’s Age of Reason [1794]) [15]. Which now appears well-established in the medical literature [16], [17], [18], [19], [20]. Not only does it delineate the two main processes (intuitive and rational) by which decisions get made, but also allows some insight into how intuition is progressively accrued and refined by experience. Intuition is a broad term in decision making, and has different meanings for different people [21]. Thomas Paine recognised it as a valuable form of understanding [15]. Myers defines it as “our capacity for direct knowledge, for immediate insight without observation or reason.” [22] Stanovich, in his book, describes four sub-processes through which intuition might be established: (i) hard-wired, genetically transmitted, naturally selected origins, (ii) those associated with emotions, inherited or acquired, (iii) those that have been acquired through explicit learning and practice, and (iv) those acquired through implicit learning [23]. For the rational process, things appear a little more straightforward, although the term may have specialised meanings in different domains, and also within medicine itself [5]. Overall, despite these variations in use and definition, DPT has provided a useful platform for investigating various aspects of decision making.

Often, what typically follows model development is a strategy of reductionism in which independent variables are identified and their individual impact on the dependent variable assessed. This is usually done in an experimental setting in the laboratory where they are experimentally isolated in order to exclude their influence. However, there are several problems in applying this process to the study of diagnosis. As noted, the dependent variable (diagnosis) is influenced by multiple independent variables (the 40+ described above). Firstly, there are ethical challenges in studying the problem in its natural habitat where randomised, double-blind, prospective studies are nigh impossible. Secondly, studying diagnosis in the laboratory may be severely compromised by taking the process out of its natural setting. The problem was summarised very well by Gruppen and Frohna: “…too often, studies of clinical reasoning seem to take place in a vacuum. A case or scenario is presented to subjects, usually in written form, stripped of any ‘irrelevant’ noise. The traditional methodology of providing clinical cases that are decontextualized and ‘clean’ may not be a particularly valid means of assessing the full range of processes and behaviors present in clinical reasoning in natural settings.” [24] The main issue here is decontextualizing which may draw the ecological validity of the study into question. Similarly, Wears and Nemeth [25] proposed abandoning laboratory studies in favour of ‘real world’ studies that focus on context and on the intuitive processing that experts use. Such post hoc analyses should be done to reconstruct the clinician’s experience at the time of the event, perhaps by interviewing the clinician, or by promoting self-awareness and introspection. Thus, the common method of studying diagnosis using computer screens that display clinical vignettes may be no more than pale imitations of what actually happens in the clinical situation. Yet, a review of studies of cognitive biases and heuristics in medicine found that 77% of 213 studies reviewed were based on hypothetical vignettes [26]. While an advantage of this approach is that studies can be done with relative ease, and the appearance of experimental respectability may hold some appeal for editors and their reviewers, their context is almost non-existent and their ecological validity may be seriously questioned [27].

There are several ways in which we can study the diagnostic process without falling into this ecological trap [28]. One is by looking at diagnostic failure using standardised or simulated patients (SP). The SP presents to a clinic unannounced and is seen and assessed ‘blind’ by the clinician. In one study, the diagnostic failure rate using this methodology was estimated at 13%, [29] with the advantage that much of the medical context was preserved. A major problem with the procedure, however, is that the SP is trained to simulate prototypical manifestations of a particular disease, an example of representative bias [30]. Prototype and the representative bias are characteristic of an anticipated context. Inevitably, atypical presentations are not investigated using this method. This is an important constraint for some illnesses e.g. atypical presentations of acute myocardial infarction (without chest pain) occur about 1/3 of the time [31]. If experimenters programmed SPs for atypical presentations of illness, this methodology would achieve an even better level of ecological validity. Another method is through the use of in vivo studies in which real clinical cases are rigorously subjected to cognitive root cause analysis, with particular attention paid to prevailing context [11]. This requires that those conducting the analysis have a sound understanding of the multiple sources of error in clinical situations, and especially of cognitive failings.

Another issue for the diagnostic process that illustrates vulnerability to the ecological problem is Artificial Intelligence (AI). While AI can employ machine learning (ML) techniques for such complex problems as diagnosis by making predictions from very large amounts of patient data by learning ML’s own associations, [32] the data is typically inputted without context. Part of the problem is that while implicit awareness of context may often be factored into clinical decision making in real life it may not be so easily articulated for AI systems. A particular problem is ‘distribution shift’ where ML systems may fail to recognise that a significant change in context has occurred. Challen et al. define distribution shift as: ‘A mismatch between the data or environment the system is trained on and that used in operation, due to bias in the training set, change over time, or use of the system in a different population, resulting in an erroneous ‘out of sample’ prediction’ [33]. A pandemic is a case in point, where new data and environmental changes may appear overnight.

An abrupt change of context

A significant current example of the impact of context is the Covid-19 pandemic, which has abruptly changed the clinical practice of medicine worldwide. It has been the ultimate context-changer. Potentially, every patient seen may have the disease, with the decision makers themselves being no exception. The signs and symptoms of presentation do not provide a pathognomonic pattern that may be readily recognised, and consequently other diseases may easily be mimicked. We can expect that several common cognitive biases normally associated with the diagnostic process will be in force, principally availability, as well as others (anchoring, framing, ascertainment bias, unpacking failure, search satisficing, diagnosis momentum, groupthink, and confirmation bias). Communication, a key aspect of the diagnostic process, is significantly changed, through provider’s donning personal protective equipment, especially facial masks, thereby attenuating significant aspects of body language. Ambient factors in the workplace will alter context: increased workload, cognitive load, stress levels and other factors known to influence the calibration of decision making are all changed. Emotional issues for providers are changed, not the least being increased fear levels for themselves and their families should they contract the disease, but also the toll of compassion fatigue through recurrent episodes of dysphoria associated with witnessing death.

What does sapere aude ask of us?

We need to accept that the process of diagnosis, although often straightforward and not requiring exceptional effort, can be tricky and deceptive at times, and will require reason and intellect in the attainment of diagnostic accuracy. An important part of our intellect is an awareness of the prevalent biases that may impact our decision making, awareness of our failures to reason logically, a capacity for mindfulness, and a commitment to paying attention [34]. Often, to be mindful requires very little extra effort yet may achieve significant results. Being mindful of the impact of context, of the constant intrusion of bias, of the ambient conditions in which we work, and of our own homeostasis and well-being allows us to keep a check on our decision making. In particular, we need to be aware that what you see is all there is (WYSIATI) [35] may be a harbinger of premature closure. Overall, we need to understand the complexity of a process that we often take for granted, and be more skeptical when things appear obvious. The novelist Lawrence Durrell saw this as our greatest challenge: ‘The greatest delicacy of judgement, the greatest refinement of intention was to replace the brutish automatism with which most of us exist, stuck like prehistoric animals in the sludge of our non-awareness’ [36]. It would seem that dealing with our ‘brutish automatism’ and un-sticking ourselves from the ‘sludge of non-awareness’ cannot be accomplished without at least some degree of mindfulness. Importantly, this has recently received some emphasis in an AHRQ report into patient safety: ‘Although there are a limited number of studies, the literature suggests that training on metacognitive skills may improve diagnostic accuracy, particularly as clinical experience increases’ [37]. Finally, to re-visit Kant: ‘If it is now asked whether we at present live in an enlightened age, the answer is: No, but we do live in an age of enlightenment’ [1]. The greater detail with which we now understand the diagnostic process signifies not that medicine currently exists in a fully enlightened age (that may not be attainable for some time), but at least it is within an age of enlightenment, where we can steadily improve it.