Perceptual errors in pediatric radiology

Introduction

Perceptual errors account for 48%–80% of all diagnostic radiology errors reported in the literature [1], [2]. While the underlying reasons for perceptual errors are not well understood, there is considerable evidence that argues strongly against the concept that visual errors are the result of carelessness, sloppiness or negligence on the part of radiologists [3].

In this review, we will discuss factors that importantly contribute to perceptual errors in radiology, and show examples of these errors encountered in a large academic pediatric radiology practice. This material is based on more than 477,000 clinical cases reviewed in our department as part of an ongoing quality assurance process in place since 2003. During that time, we identified a total of 11,346 discrepancies occurring between a radiology report and a retrospective review of the imaging study and patient outcomes (2.4% of reviewed cases). A detailed description of our case identification and categorization process has been published elsewhere [2]. For purposes of this review, we have defined perceptual errors as those primarily related to non-recognition of an imaging abnormality.

Medical image perception in a nutshell

Interpretation of medical images is a complex phenomenon that has been described as involving two interrelated processes of perception, where there is a unified awareness of the content of the image, and analysis, during which one determines the meaning of the perceived findings in the context of the medical problem at hand [4], [5]. Our understanding of how radiologists interpret images evolved from studies on the psychology of visual perception. Initially, the process was understood to be a sequence of “search and detect, recognize, and decide” [6]. More recent studies have turned this original model around, and suggest that an initial global impression plays a very important role in initiating image perception and directing focal search [7]. The process is now modeled as “detect and recognize, search, and decide”, relegating the search process to a later stage in perception [6]. After the most conspicuous lesions have been identified, a focal search-and-identify phase takes over [7].

In reality, the task of clinical radiology is much more complex. To see the diagnostic process in radiology as a dichotomous variable (target abnormality detected/target abnormality not detected) is to oversimplify the process. More than one abnormality might be present; each of the detected abnormalities may in turn represent an actionable item (e.g. cancer), or not (e.g. old inflammatory granuloma). In addition, the target abnormality is commonly embedded within complex or distorted anatomy, and there may be added distractors in the image, such as overlying medical devices, or digital annotations that increase the “noise” within which the target must be discerned [8], [9].

Cross-sectional imaging modalities, such as CT or MRI improve the visual conspicuity of target abnormalities by obtaining multiple thin slices through the body, thus removing a great deal of anatomic overlap. However, these modalities come at the cost of producing hundreds to thousands of images that the radiologist must review in order to render a complete examination of the anatomy in question.

A dual-process of decision-making is theorized to be active during radiologic interpretation. System I is quick, heuristic, highly automatic, and prototypical. It requires minimal effort, and is highly prone to bias and error. However, it is very efficient and frequently utilized. System II is slower, more analytical, deductive and rule-based. It requires effort, and is less prone to bias and error. The operating principle of the dual process theory is pattern recognition. Most diagnoses in radiology can be made rapidly and effortlessly (e.g. displaced femur fracture, or large Wilm’s tumor). When an immediate diagnosis cannot be made, then the radiologist switches to a more analytical, deliberate and deductive approach (e.g. search for pulmonary nodule). While radiologists can switch between these two modes, there is a tendency to revert to the state requiring the least cognitive effort [10], [11].

Classification of perceptual errors

Although some diagnoses may be missed because of the technical or physical limitations of the imaging modality, including image resolution, intrinsic or extrinsic contrast, and signal-to-noise ratio, most missed radiologic diagnoses are attributable to image interpretation errors by radiologists [3]. Using advanced eye-tracking techniques, Kundel et al. studied visual search patterns in radiologists and classified visual misses (false negative errors) into three categories based on visual dwell times [5], [7]. The first is the search error, in which the radiologist never fixates on the lesion, and does not appear to see it. The second is an error in recognition, where the radiologist fixates on a lesion, but does so for a period of time less than the threshold required for recognition of the finding as abnormal (0.48 s). The third and final type of error is a decision error where the radiologist fixates on the lesion for longer than the threshold time for recognition, but fails to recognize it as abnormal, or actively dismisses the finding.

Contributing factors to perceptual error

There are several inter-related factors that contribute to an incomplete evaluation of relevant image content in clinical practice, resulting in a missed or delayed diagnosis. Broadly, they are related to the radiologist’s search patterns, the radiologist’s pre-existing expectations, and human factors such as increasing clinical workload, fatigue, interruptions and distractions [3], [9], [12]. The most well-known is the phenomenon of satisfaction of search (SOS). It describes a situation during which the detection of an abnormality satisfies the search for meaning and results in premature termination of further search [13], [14]. In a seminal study in 1990, Berbaum et al. [15] showed that the addition of simulated pulmonary nodules in chest radiographs led to a significant reduction in the detection of native pulmonary nodules. SOS effects also lead to lower detection rates when multiple findings are present in clinical cases. Ashman et al. [16] showed that the detection rate for the first finding in bone radiography cases that contained multiple findings was about 78%, but detection of second and third findings decreased to approximately 40% (Figure 1). Subsequent experimental studies have shown that SOS effects appear to be the result of either a lack of visual inspection of other areas of the image, or a failure to recognize abnormalities where the image has been inspected [14], [17].

Figure 1:

Satisfaction of search error.

A 7-year-old male with elbow pain and limited range of motion after fall. Two of the four X-rays obtained are shown. Subtle, nondisplaced fractures of the radial head and olecranon were identified on oblique (A) and lateral (B) radiographs (arrows). An anterior dislocation of the radial head (arrowhead) was missed.

Another form of search satisfaction can be described as “visual isolation”, during which a radiologist’s search pattern is truncated to the main areas of an image, while little or no attention is given to peripheral areas [18]. A clinical example is shown in Figure 2. This chest radiograph was obtained to confirm feeding tube placement in a premature infant with chronic lung disease. Most of the radiologist’s attention is devoted to the location of the tube within the chest and abdomen, and an evaluation of pulmonary inflation and parenchymal opacities that might suggest the presence of atelectasis, pulmonary edema or infection. The evaluation of bony structures and the abdomen are relegated to a lower level of inspection, or ignored. A new left humeral fracture is visible in the periphery of the image, but was not identified on initial inspection.

Figure 2:

Visual isolation error.

A 2-week-old, former 24-week gestation infant with chronic lung disease. The X-ray was obtained to confirm location of a feeding tube within the stomach. Frontal view of the chest and abdomen (A) shows the feeding tube in adequate position. A healing left humerus fracture (arrows, B) was missed.

A second contributing factor to perceptual error is the phenomenon of inattentional blindness. The term was introduced by Mack and Rock in the 1990s, and is defined by the failure to notice a fully visible, but unexpected object because attention was engaged on another task, event, or object [19]. In one study, participants were asked to count basketball passes by players wearing white shirts and to ignore passes made by players wearing black. Approximately 50% of subjects watching the video did not notice a person in a gorilla suit entering the scene, stopping to face the camera and thump its chest before exiting on the other side [20]. Thus, if a finding is not expected on an imaging study, there is a strong likelihood that it will be missed (“I will see it when I believe it”). This phenomenon was shown in a study of 24 chest radiologists asked to identify pulmonary nodules in a stack of chest CT images. A gorilla, 48 times larger than the average nodule, was inserted in the last case. More than 80% of radiologists failed to see the gorilla. Of note is that failure to see the gorilla was not due to a truncated visual search. In fact, visual tracking of the radiologists showed that the majority of those who missed the gorilla had actually looked directly at the location of the gorilla [21] (Figure 3).

Figure 3:

Inattentional blindness error.

Voiding cystourethrogram in a 9-year-old male with treated posterior urethral valves. Three images from the study are shown. Oblique view of the bladder during voiding (A) shows a dilated posterior urethra (asterisk), and reflux into a dilated utricle (arrow). Right oblique view (B) of the bladder shows a normal femoral head (arrow heads). Left oblique view (C) shows sclerosis and fragmentation changes of the left femoral head due to avascular necrosis (arrows). This unexpected, but radiographically obvious finding was initially missed.

Failure to identify the absence of an expected image feature is another manifestation of inattentional blindness. In one study, radiologists were shown chest X-rays with a clavicle removed. When no cues were given, approximately 60% of radiologists failed to identify the abnormality. When historical cues were added (e.g. chest X-ray for metastatic survey) the detection rate increased to 83% [22] (Figure 4).

Figure 4:

Inattentional blindness error.

Frontal radiograph of the entire spine (A) obtained for monitoring of scoliosis in a 15-year-old male with upper back pain. An unexpected, large left pneumothorax (B, arrows) was missed.

The effect of clinical history on radiological interpretation has been studied extensively, and has been shown to also have an important role on visual error [23], [24], [25]. In one study, the availability of pertinent clinical history increased the accuracy of chest radiographs from 16% to 72% for trainees, and from 38% to 84% for experienced radiologists [12]. Accurate clinical history appears to improve the radiologist’s accuracy by focusing perceptual attention to the identification of specific lesions. A distracting, or incorrect clinical history however, can have the opposite effect by misdirecting the radiologist’s search [26].

Finally, it is thought that an increased incidence of perceptual errors may be attributable to certain human risk factors such as increased pace of interpretation, fatigue, and work-place distractions such as phone calls and e-mails [3], [27], [28]. The average radiologist work-load has increased dramatically. Over the last decade, radiologists have been required to read more cases with more images per case resulting in an increasing pace of interpretation in order to “keep up” [29], [30]. In our own department, it is not unusual for a single magnetic resonance imaging (MRI) or computed tomography (CT) examination to contain more than 3000 images (Figure 5). As a result, it becomes impossible for the radiologist to devote an adequate amount of attention to each image. Even an excellent radiologist is not as keen at the end of the day as she is at the beginning of the work day. Eye strain and fatigue take their toll and lead to decreased visual accommodation at the end of the day compared to early morning resulting in more myopia as the day progresses, leading to a possible drop in diagnostic accuracy. Fatigue can also result in a decrease in the radiologist’s working memory, particularly after several hours of continuous and prolonged decision making. The increased amount of effort (cognitive load) required to make an accurate diagnosis when tired can result in the radiologist taking short-cuts and subsequent diagnostic errors [31], [32].

Figure 5:

Summary image from a chest CT obtained for evaluation of possible interstitial lung disease shows the number of images contained within the study (3184).

Strategies for error reduction

Some strategies are simple and self-evident, and can help to improve the conspicuity of lesions. These include the routine use of window and level tools, review of prior imaging studies, and availability and review of appropriate patient history [10], [18].

Complete elimination of error from radiology has proved impossible over many decades of effort. However, there are a number of corrective strategies that, when taken together, have the potential for serious mitigation of perceptual errors.

The first strategy attempts to correct for framing bias (interpretation strongly influenced by the way in which the clinical problem is presented) by an initial “masked” interpretation of the image, performed without knowledge of the clinical indication for the imaging study. This allows a “gestalt” evaluation of the entire image without selective focus. A second, more focused and detailed search is performed after a specific clinical concern is articulated. This approach is thought to effectively utilize both systems of decision-making [32].

The second strategy is the use of a structured report as an attempt to minimize SOS error. Structured reports act as a checklist to promote consistency in visual approach, and help avoid predictable biases. This approach can be helpful in considering common and serious “do-not-miss” diagnoses [32].

Self-prompting verbalization of the focus of attention has also been shown to be an effective strategy in reducing search satisfaction errors. In a study of SOS errors in chest X-rays, Berbaum et al. [33] showed improved detection of pulmonary nodules when the radiologist verbally described the focus of attention during reporting a case. They attributed this improvement primarily to fewer decision-making errors rather than faulty pattern recognition. This is especially useful in patients with marked physical deformity where anatomic structures may be displaced in unexpected ways, and poorly positioned endovascular catheters and tubes may be misinterpreted as being in the correct location [8].

Conclusions

The process of radiologic image interpretation is a very complex process that we are still attempting to fully understand. The interplay among cognitive, visual physiologic and psychologic processes makes it unlikely that any one strategy will be effective in eliminating errors. However, interventions exist and can help mitigate errors. In addition, continued mindfulness of individual biases and short-cuts, and an institutional culture of education and safety are key to mimimize interpretive errors in radiology.