Focused ethnography: a new tool to study diagnostic errors?

Ethnography, defined as a “process of learning about people by learning from them”, is a research methodology that provides a unique view of participant behavior within a local context [1]. In ethnography, researchers embed themselves (over an extended period of time) into the social world of participants so as to better understand behaviors, organizations, communities, (sub)cultures and society. Ethnography primarily uses three inter-related sources of information: participant observation, formal and informal interviews, and examination of relevant documents or artifacts. By examining these three unique data streams, researchers gain a deep understanding of not only processes and structure, but also the underlying culture of the people being studied [2]. By definition, then, ethnography seeks to develop a broad view of a society from the people within it so as to permit a holistic understanding of various processes.

In contrast, focused ethnography is aimed at eliciting and evaluating information on a specific topic [3]. The methodology is particularly valuable for studying healthcare processes and medicine because it can examine specific questions, situations or problems among a smaller group of individuals. Focused ethnography differs from the ethnographic tradition in several ways [4]. First, because participants are purposefully sampled, a richer understanding of complexities from participants’ perspectives (“emic view”) can be developed and paired with an external framework (“etic”). Second, because the goal of the process is to examine an explicit question, focused ethnography targets a smaller number of participants rather than a population at large. Third, the approach encourages episodic, short-term observations (field visits or rapid immersions) that include multiple sources of data (e.g. observations, audio, video, etc.). Fourth, unlike ethnography which can vary in the scope and degree of data generated, focused ethnographies generate a large amount of topic-specific data from multiple sources (e.g. field notes, interviews, artifacts). Finally, focused ethnography emphasizes collective data analysis where data are not only collected by multiple team members, but also analyzed and interpreted as a group. Thus, the evaluation process embeds the perspectives of the individuals who conducted the observations, making the findings easier to place into an external framework or conceptual model.

For all of these reasons, focused ethnography represents an under-utilized but potentially powerful tool to study diagnostic errors. As defined by the National Academies, diagnostic errors are, “the failure to (a) establish an accurate and timely explanation of the patient’s health problem(s) or (b) communicate that explanation to the patient” [5]. The definition frames errors from a patient-centered view, emphasizing the fact that patients are key stakeholder in outcomes. Because such errors have myriad causes ranging from provider-, system-, cognitive-, process- and measurement sources, a key focus of prevention efforts is the system in which the work is performed. Within such systems, targets such as teamwork, collaboration and provider education, a culture that supports diagnostic excellence, and novel approaches to identify and learn from errors have been identified [6]. But how best to define the current state so as to identify gaps remains unclear. We believe focused ethnographies are an excellent tool for developing a rich understanding of the current state of diagnostic errors.

In recent work, we used focused ethnography to examine the diagnostic process among trainees in academic hospitals [7]. We used teams of observers that included clinicians, industrial and operational engineers, qualitative methodologists, and non-clinical researchers to shadow medical teams as they provided care for ward patients. We purposefully crafted a multi-disciplinary team as it allowed us to capture not only the clinical but also the social and technical aspects of diagnosis. Following best practices for focused ethnography, we observed team rounds documenting team structure and behavior, the types of formal and informal communication that occurred during the day, how diagnoses were considered, management refined during rounds, and the environment in which all of the work occurred. Each of us took detailed notes on what we saw and what was said, using a template to collect data systematically. We observed teams on both admitting and non-admitting days, as they rounded and did work outside of rounds, and when and why they reached out to consultants for help. At the end of our observations, we interviewed team members to narrate what we saw, gather their thoughts and opinions on what we observed, and ensure our findings were consistent with what was being done.

Our results, published earlier [7], encompassed close to 170 h of observations and hundreds of pages of field notes. Even though we work in the same system, looking as outsiders led to surprising findings. First, we were struck by how often diagnoses were formed within a social interaction. Traditionally, we have always viewed the diagnostic process as occurring “in someone’s head”. Indeed, many of the current interventions aimed at preventing errors are centered on cognitive heuristics and biases or system interventions that aim to prevent these issues – with mixed success [8]. We found that the culture among trainees was oriented toward group- rather than individual-think. Approaches to testing, treatments and communication of findings were socialized in the team, pressure tested, and then processed at large with the attending in presence. Supportively, we saw first-hand how teams with well-honed social interactions (flat leadership structures where everyone’s voice seemed to matter) did a much better job at these types of discussions than others. We were also surprised by the fragmentation of data sources needed to inform diagnoses. While the electronic medical record was often the go-to source for information, navigating the record to find specific elements was challenging – even for experienced users of this platform. When teams needed key information for diagnosis, information retrieval was sometimes a contorted process that either necessitated asking for help from a more experienced user, calls to laboratories to clarify findings, or discussions with consultants and experts who had access to relevant data [9]. We thus also observed how information was also fragmented over social networks – with some members of the team having access to key pieces of information. To make matters worse, the work was far from linear; distractions from pages related to other patients, interruptions from other care teams, or non-availability of patients at the time of rounds posed obstacles to diagnosis and patient care. Finally, we noted how time pressures influenced this process in very tangible ways. The goal of finishing rounds by specific times to facilitate patient work, attend multi-disciplinary discharge rounds or participate in educational activities was a palpable tension that clashed with the time needed for the diagnostic process. In the words of a resident whom we interviewed at the end of our observations, “if you wanted to design a system to fail – this is how one would design it.”

None of these insights would have been possible without the use of focused ethnography. In fact, our observations and findings so influenced our thinking about potential interventions to improve errors – ones that were provider and system centric – that we created our own model to improve diagnostic safety in training hospitals (Figure 1). We also understood that a binary view of classifying errors as those that are system-related vs. those that are cognitive was overly simplistic; rather, the overlap between the two is perhaps where interventions should focus [10]. While the effort and output were worth it, we learned important lessons as enumerated in the following:

Figure 1:

The Michigan model of diagnosis – informed via focused ethnography.

1. First, to perform good focused ethnographies – you need a great team. In particular, the availability and participation of a skilled qualitative methodologist who has performed this type of work or is familiar with the method is invaluable. Also, including observers from different backgrounds such as nursing, engineering or pharmacy can add depth to observations – as each of these perspectives are unique. Be creative about who you bring on to the team to gain the greatest insights.

2. Second, ensure you observe at short bursts at various times of the day. Being able to view how diagnosis evolves over time and how the team interacts around new and existing patients was invaluable to our understanding of the diagnostic process. This being said, one weakness in our approach was that we did not observe nights or weekends; we simply did not have the resources. While we could observe and contextualize the rationale and impact of decisions made the following day, having the flexibility to observe cross-cover and night shifts would have added important depth to our findings.

3. Third, consider engaging patients in observations. While we did not do this for our study (we were unsure of how teams and patients would react to our presence in rooms), in hindsight we wish we had. Hearing from patients during the actual observation process would have shed new light and understanding on communication gaps and potential sources of errors.

4. Fourth, the importance of formal interviews following observations cannot be discounted. The interviews we conducted with residents and with attendings let us very clearly link what we observed to the cognitive processes that were occurring in parallel. Interviews were also an important source of hearing explanations for aspects observers did not fully understand or appreciate.

5. Finally, keep meticulous records. Date and log all observations so that when the intricate story of what happened when, where and how needs to be reconstructed – a coherent story can be constructed. Weekly meetings with the team performing observations and interviews to provide ongoing reflections, comparing and contrasting what was seen against prior observations are invaluable in ensuring a cogent assessment.

Diagnostic errors are among the greatest dangers in healthcare today. If we are to truly improve the status quo, we must be willing to embrace new tools to tackle this old problem. Focused ethnography offers a novel approach. Why not give it a try?