Understanding evidence-based diagnosis

The evidence-based medicine “revolution”

Depending on who you ask, the term “evidence-based medicine (EBM)” was coined by either David Eddy of Duke University or Gordon Guyatt and colleagues of McMaster University around 1991. Guyatt et al. characterized EBM as a new scientific paradigm of the sort described in Thomas Kuhn’s 1962 book “The Structure of Scientific Revolutions” [1, 2]. One could question whether EBM, “which involves using the medical literature more effectively in guiding medical practice” [1], is profound enough to constitute a “paradigm shift” as described by Kuhn. I define EBM as using the best available evidence to help in two related areas:

• Evidence-Based Treatment: How to determine whether a treatment is beneficial in patients with a given disease, and if so, whether the benefits outweigh the costs and risks.

• Evidence-Based Diagnosis: How to evaluate a test and then use it to guide treatment decisions.

Both of these areas of EBM include structured approaches to the evidence and helpful quantitative techniques. For example, in evaluating a treatment we often want to calculate the number needed to treat (NNT) to prevent one bad outcome. In evaluating a test, we need to know how to use the test’s result to update a patient’s probability of disease. Evidence-based diagnosis has significant limitations, but understanding it can improve the diagnostic process and reduce diagnostic errors. However, before going further into evidence-based diagnosis, I need to acknowledge that evaluating and using medical tests to guide treatment decisions is not really “diagnosis” as we commonly understand it.

Name that disease

The Oxford English Dictionary (2nd edition 1989, accessed online) defines diagnosis as “identification of a disease by careful investigation of its symptoms and history.” While the OED should also have included the physical exam, laboratory evaluation, and other testing, including imaging, the point is that diagnosis involves assigning a name to a person’s illness, not making a decision about treatment or anything else. Just as we name a recognizably distinct animal, vegetable, or mineral, we name a recognizably distinct disease so we can talk about it and study it. Associated with a currently named disease might be a pathophysiologic mechanism, histopathologic findings, a causative micro-organism (if the disease is infectious), and one or more possible treatments. But more than two millennia before any of these were available, asthma, diabetes mellitus, gout, tuberculosis, leprosy, malaria and many other diseases were recognized as discrete, named entities. Although we now understand and treat diabetes and malaria better than the ancient Greeks, we still diagnose infantile colic, autism, and fibromyalgia without really knowing what they are. We have anything but a complete pathophysiologic understanding of schizophrenia, amyotrophic lateral sclerosis, and rheumatoid arthritis, all diseases for which treatment can only be supportive and symptomatic, never curative.

The cognitive process

Recent popular books, such as Lisa Sanders’s “Every Patient Tells a Story: Medical Mysteries and the Art of Diagnosis” [3] and Jerome Groopman’s “How Doctors Think” [4] have drawn attention to the cognitive process of diagnosis and potential errors in this process. Somehow Sherlock Holmes always comes up. His creator, Arthur Conan Doyle, was a doctor. Doyle based the Holmes character on a medical school professor named Joseph Bell, who was, presumably, a brilliant diagnostician. But this meant that he could name diseases, not necessarily treat them. The cognitive process of diagnosis involves integrating information from history, observation, exam, and testing using an ill-defined combination of knowledge, experience, pattern recognition, and intuition to name a patient’s illness. How can we train clinicians in this process? The answer probably involves education in anatomy, physiology, and pathology followed by exposure to as many cases as possible. Since actually interviewing, examining, and following up on the number of real patients required to develop diagnostic expertise requires more than a lifetime, diagnostic training should consist of exposure to already-solved cases in as realistic a format as possible. Computerized case presentations including photographs and video clips are very promising in this regard. How do we minimize the cognitive errors inherent to a task like diagnosis? The answer is even less clear, but computerized diagnostic support tools may have a role. Frankly, the tools and techniques of evidence-based diagnosis, as I define it, are only a small part of the solution.

Dealing with uncertainty

William Osler said, “Medicine is a science of uncertainty and an art of probability.” Osler had to make his diagnoses without the benefit of modern laboratory and imaging tests, but today’s clinicians still have to decide which, if any, tests to obtain and how to interpret the results. The tests are often imperfect, and we still make treatment decisions based on tentative diagnoses. Evidence-based diagnosis explicitly considers and quantifies the uncertainty about whether a particular disease is causing a patient’s illness, describes how to use test results to update this probability, and assuming that the costs and benefits of both the treatment and the test are known, determines whether the test is worth doing at all. As mentioned above, this does not correspond to what we commonly understand as diagnosis. Evidence-based diagnosis changes the question from “What is the name of this patient’s disease?” to three questions: “How likely is the patient to have a particular disease?” “How good is this test for the disease in question?” and “Is the test worth performing to guide treatment?” Advanced evidence-based diagnosis replaces the consideration of a single test with the consideration of multiple tests done either in series or in parallel. With this narrower approach, it does not make sense to test for a disease with no effective treatment nor to distinguish between two diseases with the same treatment.

Interpreting test results

Using a test result to update the probability of a disease is the most straightforward part of evidence-based diagnosis (Appendix). Using the best evidence available, you associate each possible test result with a “likelihood ratio.” Multiplying the pre-test odds of disease by the likelihood ratio gives the post-test odds. If the test result has a likelihood ratio greater than one, it makes the disease more likely (increases the odds); if it has a likelihood ratio less than one, it makes the disease less likely (decreases the odds); and if it has a likelihood ratio close to one, it does not change the likelihood of disease. Understanding this is particularly helpful for biomarkers such as B-type natriuretic peptide (BNP) for congestive heart failure, D-dimer for pulmonary embolism, and serum lactate for sepsis. Some of the values that the hospital lab will report as abnormal or even critical are associated with likelihood ratios less than one; they actually reduce the probability of the disease in question. Most of us are cognizant of the potential for falsely reassuring results such as a normal ECG in a patient with acute myocardial infarction, a normal white blood cell count in a patient with appendicitis, or a normal oxygen saturation in a patient with pulmonary embolism, but we may unknowingly over-react to a slightly elevated BNP, D-Dimer, or serum lactate. Understanding probability updating can also prevent over-reaction to an abnormal screening mammogram. While an abnormal mammogram increases the odds of breast cancer by a factor of 10, if the baseline chance of cancer is only 1 in 1000, then the abnormal test increases this to only 1 in 100. (When the risk of disease is very low, you can use odds and probability interchangeably.)

Deciding whether to test

Whether to test for a particular disease depends on 1) the pre-test probability of disease, 2) the accuracy and cost of the test, and 3) the relative costs of failing to treat someone with the disease versus unnecessarily treating someone without the disease. Since it is about testing for diseases already under consideration, evidence-based diagnosis is not particularly helpful in determining which diseases to consider in the first place, i.e., generating a differential diagnosis. Also, the patient’s individual characteristics and presentation may make estimating a pre-test probability difficult, and the costs associated with failing to treat or treating unnecessarily can vary from patient to patient. Despite these limitations, evidence-based diagnosis helps us by

• reminding us to balance the cost of missing a potential disease against the often neglected costs and risks of testing for that disease,

• acknowledging two types of diagnostic error: missing disease that is present and treating a disease that is not (over-diagnosis).

The costs and risks of testing

In our fervor to avoid missing something serious, we often underestimate the costs and risks of testing. Of course we worry about failing to diagnose a serious intracranial hemorrhage after a minor head injury, but we should also worry about causing cancer with the ionizing radiation associated with a head CT scan. A febrile infant may have a urinary tract infection, but is the risk of infection high enough and the danger serious enough to warrant catheterizing every infant with a fever? Pulmonary embolism is a dangerous and treatable condition, but the test of choice, CT angiogram, involves exposure to both radiation and intravenous contrast.

Over-diagnosis and over-treatment

Missing a serious disease is not the only type of diagnostic error. We can also diagnose a serious disease that isn’t there. For the emergency physician, discharging a chest-pain patient who turns out to have a myocardial infection is a diagnostic error, but so is admitting a chest-pain patient who turns out to have esophageal reflux, even if we are willing to make 10 of the second type of error (unnecessary hospitalization) in order to avoid one of the first type of error (“missing” a myocardial infarction). Ordering antibiotics to treat cellulitis in a patient with bilateral leg redness is usually a diagnostic error because cellulitis is almost never bilateral, and antibiotics can cause colitis or an allergic reaction [5]. In “Over-Diagnosed: Making People Sick in the Pursuit of Health,” H. Gilbert Welch recounts using an oral hypoglycemic agent to treat a patient with moderate hyperglycemia. The patient subsequently passed out while driving because his blood sugar was too low. The patient has recovered from his neck fracture and is doing fine without diabetes treatment [6].

Conclusions

The real meaning of the word “diagnosis” is naming the disease that is causing a patient’s illness. The cognitive process of assigning this name is a mysterious combination of pattern recognition and the hypothetico-deductive approach that is only remotely related to the mathematical process of using test results to update the probability of a disease. What I refer to as “evidence-based diagnosis” is really evidence-based use of medical tests to guide treatment decisions. Understanding how to use test results to update the probability of disease can help us interpret test results more rationally. Also, evidence-based diagnosis reminds us to consider the costs and risks of testing and the dangers of over-diagnosis and over-treatment, in addition to the costs and risks of missing serious disease.