Conceptually, EBM involves 3 fundamental principles. First, optimal clinical decision making requires awareness of the best available evidence, which ideally will come from systematic summaries of that evidence. Second, EBM provides guidance to decide whether evidence is more or less trustworthy—that is, how confident can we be of the properties of diagnostic tests, of our patients' prognosis, or of the impact of our therapeutic options? Third, evidence alone is never sufficient to make a clinical decision. Decision makers must always trade off the benefits and risks, burden, and costs associated with alternative management strategies and, in doing so, consider their patients' unique predicament and values and preferences.1
In 1992, Antman et al2 published an article that compared the recommendations of experts for management of patients with myocardial infarction to the evidence that was available at the time the recommendations were made. Figures 2-1 and 2-2 summarize their results in forest plots. Both are cumulative meta-analyses: the first of thrombolytic therapy for myocardial infarction and the second for lidocaine antiarrhythmic therapy. In both cases, the line in the center represents an odds ratio of 1.0 (treatment is neither beneficial or harmful). As in any forest plot, the dots represent the best estimates of treatment effect (often from individual studies; in this case from the totality of accumulated evidence), and the associated lines represent the 95% confidence intervals (CIs).
Thrombolytic Therapy in Acute Myocardial Infarction
Abbreviation: CI, confidence interval; RCTs, randomized clinical trials.
This is a cumulative meta-analysis of thrombolytic therapy for myocardial infarction. The line down the center, the odds ratio, equals 1.0. The dots represent best estimates, and the lines around the dots are 95% CIs. The numbers on the left side of the figure are trials and patient totals across trials.
Early on, the CIs are very wide. By 10 trials, it appears therapy reduces mortality, but the effect is still uncertain. By 30 trials, the effect seems secure. However, 40 000 more patients were enrolled after the answer was in. Why?
The right side of the figure displays current reviews and textbook recommendations as data accumulated. Recommendations are in favor (“Yes”), against (“No”), or “Not mentioned.” Two key points: (1) at the same time, experts disagreed, and (2) it took 10 years for experts to catch up with evidence.
Adapted from Antman et al.2
Prophylactic Lidocaine in Acute Myocardial Infarction
Abbreviation: CI, confidence interval; RCTs, randomized clinical trials.
This slide shows a cumulative meta-analysis of the effect of prophylactic lidocaine in preventing death from myocardial infarction. In this case, there is never any evidence of benefit. Ultimately, harm is not proved, but there clearly is no benefit. Most experts, however, were recommending therapy despite RCT evidence. Also, as in Figure 2-1, there was a lot of disagreement among experts.
Adapted from Antman et al.2
The “Patients” column presents the total number of patients enrolled in all randomized clinical trials (RCTs) conducted to the date specified in the “Year” column—the reason we call it a cumulative meta-analysis. In both figures, early on, with relatively few patients, the CIs are wide, but they progressively narrow as new trials were reported.
For the thrombolytic example, by 10 trials and approximately 2500 patients, it appears that thrombolytic therapy reduces mortality, but the CIs are still wide enough to permit residual uncertainty. By 30 trials and more than 6000 patients, the reduction in odds of death of approximately 25% seems secure.
Despite this apparently definitive result, additional trials that enrolled 40000 patients—half of whom did not receive the benefits of life-prolonging thrombolytic therapy—were conducted. Why was this necessary?
The right side of each figure, which presents the guidance expressed in then-current reviews and textbooks as the data were accumulating, provides the answer to this question. Until approximately a decade after the answer was in, there was considerable disagreement among experts, with many recommending against, or not mentioning, thrombolytic therapy. To the detriment of patients who did not receive thrombolytic therapy during this period, it took a decade for the experts to catch up with the evidence.
Figure 2-2 tells a perhaps even more disturbing story. This cumulative meta-analysis reveals that there was never any RCT evidence that suggested a lower mortality with prophylactic lidocaine after myocardial infarction—indeed, point estimates suggested an increase in death rate. Nevertheless, although we once again see widespread disagreement among the experts, most texts and reviews were recommending prophylactic lidocaine during the 2 decades during which the RCT evidence was accumulating.
Why the expert disagreement, the lag behind the evidence, and the recommendations inconsistent with the evidence? These stories come from the era before systematic reviews and meta-analyses were emerging in the late 1980s. If the evidence summaries presented in the forest plots had been available to the experts, they would have grasped the benefits of thrombolytic therapy far earlier than they did and abandoned prophylactic lidocaine far earlier. Indeed, following EBM principles that limit reliance on biologic rationale and place far more emphasis on empirical evidence (see Chapter 3, Evidence-Based Medicine and the Theory of Knowledge), the experts may never have started using lidocaine.
Rational clinical decisions require systematic summaries of the best available evidence. Without such summaries, clinicians—expert or otherwise—will be unduly influenced by their own preconceptions and by unrepresentative and often lower-quality evidence. This, the first principle of EBM, immediately raises another question: “How does one recognize the best evidence?”
Guides to Confidence in Estimates
Summaries of the best evidence for diagnosis, prognosis, or treatment present evidence, respectively, for how to interpret test results, predict patients' likely fate, or understand the impact of alternative management strategies. Sometimes, such evidence is trustworthy—we have high confidence in estimates of test properties, patients' prognosis, or treatment effects. At other times, limitations in evidence leave us uncertain. Evidence-based medicine provides guidance to distinguish between these situations and the range of confidence between them.
Historically, EBM answered the question, “What is the best evidence?” with hierarchies of evidence, the most prominent of which was the hierarchy related to evidence that supported therapeutic interventions (Figure 2-3). Issues of diagnosis or prognosis require different hierarchies. For studies of the accuracy of diagnostic tests, the top of the hierarchy includes studies that enrolled patients about whom clinicians had diagnostic uncertainty and that undertook a blind comparison between the candidate test and a criterion standard (see Chapter 18, Diagnostic Tests, and Chapter 20, Prognosis). For prognosis, prospective observational studies accurately documenting exposures and outcomes and following up all patients during relevant periods would sit atop the hierarchy.
Hierarchy of Evidence
Because we would like to optimally individualize patient care, n-of-1 randomized clinical trials are at the top of the hierarchy of study designs, followed by conventional randomized trials. Next in the hierarchy are observational studies; we should try to find studies that focus on outcomes important to the patient. Next, if there are no clinical studies available, we may look at basic scientific research, although caution must be used in extrapolating the results to the clinical setting. Clinical experience is at the bottom of the hierarchy, either your own or that of colleagues or experts.
Returning to the hierarchy of therapy, noting the limitations of human intuition,3 EBM places the unsystematic observations of individual clinicians lowest on the hierarchy. Noting that predictions based on physiologic experiments are often right but sometimes disastrously wrong, EBM places such experiments at the next step up in the hierarchy. Observational studies that measure the apparent impact on patient-important outcomes and RCTs constitute the next 2 steps up the hierarchy of evidence.
All of the sources of evidence mentioned thus far involve generalizations from groups of patients to an individual, and all are limited in this regard. The same strategies that minimize bias in conventional therapeutic trials that involve multiple patients, however, can guard against misleading results in studies that involve single patients.4 In the n-of-1 RCT, a patient and clinician are blind to whether that patient is receiving active or placebo medication. The patient makes quantitative ratings of troublesome symptoms during each period, and the n-of-1 RCT continues until both the patient and the clinician conclude that the patient is or is not obtaining benefit from the target intervention. An n-of-1 RCT can provide definitive evidence of treatment effectiveness in individual patients5,6 and is thus at the top of the evidence hierarchy. Unfortunately, n-of-1 RCTs are restricted to chronic conditions with treatments that act and cease acting quickly and are subject to considerable logistic challenges. We therefore must usually rely on studies of other patients to make inferences regarding our patient.
This hierarchy is far from absolute, and a more sophisticated framework has emerged for judging confidence in estimates of effect. Table 2-1 summarizes that framework, formulated by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group, originally to provide an approach to the development of clinical practice guidelines.7,8 The GRADE approach involves rating our confidence in estimates of the effects of health care interventions (also referred to as quality of evidence) as high, moderate, low, or very low. Consistent with the previous hierarchy approach, in the GRADE guidance, RCTs begin as high confidence and observational studies begin as low confidence. We lose confidence in a body of RCT evidence, however, if studies have major problems in design and execution (risk of bias); results are imprecise, inconsistent, or indirect (eg, the population of interest differs from the population studied—see Chapter 13.4, Surrogate Outcomes); or we have a high suspicion of publication bias (see Chapter 23, Understanding and Applying the Results of a Systematic Review and Meta-analysis). When a body of RCT evidence suffers from a number of these limitations, the confidence in estimates may be low or even very low.
Confidence Assessment Criteria8
Similarly, if treatment effects are sufficiently large and consistent, the GRADE approach allows for moderate or even high confidence ratings from carefully conducted observational studies. For example, observational studies have allowed extremely strong inferences about the efficacy of insulin in diabetic ketoacidosis or that of hip replacement in patients with debilitating hip osteoarthritis.
The EBM approach implies a clear course of action for clinicians addressing patient problems. They should seek the highest-quality evidence available to guide their clinical decisions. This approach makes it clear that any claim that there is no evidence for the effect of a particular treatment is a non sequitur. The available evidence may warrant very low confidence—it may be the unsystematic observation of a single clinician or physiologic studies that point to mechanisms of action that are only indirectly related—but there is always evidence.
Evidence Is Never Enough to Drive Clinical Decision Making
First, picture a woman with chronic pain from terminal cancer. She has come to terms with her condition, resolved her affairs, said her good-byes, and wishes to receive only palliative care. She develops severe pneumococcal pneumonia. Evidence that antibiotic therapy reduces morbidity and mortality from pneumococcal pneumonia warrants high confidence. This evidence does not, however, dictate that this patient should receive antibiotics. Her values—emerging from her comorbidities, social setting, and beliefs—are such that she would prefer to forgo treatment.
Now picture a second patient, an 85-year-old man with severe dementia who is mute and incontinent, is without family or friends, and spends his days in apparent discomfort. This man develops pneumococcal pneumonia. Although many clinicians would argue that those responsible for his decision making should elect not to administer antibiotic therapy, others would suggest that they should. Again, evidence of treatment effectiveness does not automatically imply that treatment should be administered.
Finally, picture a third patient, a healthy 30-year-old mother of 2 children who develops pneumococcal pneumonia. No clinician would doubt the wisdom of administering antibiotic therapy to this patient. This does not mean, however, that an underlying value judgment has been unnecessary. Rather, our values are sufficiently concordant, and the benefits so overwhelm the risk of treatment that the underlying value judgment is unapparent.
By values and preferences, we mean the collection of goals, expectations, predispositions, and beliefs that individuals have for certain decisions and their potential outcomes. The explicit enumeration and balancing of benefits and risks that are central to EBM bring the underlying value judgments involved in making management decisions into bold relief.
Acknowledging that values play a role in every important patient care decision highlights our limited understanding of how to ensure that decisions are consistent with individual and, where appropriate, societal values. As we discuss further in the final section of this chapter, developing efficient processes for helping patients and clinicians work together toward optimal decisions consistent with patient values and preferences remains a frontier for EBM.
Next, we comment on additional skills that clinicians must master for optimal patient care and the relation of those skills to EBM.