Random-Effects Meta-analysis: Summarizing Evidence With Caveats

INTRODUCTION

This JAMA Guide to Statistics and Methods explains the difference between fixed and random effects in treatment effect estimates, and the rationale for using random-effects meta-analysis to determine treatment effects across randomized trials conducted in heterogeneous patients and settings.

Questions involving medical therapies are often studied more than once. For example, numerous clinical trials have been conducted comparing opioids with placebos or nonopioid analgesics in the treatment of chronic pain. In a study published in JAMA, Busse et al¹ evaluated the evidence on opioid efficacy from 96 randomized clinical trials and, as part of that work, used random-effects meta-analysis to synthesize results from 42 randomized clinical trials on the difference in pain reduction among patients taking opioids vs placebo using a 10-cm visual analog scale (Figure 2 in Busse et al¹). Meta-analysis is the process of quantitatively combining study results into a single summary estimate and is a foundational tool for evidence-based medicine. Random-effects meta-analysis is the most common approach.

WHY IS RANDOM-EFFECTS META-ANALYSIS USED?

Each study evaluating the effect of a treatment provides its own answer in terms of an observed or estimated effect size. Opioids reduced pain by 0.54 cm more than placebo on a visual analog scale in 1 study²; this was the observed effect size and represents the best estimate from that study of the true opioid effect. The true effect is the underlying benefit of opioid treatment if it could be measured perfectly, and is a single value that cannot directly be known.

If a particular study was replicated with new patients in the same setting multiple times, the observed treatment effects would vary by chance even though the true effects would be the same in each. The belief that the true effect was the same in each study is called the fixed-effect assumption, whereby the fixed effect is the common, unknown true effect underlying each replication. A meta-analysis making the fixed-effect assumption is called a fixed-effect meta-analysis. The corresponding fixed-effect estimate of the treatment effect is a weighted average of the individual study estimates and is always more precise (ie, it has a narrower confidence interval [CI] than that of any individual study, making the estimate appear closer to the true value than any individual study).

However, medical studies addressing the same question are typically not exact replications and they can use different types of medication or interventions for different amounts of time, at different intensities, within different populations, and have differently measured outcomes.³ Differences in study characteristics reduce the confidence that each study is actually estimating the same true effect. The alternative assumption is that the true effects being estimated are different from each other or heterogeneous. In statistical jargon, this is called the random-effects assumption. The plural in effects implies there is more than 1 true effect and random ...