Chapter 11.3: Randomized Trials Stopped Early for Benefit

Randomized Clinical Trials Stopped Early for Benefit Play a Prominent Role in the Medical Literature

Investigators may stop randomized clinical trials (RCTs) earlier than planned because of perceived harm of the experimental intervention, because they lose hope in achieving a positive result, or because the sponsor wishes to save money.¹ The reason for early stopping that may have the most effect on clinical practice, however, is that investigators note treatment effects that appear to be unlikely by chance—and that are usually large—that persuade them that the experimental intervention is beneficial. Trials stopped early for apparent benefit—which we will refer to as truncated RCTs (tRCTs)—often receive considerable attention. They appear in the most prominent journals and in the popular press^2,3 markedly increasing the likelihood of widespread dissemination and subsequent citation. These trials may, with remarkable rapidity, form the basis of practice guidelines and criteria for quality of medical care—and such recommendations may persist after subsequent studies have debunked the results of the tRCTs. For example, such has been the fate of stopped-early RCTs documenting the effect of tight glucose control with insulin in patients in the intensive care unit, β-blockers in patients undergoing vascular surgery, and activated protein C in sepsis.⁴

Truncated Randomized Clinical Trials Are at Risk of Overestimating Treatment Effects

Truncated RCTs will, on average, overestimate treatment effects, and this overestimation may be large, particularly when tRCTs have a small number of outcome events. To understand this overestimation, imagine a number of similar RCTs that address a particular research question in which the truth is a small treatment effect. If trials are at low risk of bias, their results will vary only because of chance. Some trials will start and continue near the truth. However, because of imprecision when the sample size is still small, some will reveal apparent harm early on, and some will reveal large overestimates: the latter 2 categories of trials will approach the truth as the data accumulate (Figure 11.3-1).

Truncated RCTs will belong to the group of trials that overestimate because they are at the high end of the random distribution of results. Correspondingly, the non-tRCTs will tend to slightly underestimate. Thus, the overestimation from tRCTs is largely the result of random error. If such studies were to continue to their planned sample sizes, then because of what Pocock and White⁵ have described as “regression to the truth,” they would still produce overestimates of effect, but those overestimates would be smaller than those seen with early stopping.

As Figure 11.3-1 suggests, large random differences from the true effect are more likely to happen early in a trial when sample sizes are small.^5,6 Thus, trials ...