Chapter 12.1

### Introduction

For every treatment, there is a true, underlying effect that any individual experiment can only estimate (see Chapter 6, Why Study Results Mislead: Bias and Random Error). Investigators use statistical methods to advance their understanding of this true effect. This chapter explores the logic underlying one approach to statistical inquiry: hypothesis testing. Readers interested in how to teach the concepts reviewed in this chapter to clinical learners may be interested in an interactive script we have developed for this purpose.1

The hypothesis-testing approach to statistical exploration is to begin with what is called a null hypothesis and try to disprove that hypothesis. Typically, the null hypothesis states that there is no difference between the interventions being compared. To start our discussion, we will focus on dichotomous (yes/no) outcomes, such as dead or alive or hospitalized or not hospitalized.

For instance, in a comparison of vasodilator treatment in 804 men with heart failure, investigators compared the proportion of enalapril-treated patients who died with the proportion of patients who received a combination of hydralazine and nitrates who died.2 We start with the assumption that the treatments are equally effective, and we adhere to this position unless the results make it untenable. We could state the null hypothesis in the vasodilator trial more formally as follows: the true difference in the proportion of patients surviving between those treated with enalapril and those treated with hydralazine and nitrates is 0.

In this hypothesis-testing framework, the statistical analysis addresses the question of whether the observed data are consistent with the null hypothesis. Even if the treatment truly has no positive or negative effect on the outcome (ie, the effect size is 0), the results observed will rarely agree exactly with the null hypothesis. For instance, even if a treatment has no true effect on mortality, seldom will we see exactly the same proportion of deaths in treatment and control groups. As the results diverge farther and farther from the finding of “no difference,” however, the null hypothesis that there is no true difference between the treatments becomes progressively less credible. If the difference between results of the treatment and control groups becomes large enough, we abandon belief in the null hypothesis. We further develop the underlying logic by describing the role of chance in clinical research.

### The Role of Chance

In Chapter 6, Why Study Results Mislead: Bias and Random Error, we considered a balanced coin with which the true probability of obtaining either heads or tails in any individual coin toss is 0.5. We noted that if we tossed such a coin 10 times, we would not be surprised if we did not see exactly 5 heads and 5 tails. Occasionally, we would get results quite divergent from the 5:5 split, such as 8:2 or even 9:1. Furthermore, very infrequently, the 10 coin tosses ...

Sign in to your MyAccess profile while you are actively authenticated on this site via your institution (you will be able to verify this by looking at the top right corner of the screen - if you see your institution's name, you are authenticated). Once logged in to your MyAccess profile, you will be able to access your institution's subscription for 90 days from any location. You must be logged in while authenticated at least once every 90 days to maintain this remote access.

Ok

## Subscription Options

### JAMAevidence Full Site: One-Year Subscription

Connect to the full suite of JAMAevidence content and resources including interactive self-assessment, videos, and more.