Skip to Main Content

++

An Illustration of Bias and Random Error

++

As is true of any area of intellectual endeavor, students of evidence-based medicine face challenges both in understanding concepts and in becoming familiar with technical language. When asked to say what makes a study valid or reduces its risk of bias, students often respond, “large sample size.” Small sample size does not produce bias, but it can increase the likelihood of a misleading result through random error.

++

An error refers to any deviation from the truth. It can be random, occurring by chance, or systematic, tending in a certain direction. The way we use technical language, “bias” is a synonym for systematic error. You may find the following exercise helpful in clarifying these notions.

++

Consider a set of studies with identical design and sample size. Each study recruits from the same patient pool. Will these studies, with exactly the same type of patients and exactly the same study design, yield identical results? No, they will not. Just as an experiment of 10 coin flips will not always yield 5 heads and 5 tails, the play of chance will ensure that, despite their identical design, each study will have a different result.

++

Consider 4 sets of such studies. Within each set, the design and sample size of each trial are identical. Two of the 4 sets of studies have a small sample size and 2 have a large sample size.

++

Two sets of studies include only randomized clinical trials (RCTs) in which patients, caregivers, and those assessing outcome are all blinded. Design features, such as blinding and complete follow-up, reduce bias. The remaining sets of studies use an observational study design (eg, patients are in treatment or control groups according to their choice or their clinician's choice), which is far more vulnerable to bias. In this exercise, we are in the unique position of knowing the true treatment effect. In Figure 11.1-1, each of the bull's-eyes in the center of the 4 components of the figure represents the truth. Each smaller dot represents not a single patient but the results of 1 repetition of the study. The farther a smaller dot lies from the central bull's-eye, the larger the difference between the study result and the underlying true treatment effect.

++
FIGURE 11.1-1

Four Sets of Identically Conducted Studies Showing Various Degrees of Bias and Random Error

A and B represent randomized trials. C and D represent observational studies. In each part, the studies are of identical sample size and identical design.

Graphic Jump Location
++

Each set of studies represents the results of RCTs or observational studies and of studies of large or small sample size. Before reading further, examine Figure 11.1-1 and draw your own conclusions about the study designs and number ...

Want remote access to your institution's subscription?

Sign in to your MyAccess profile while you are actively authenticated on this site via your institution (you will be able to verify this by looking at the top right corner of the screen - if you see your institution's name, you are authenticated). Once logged in to your MyAccess profile, you will be able to access your institution's subscription for 90 days from any location. You must be logged in while authenticated at least once every 90 days to maintain this remote access.

Ok

About MyAccess

If your institution subscribes to this resource, and you don't have a MyAccess profile, please contact your library's reference desk for information on how to gain access to this resource from off-campus.

Subscription Options

JAMAevidence Full Site: One-Year Subscription

Connect to the full suite of JAMAevidence content and resources including interactive self-assessment, videos, and more.

$495 USD
Buy Now

Pay Per View: Timed Access to all of JAMAevidence

24 Hour Subscription $34.95

Buy Now

48 Hour Subscription $54.95

Buy Now

Pop-up div Successfully Displayed

This div only appears when the trigger link is hovered over. Otherwise it is hidden from view.