1954 Salk vaccine field trials Biggest public health experiment ever Polio epidemics hit U.S. in 20th century Struck hardest at children Responsible for 6% of deaths among 5-9 year olds

Salk vaccine field trial Polio is rare but virus itself is common Most adults experienced polio infection without being aware of it. Children from higher-income families more vulnerable to polio! Paradoxical, but Children in less hygienic surroundings contract mild polio early in childhood while still protected from mother’s antibodies. Develop immunity early. Children from more hygienic surroundings don’t develop such antibodies.

Salk vaccine field trial By 1954, Salk poliomyelitis vaccine was promising Public Health Service and National Foundation for Infantile Paralysis (NFIP) ready to try the vaccine in population Vaccine could not be distributed without controls A yearly drop might mean the drug was effective, or that that year was not an epidemic year. Some children would get vaccine, some would not Raises question of medical ethics

Salk vaccine trial Polio rate of occurrence about 50 per 100,000 Clinical trials needed on massive scale Suppose vaccine was 50% effective and 10,000 subjects in control group, 10,000 subjects in treatment group Would expect 5 polio cases in control group and 2-3 in treatment group Difference could be attributed to random variation Ultimate experiment involved over 1 million

How to design the experiment Treatment and control groups should be as similar as possible Any difference in response should be due to the treatment rather than something else Taking volunteers biases the experiment Fact: volunteers tend to be better educated and more well-to-do than those who don’t participate Relying on volunteers biases the results because subjects are not representative of the population Definition: A study is biased if it systematically favors certain outcomes

NFIP study: Observed Control approach Offer vaccination (treatment) to 2nd graders Control group: 1st and 3rd graders Three grades drawn from same geographical location Advantage: Not much variability between grades Objections: Uncertainty of the diagnostic process Selective use of volunteers

NFIP Observed Control study In making diagnosis physicians would naturally ask whether child was vaccinated Many forms of polio hard to diagnose Borderline cases could be affected by knowledge of whether child was vaccinated Volunteers would result in more children from higher income families in treatment group Treatment group is more vulnerable to disease than control group Biases the experiment against the vaccine

Randomized control approach Subjects randomly assigned to treatment and control groups Control group given placebo Placebo material prepared to look exactly like vaccine Each vial identified only by code number so no one involved in vaccination or diagnostic evaluation could know who got vaccine Experiment was double-blind, neither subjects nor those doing the evaluation knew which treatment any subject received

Results of vaccine trials The randomized, controlled experiment Size Rate (per 100,000) Treatment 200,000 28 Control 71 No consent 350,000 46 The NFIP/Observed Control study Size Rate (per 100,000) Grade 2 (vaccine) 225,000 25 Grade 1, 3 (control) 725,000 54 Grade 2 (no consent) 125,000 44 Source: Thomas Francis, J r., “An evaluation of the 1954 Poliomyelitis vaccine trials---summary report,” American Journal of Public Health vol 45 (1955) pp. 1-63.

Are the results significant? Results show NFIP study biased against vaccine Chance enters the study in a haphazard way: what families will volunteer, which children are in grade 2, etc. For randomized controlled experiment chance enters the study in a planned and simple way: each child has 50-50 chance to be in treatment or control Allows for use of probability to determine if the results are significant

Are the results significant? Two competing positions 1: The vaccine is effective. 2: (Devil’s Advocate) The vaccine has no effect. The differences between the two groups is due to chance. Probability to the rescue: Suppose vaccine has no effect. What are the chances of seeing such a large difference in the two groups? We’ll do the calculations in a few weeks. But they are a billion to one against! Definition: An outcome is statistically significant if the effect is so large that it would rarely occur by chance

Basic principles of statistical design of experiments Randomization Uses chance to assign subjects to treatments Tends to prevent bias, or systematic favoritism, in experiments Replication Repeating the treatment on many subjects reduces role of chance variation Comparison and Control Compare treatments to prevent confounding effect of treatment with other influences Also tends to prevent bias

More complex designs Blocking allows for greater control of influential variables Perhaps vaccine works differently on men and women Set up separate “blocks” of men and women Carry out randomization separately within each block Allows for separate conclusions about each block

Matched pairs design Special case of blocking Pair up individuals or apply two treatments to same individual Often used for before-and-after studies Example: Effectiveness of a diet using weights of subjects measured before and after the diet treatment

Randomization How to assign subjects at random Pick from a hat, computer generator, tables of random numbers http://bcs.whfreeman.com/ips4e/pages/bcs-main.asp?v=category&s=00010&n=99000&i=99010.01&o= Table B: Line 101 19223 95034 05756 28713 12531 42544 . . . 102 73676 47150 99400 01927 27754 42648 . . . 103 45467 71709 77558 00095 32863 29485 . . . . Simulates random digits Every position has “the same probability” of being 0, 1, . . . , 9 The digits in any position have “no influence” over the digits in any other position (e.g., they are “independent” of each other) Doesn’t matter whether you pick a row, a column or a block as long as you do so consistently and without peeking

Using Table B Assign numerical labels to population 00 Alex Baum 01 Tim Blaha 02 Rachel Br. 03 Kari Chr. 04 Danielle D. 05 Alex Eis. 06 Patricia Glab 07 Danny Gr. 08 Vince Huang 09 Nita Jain 10 Peter Juul 11 Mou Khan 12 Asad Khawar 13 Lyndsey Kl. 14 Jamie Ly. 15 Dan Masello 16 Alicia Mazzara 17 Martha Mont. 18 Jesse Moreno 19 Oyeyinka Oy. 20 Ginger Price 21 Claire Rich. 22 Leon Schneider 23 Richard South. 24 Hikaru Ter. 25 Jacob Titus 26 Jesse Trent. 27 Andrew Tulloch 28 Bryce Turner 29 Jonathan White 30 Matthew Yelle Assign numerical labels to population Start anywhere in Table B and read off groups of numbers Example: Pick random sample of 5 students Label students from 00 to 30 We used 2 digits to label so pick digits in pairs. E.g., from Line 116 pick . . .