BIAS IN SURVEYS AND EXPERIMENTS

Literary Digest Poll of 1936 ■The Literary Digest’s straw poll predictions regarding presidential elections had a perfect record up until 1936 (correct predictions made from ) ■1936: conducted a poll by mail using addresses culled from its subscribers and publicly available lists (registered automobile owners and telephone users) –10 million polls sent –2.4 million responses received ■Predicted landslide victory for Governor Alfred Landon over Franklin Delano Roosevelt

Lanarkshire Milk Experiment (1930) ■From 4 months (Feb – Jun), 20,000 school children in Lanarkshire, Scotland were studied –5,000 received ¾ pint of raw milk or pasteurized milk each –10,000 received no milk ■Growth measured by taking difference in weight at beginning and end of experiment (weight was measured in the students clothes) ■Students were placed into groups based on the alphabet, but the head teachers who helped organize the experiment were instructed, “In any particular school where there was any group to which these methods had given an undue proportion of well fed or ill nourished children [to substitute some students for others] in order to obtain a more level selection.” ■The results suggested that drinking milk would stunt student growth

The Pepsi Challenge ■A Pepsi representative gives the participant two unlabeled samples of soda and asks them which they prefer ■The results leaned toward a consensus that Pepsi was preferred by more Americans

According to the tabloids… Cause Cancer ■Divorce ■Wi-fi ■Toiletries ■Coffee Prevent Cancer ■Crusts ■Red-pepper ■Liquorice ■Coffee

Prison Drug Treatment Program ■A prison starts a drug treatment program where prisoners can opt in. ■Prisoners take classes and receive counseling, and successful completion of the course may result in a sentence reduction of up to a year. ■5 years later, it is recorded that the recidivism rates (percentage of former prisoners who are rearrested for a similar offense) are much lower for inmates who had gone through the program

Other sources of bias ■Publication Bias: Results that are interesting or serve a particular interest may be more likely to get published –Suppose two groups managed to receive a small amount of funding to study the effect between wearing purple pajamas and cancer risk. One group determines there is a reduction in cancer by wearing the pajamas and the other does not. –Which is more likely to get published? ■Healthy User Bias: People who will faithfully engage in activities that are good for them are very different than people who don’t –Now that purple pajamas have been confirmed by an extensive longitudinal study to reduce cancer rates, who is more likely to wear purple pajamas: a student at MIT or a WalMart employee? –Olive oil and granola probably do not prevent cancer