1. The Practice of Statistics in the Life Sciences Fourth Edition
Chapter 7: Samples and observational studies
Copyright © 2018 W. H. Freeman and Company

2. Objectives Samples and observational studies
Observational study versus experiment
Population versus sample
Randomness and bias
The simple random sample (SRS)
Other probability samples
Sample surveys
Comparative observational studies

3. Observational versus experimental studies (1 of 2)
Observational study: record data on individuals without attempting to influence the responses.
In 1992, several major medical organizations said that women should take hormones such as estrogen after menopause, because women who took hormones seemed to reduce their risk of a heart attack by 35% to 50%.
The first statement describes the results of observational studies. Access to medical care, lifestyle, socioeconomic status, are possible confounding variables.
The second statement describes experiments, with a random assignment of subjects to treatment.

4. Observational versus experimental studies (2 of 2)
Experimental study: Deliberately impose a treatment on individuals and record their responses. Influential factors can be controlled.
By 2002, several studies concluded that hormone replacement does not reduce the risk of heart attacks. These studies had assigned women to either hormone replacement or to placebo pills. The assignment was done by a coin toss.
The first statement describes the results of observational studies. Access to medical care, lifestyle, socioeconomic status, are possible confounding variables.
The second statement describes experiments, with a random assignment of subjects to treatment.

5. Observational study or experiment? (1 of 2)
A 2013 Gallup study investigated how phrasing affects the opinions of Americans regarding physician-assisted suicide. Telephone interviews were conducted with a random sample of 1,535 national adults. Using random assignment, 719 heard the question in Form A and 816 the one in Form B.
Form A: When a person has a disease that cannot be cured, do you think doctors should be allowed by law to end the patient’s life by some painless means if the patient and his or her family request it?
The individuals surveyed did not choose which question they heard. This was done by Gallup, using random assignment. Therefore, this is a comparative randomized experiment.

6. Observational study or experiment? (2 of 2)
Form B: When a person has a disease that cannot be cured and is living in severe pain, do you think doctors should or should not be allowed by law to assist the patient to commit suicide if the patient requests it?
70% of those given Form A answered “should be allowed”, compared with only 51% of those given Form B. What type of study is this?
Observational study
Randomized experiment
Neither. This is just anecdotal evidence.
The individuals surveyed did not choose which question they heard. This was done by Gallup, using random assignment. Therefore, this is a comparative randomized experiment.

7. Confounding
Two variables are confounded when their effects on a response variable cannot be distinguished. Observational studies often fail to yield clear causal conclusions, because the explanatory variable is confounded with lurking variables.
Observational studies play an important role in the building of scientific knowledge, but they cannot establish whether observed differences between groups can be attributed to the groups’ differentiating feature or to confounding variables. Experiments, on the other hand, provide an opportunity for manipulating the environment and confounding variables. Therefore, well-designed experiments yield stronger conclusions.

8. Population versus sample
Population: The entire group of individuals in which we are interested but can’t usually assess directly
A parameter is a number summarizing a characteristic of the population.
How well the sample represents the population depends on the sample design.
Parameters are often referred to by a Greek letter, whereas statistics are typically labeled using English letters (sometimes in combination with math symbols).
Sample: The part of the population we actually examine and for which we do have data
A statistic is a number summarizing a characteristic of a sample.

9. The role of randomness in sampling
How do you select the individuals/units in a sample?
Probability sampling: individuals or units are randomly selected; the sampling process is unbiased.
Voluntary response samples and convenience samples are not scientific, because they are not truly representative of the whole population of interest; rather, they tend to be strongly biased.

10. Bias
Bias is the systematic tendency for a study to favor certain outcomes.
In the diagram featured on this slide, you can see how the lower right figure is “off target,”, while the other three are, on average, centered around their target.
Figure 6.3 Baldi/Moore, The Practice of Statistics in the Life Sciences, 4e, © 2018 W.H. Freeman and Company

11. Examples of bad sampling (1 of 2)
Ann Landers summarizing responses of readers: 70% of (~10,000) parents wrote in to say that having kids was not worth it—if they had to do it over again, they wouldn’t. But a random sample showed that 91% of parents WOULD have kids again. What do you think explains such drastically different responses?
Most letters to newspapers are written by disgruntled people. They are not representative of the population of interest (all parents).
People you find in one location may be very different from the people you would find at a different location. Restricting your sample to one location for convenience means that your sample will not be representative of your whole population of interest (here, probably the adult population).
Visit for other examples of bad sampling.

12. Examples of bad sampling (2 of 2)
Would you expect very different responses on the potential legalizing of marijuana if you asked the first people you saw on the parking lot of a university or the first people you saw on the parking lot of a church?
Most letters to newspapers are written by disgruntled people. They are not representative of the population of interest (all parents).
People you find in one location may be very different from the people you would find at a different location. Restricting your sample to one location for convenience means that your sample will not be representative of your whole population of interest (here, probably the adult population).
Visit for other examples of bad sampling.

13. The simple random sample
A simple random sample (SRS) is made of randomly selected individuals. Each individual in the population has the same probability of being in the sample. All possible samples of size n have the same chance of being the sample drawn.
How to choose an SRS?
Draw from a hat (lottery style).
Use a table of published random numbers (Table A).
Use software that generates random numbers.

14. Choosing a simple random sample (1 of 2)
We need to select a random sample of 5 from a class of 20 students.
List and number all members of the population, which is the class of 20 students, listed to the right.
Use Table A or technology to get 5 random numbers. Assume we got the number 17, 9, 13, 7, and 2.
The first five random numbers matching numbers assigned to people make the SRS.
The first individual selected is Ramon, number 17.
Then Henry (9).
The next three to be selected are Moe, George, and Amy (13, 7, and 2).

15. Choosing a simple random sample (2 of 2)01
Alison 02
Amy 03
Brigitte 04
Darwin 05
Emily 06
Fernando 07
George 08
Harry 09
Henry 10
John 11
Kate 12
Max 13
Moe 14
Nancy 15
Ned 16
Paul 17
Ramon 18
Rupert 19
Tom 20
Victoria

16. Other probability samples (1 of 2)
A stratified random sample: Make sure your sample has known percentages of individuals of certain types (strata).
America's State of Mind report was based on a probability sample of Medco's de-identified database of members with 24 months of continuous insurance enrollment.  Sampling was stratified by age group and sex to match the demographics of the whole customer base.
A multistage sample: Select your final sample in stages, by sampling successively within a sample within a sample.
Multistage samples are often used by the government to survey the whole population, because they are more economical. However, statistical analysis for a multistage sample is more complex than for an SRS.

17. Other probability samples (2 of 2)
The National Youth Tobacco Survey administered in schools uses a sampling procedure to generate a nationally representative sample of students in grades 6–12. Sampling is probabilistic and consists of selecting:
Counties as Primary Sampling Units (PSU)
Schools within each selected PSU
Classes within each selected school
Multistage samples are often used by the government to survey the whole population, because they are more economical. However, statistical analysis for a multistage sample is more complex than for an SRS.

18. Sample surveys
A sample survey is an observational study that relies on a random sample drawn from the entire population.
Opinion polls are sample surveys that typically use voter registries or telephone numbers to select their samples.
In epidemiology, sample surveys are used to establish the incidence (rate of new cases per year) and the prevalence (rate of all cases at one point in time) of various medical conditions, diseases, and lifestyles. These are typically stratified or multistage samples.

19. Some survey challenges
Undercoverage: Parts of the population are systematically left out.
Nonresponse: Some people choose not to answer/participate.
Wording effects: Biased or leading questions, and complicated/confusing statements can influence survey results.
Response bias: Fancy term for lying or forgetting (especially on sensitive/personal issues). Can be amplified by the survey method (in person vs. by phone or online).
Undercoverage: For instance, surveys of households omit homeless individuals. Surveys conducted online tend to include a younger and more urban population. To avoid undercoverage of individuals with a cell phone but no landline phone (>30% of Americans since 2011), most phone-based surveys now control for phone access (landline only, cell only, both) by stratifying their samples.
Nonresponse: A growing issue, especially in commercial polls.
Wording effects: Questions worded like “Do you agree that it is awful that…” are prompting you to give a particular response. Small changes in wording can have a substantial effect on responses.
Response bias: Questions about the past or sensitive issues often get biased responses (“How much do you drink?” or “How much do you weigh?”).

20. How bad is nonresponse in surveys? (1 of 2)
The Census Bureau’s American Community Survey (ACS): ~2.5% via mail with reminders. Response is mandatory.
University of Chicago’s General Social Survey (GSS): ~30%—in person.
Pew Research Center methodology survey up to ~90% in 2012
Private polling firms such as SurveyUSA: ~90% as of 2002 (stopped showing after that) phone (with interviewer or automated call) or online.
Source:

21. How bad is nonresponse in surveys? (2 of 2)
Source:

22. Wording effects (1 of 2)
A 2013 Gallup study investigated how phrasing affects the opinions of Americans regarding physician-assisted suicide. Telephone interviews were conducted with a random sample of 1,535 national adults. Using random assignment, 719 heard the question in Form A and 816 the one in Form B.
Form A: When a person has a disease that cannot be cured, do you think doctors should be allowed by law to end the patient’s life by some painless means if the patient and his or her family request it?
Because the two questions were randomly assigned, only question wording differed consistently between the two forms. Any issue of response bias, nonresponse, and undercoverage would apply equally to both questions, and therefore could not explain the substantial difference in opinions.

23. Wording effects (2 of 2)
Form B: When a person has a disease that cannot be cured and is living in severe pain, do you think doctors should or should not be allowed by law to assist the patient to commit suicide if the patient requests it?
Question wording resulted in a substantial difference in opinions: 70% of those given Form A answered “should be allowed,” compared with only 51% of those given Form B.
Because the two questions were randomly assigned, only question wording differed consistently between the two forms. Any issue of response bias, nonresponse, and undercoverage would apply equally to both questions, and therefore could not explain the substantial difference in opinions.

24. An example of response bias (1 of 2)

25. An example of response bias (2 of 2)
From this data we see that men tend to over report their height by 1.22 cm, and overreport their weight by 0.3 kg. Women tend to overreport their height by 0.68 cm, but underreport their weight by 1.39 kg.

26. Comparative observational studies
Case-control studies start with 2 random samples of individuals with different outcomes, and look for exposure factors in the subjects’ past (“retrospective”).
Individuals with the condition are cases, and those without are controls.
Good for studying rare conditions. Selecting controls is challenging.
Cohort studies enlist individuals of common demographic and keep track of them over a long period of time (“prospective”). Individuals who later develop a condition are compared to those who don’t develop the condition.
Cohort studies examine the compounded effect of factors over time.
Good for studying common conditions. Very expansive.

27. A case-control study example (1 of 2)
Aflatoxicosis epidemics Aflatoxins are secreted by a fungus found in damaged crops and can cause severe poisoning and death. The Kenya Ministry of Health investigated a 2004 outbreak of aflatoxicosis resulting in over 300 cases of liver failure. A sample of 40 case-patients and 80 healthy controls were asked how they had stored and prepared their maize.
The study findings indicate that improper corn storage was a principal source of poisoning.

28. A case-control study example (2 of 2)
The case-patients were randomly selected from a list of individuals admitted to a hospital during the 2004 outbreak for unexplained acute jaundice.
Control individuals were selected to be as similar to the case-patients as possible, yet randomly selected.
Preliminary data suggested that soil, microclimate, and farming practices might have played a role, but not age or gender.
For each case-patient, two individuals from the patient’s village with no history of jaundice symptoms were randomly selected.
The study findings indicate that improper corn storage was a principal source of poisoning.

29. Examples of cohort studies (1 of 2)
The Nurses’ Health Study is one of the largest prospective observational studies designed to examine factors that may affect major chronic diseases in women.
Since 1976, the study has followed a cohort of over 100,000 registered nurses. Every two years, they receive a follow-up questionnaire about diseases and health-related topics. Response rate: ~90% each time.

30. Examples of cohort studies (2 of 2)
2007 report on age-related memory loss: About 20,000 women ages 70+ had completed telephone interviews every two years to assess their memory with a set of cognitive tests. One of the findings: the more women walked during their late 50s and 60s, the better their memory score was at age 70 and older. However, because this is an observational study, we cannot unambiguously conclude that walking has a protective effect against memory loss.