1. Chapter 4: Designing Studies
Section 4.1 –
Sampling & Surveys

3. Vocabulary
The population in a statistical study is the entire group of individuals we want information about. A census collects data from every individual in the population. A sample is a subset of individuals in the population from which we actually collect data.

4. Example – Sampling students and soda
Identify the population and sample in each of the following settings.
(a) The student government at a high school surveys 100 students at the school to get their opinions about a change to the bell schedule.
Population: Sample:
(b) The quality control manager at a bottling company selects a sample of 10 cans from the production line every hour to see whether the volume of the soda is within acceptable limits.

5. The Idea of a Sample Survey
We often draw conclusions about a whole population on the basis of a sample. Choosing a sample from a large, varied population is not that easy.

6. Example – How is consumer confidence measured?
Each month, the Conference Board takes a representative sample of 5000 households and asks them five questions about their appraisal of current business and employment conditions and about their expectations regarding business conditions, employment conditions, and total family income 6 months from the time of the survey. For each question, there are three possible responses: positive, negative, and neutral. These are combined into an index that is calculated relative to the value of 100 in In July the index stood at 80.3, which was weak with respect to historical levels but much better than in the recent past. In May 2010 the index stood at 63.3!

7. How to Sample Badly
Choosing individuals from the population who are easy to reach results in a convenience sample. Convenience samples often produce unrepresentative data. The design of a statistical study shows bias if it would consistently underestimate or consistently overestimate the value you want to know. Any sample may over/underestimate the value you want to know. Bias occurs when this consistently occurs.

8. AP Exam Tip!!
If you are asked to describe how the design of a study leads to bias, you’re expected to do TWO things: 1) Identify a problem with the design 2) Explain how this problem would lead to an underestimate or overestimate. **Bias is not just bad luck in one sample.

9. How to Sample Badly
A voluntary response sample consists of people who choose themselves by responding to a general invitation. Voluntary response samples show bias because people with strong opinions (often in the same direction) are most likely to respond.

10. Example – Illegal Immigration
In May 2010 the Los Angeles City Council voted to ban most travel and contracts with the state of Arizona to protest Arizona’s new immigration enforcement law. The Los Angeles Times conducted an online poll that asked whether the City Council was right to pass a boycott of Arizona. The results showed that 96% of the 41,068 people in the sample said “No.” What type of sample did the Times use in this poll? Explain how this sampling method could lead to bias in the poll results.

11. Check Your Understanding
For each of the following situations, identify the sampling method used. Then explain how the sampling method could lead to bias. 1) A farmer brings a juice company several crates of oranges each week. A company inspector looks at 10 oranges from the top of each crate before deciding whether to buy all the oranges. 2) The ABC program Nightline once asked whether the United Nations should continue to have its headquarters in the U.S. Viewers were invited to call one telephone number to respond “yes” and another to respond “no.” There was a charge for calling either number. More than 186,000 callers responded, and 67% said “no.”

12. How to Sample Well: Simple Random Sampling
A sample chosen by chance rules out both favoritism by the sampler and self-selection by respondents.
Random sampling involves using a chance process to determine which members of a population are included in the sample.
A simple random sample (SRS) of size n is chosen in such a way that every group of n individuals in the population has an equal chance to be selected as the sample.
(Use a random number generator or random number table.)

13. How to Choose a SRS

14. Example – Going Flat
Environmental researchers want to know what proportion of drivers have cars with underinflated tires. They number the cars in the parking lot of a local mall from 1 to 583 and use the random integer generator to select 50 different integers from 1 to 583. Use your graphing calculator to randomly select 50 cars. (Do not use RandInt(1, 583, 50) because you may get repeated cars. )

15. AP Exam Common Error...
When describing how to select a sample using a random integer generator, many students forget to address what to do with repeated integers. Students MUST explicitly state that repeated integers should be ignored or say they will generate random integers until they get n different numbers in the specified range.

16. Example – Mall Hours
The management company of a local mall with 21 stores plans to survey 3 of them to determine the hours they would like to stay open during the holiday season. Use Table D at line 101 to select an SRS of size 3 stores.

17. Stratified Random Sample
To get a stratified random sample, start by classifying the population into groups of similar individuals, called strata.
Then choose a separate SRS in each stratum and combine these SRSs to form the sample of size n.
**Your total SRS will be divided among all strata to form a sample size of n.

18. Cluster Sample
Although a stratified random sample can sometimes give more precise information about a population than an SRS, both sampling methods are hard to use when populations are large and spread out over a wide area.
In that situation, we’d prefer a method that selects groups of individuals that are “near” one another.
To get a cluster sample, start by classifying the population into groups of individuals that are located near each other, called clusters. (Ideally, a cluster should be a small version of the population.)
Then choose an SRS of the clusters. All individuals in the chosen clusters are included in the sample.

19. Example – A Good Read
A school librarian wants to know the average number of pages in all the books in the library. The library has 20,000 books, arranged by type (fiction, biography, history, and so on) in shelves that hold about 50 books each. You want to select a random sample of 500 books. (a) Explain how to select a simple random sample of 500 books. (b) Explain how to select a stratified random sample of 500 books. Justify your choice of strata. Why might the librarian want to choose a stratified random sample? (c) Explain how to select a cluster sample of 500 books. Justify your choice of clusters. Why might the librarian want to choose a cluster sample?

20. Homework
Page 229 – 232 #1, 3, 5, 7, 9, 11, 13, 17, 19, 21, 23, 25

21. Inference for Sampling
The purpose of a sample is to give us information about a larger population. The process of drawing conclusions about a population on the basis of sample data is called inference.

22. The word “error” in the phrase “margin of error” does not mean that a mistake has been made. The margin of error compensates for the variability that results from taking a random sample from a population. It does not account for a mistake made during the data collection process.

23. Sample Surveys: What Can Go Wrong?
Most sample surveys are affected by errors in addition to sampling variability.
Good sampling technique includes the art of reducing all sources of error.
There are FOUR types of errors we need to worry about.

24. Example UNDERCOVERAGE
Each of the following sampling methods has a similar underlying problem.
UNDERCOVERAGE

25. Example
In 1988, Shere Hite published a book entitled Women and Love: A Cultural Revolution in Progress, in which she claimed that 70% of women who have been married five years or more are having extramarital affairs.
NONRESPONSE

26. Example
Think about the reason in the following questions why a person might give an incorrect response. What aspects of the surveying process can influence a person’s answer?
RESPONSE BIAS

27. Example WORDING OF QUESTIONS
A badly worded question seemed to indicate that nearly one-third of all Americans had some doubt that the Holocaust occurred. Here was the question that the Roper polling organization asked in 1992: “Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?” Only 65% said it was impossible that it never happened. But, the wording of the question included a double-negative, making it difficult to understand. When the question was revised, the percentage who were certain that the Holocaust occurred dramatically increased. See holocaust-is-corrected.html for more details.
WORDING OF QUESTIONS

28. Sampling Error
Non-Sampling Error

29. Homework
Page 232 #27, 29, 31, 33, 35