2. Copyright © 2009 Pearson Education, Inc. Chapter 12 Sample Surveys

3. Copyright © 2009 Pearson Education, Inc. Slide 1- 3 Objectives: Classify samples, populations, parameters and statistics. Identify and explain the different sampling techniques: census, simple random samples, stratified, cluster, systematic, convenience and voluntary response. Recognize different types of bias.

4. Copyright © 2009 Pearson Education, Inc. 4 Population vs. Sample For a given statistical inquiry: The population consists of all items of interest (people, places, firms, etc.) A sample is a (hopefully representative and random) subset of the population The sampling frame is the set of items belonging to the population from which samples will be drawn A numerical value/characteristic of a population is called a parameter. These are usually unknown. A numerical value/characteristic of a sample is called a statistic

5. Copyright © 2009 Pearson Education, Inc. Slide 1- 5 Notation We typically use Greek letters to denote parameters and Latin letters to denote statistics.

6. Copyright © 2009 Pearson Education, Inc. Slide 1- 6 Sample Size How large a random sample do we need for the sample to be reasonably representative of the population? It’s the size of the sample, not the size of the population, that makes the difference in sampling. Exception: If the population is small enough and the sample is more than 10% of the whole population, the population size can matter. The fraction of the population that you’ve sampled doesn’t matter. It’s the sample size itself that’s important. Think about sampling a pot of soup for one versus for a banquet, do you need a bigger spoonful to get an idea of how the soup tastes? No!

7. Copyright © 2009 Pearson Education, Inc. Slide 1- 7 Simple Random Samples We draw samples because we can’t work with the entire population. We need to be sure that the statistics we compute from the sample reflect the corresponding parameters accurately. A sample that does this is said to be representative.

8. Copyright © 2009 Pearson Education, Inc. Slide 1- 8 Sampling Techniques Census Simple random samples Stratified (Random) Sampling Cluster sampling Systematic Sampling Convenience Sampling Voluntary response sampling

9. Copyright © 2009 Pearson Education, Inc. Types of Bias Voluntary Response Bias Undercoverage Nonresponse Bias Response Bias Slide 1- 9

10. Copyright © 2009 Pearson Education, Inc. Practice Gallup Poll The Gallup Poll publishes a new survey every day. Scroll to the end of the website and you’ll find a statement that includes words such as these: Results are based on telephone interviews with 1,008 national adults, aged 18 and older, conducted April 2-5, 2007… In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls a) For this survey identify the population of interest b) Gallup performs its surveys by phoning numbers generated at random by a computer program. What is the sampling frame? c) What problems, if any, would you be concerned about in matching the sampling frame with the population Slide 1- 10

11. Copyright © 2009 Pearson Education, Inc. A company packaging snack foods maintains quality control by randomly selecting 10 cases from each day’s production and weighing the bags. Then they open one back from each case and inspect the contents What is the population? The population parameter of interest? The sampling frame? The sample? The sampling method? Any potential sources of bias or problems generalizing to the population of interest Slide 1- 11

12. Copyright © 2009 Pearson Education, Inc. Parent Opinion In a large city school system with 20 elementary schools, the school board is considering the adoption of a new policy that would require elementary students to pass a test in order to be promoted to the next grade. The PTA wants to find out whether parents agree with this plan. For each idea about how to gather data indicate what kind of sampling strategy is involved and what (if any) biases might result a) Put an ad in the newspaper asking people to log their opinions on the PTA website b) Randomly select one of the elementary schools and contact every parent by phone c) Send a survey home with every student and ask parents to fill it out and return it the next day. d) Randomly select 20 parents form each elementary school. Send them a survey and follow up with a phone call if they do not return the survey within a week Slide 1- 12

13. Copyright © 2009 Pearson Education, Inc. Slide 1- 13 Parent opinion (continued) e) Run a poll on the local TV news, asking people to dial one of two phone numbers to indicate whether they favor or oppose the plan f) Hold a PTA meeting at each of the 20 elementary schools, and tally the opinions expressed by those who attend the meetings g) Randomly select one class at each elementary school and contact each of those parents h) Go through the district’s enrollment records, selecting every 40 th parent. PTA volunteers will go to those homes to interview the people chosen

14. Copyright © 2009 Pearson Education, Inc. Additional Information Slide 1- 14

15. Copyright © 2009 Pearson Education, Inc. 15 Random Sampling: Random & Simple Random Samples In a random sample members from the population are selected in such a way that each individual member has an equal chance of being selected A simple random sample of n subjects is selected in such a way that every possible sample of size n has the same chance of being chosen

16. Copyright © 2009 Pearson Education, Inc. Slide 1- 16 Simple Random Samples (cont.) To select a sample at random, we first need to define where the sample will come from. The sampling frame is a list of individuals from which the sample is drawn. Once we have our sampling frame, the easiest way to choose an SRS is with random numbers.

17. Copyright © 2009 Pearson Education, Inc. Slide 1- 17 Stratified Sampling (cont.) Designs used to sample from large populations are often more complicated than simple random samples. Sometimes the population is first sliced into homogeneous groups, called strata, before the sample is selected. Then simple random sampling is used within each stratum before the results are combined. This common sampling design is called stratified random sampling.

18. Copyright © 2009 Pearson Education, Inc. Slide 1- 18 Stratified Sampling (cont.) Stratified random sampling can reduce bias. Stratifying can also reduce the variability of our results. When we restrict by strata, additional samples are more like one another, so statistics calculated for the sampled values will vary less from one sample to another.

19. Copyright © 2009 Pearson Education, Inc. Slide 1- 19 Cluster and Multistage Sampling Sometimes stratifying isn’t practical and simple random sampling is difficult. Splitting the population into similar parts or clusters can make sampling more practical. Then we could select one or a few clusters at random and perform a census within each of them. This sampling design is called cluster sampling. If each cluster fairly represents the full population, cluster sampling will give us an unbiased sample.

20. Copyright © 2009 Pearson Education, Inc. Slide 1- 20 Cluster and Multistage Sampling (cont.) Cluster sampling is not the same as stratified sampling. We stratify to ensure that our sample represents different groups in the population, and sample randomly within each stratum. Strata are homogeneous, but differ from one another. Clusters are more or less alike, each heterogeneous and resembling the overall population. We select clusters to make sampling more practical or affordable.

21. Copyright © 2009 Pearson Education, Inc. Slide 1- 21 Cluster and Multistage Sampling (cont.) Sometimes we use a variety of sampling methods together. Sampling schemes that combine several methods are called multistage samples. Most surveys conducted by professional polling organizations use some combination of stratified and cluster sampling as well as simple random sampling.

22. Copyright © 2009 Pearson Education, Inc. 22 Random Sampling: Systematic Samples Sometimes we draw a sample by selecting individuals systematically. For example, you might survey every 10th person on an alphabetical list of students. To make it random, you must still start the systematic selection from a randomly selected individual. In a systematic sample every kth item of the sampling frame is selected, starting from a first element “k” is referred to as the periodicity When there is no reason to believe that the order of the list could be associated in any way with the responses sought, systematic sampling can give a representative sample. Problems can occur if patterns in the sampling frame recur at periodicity k

23. Copyright © 2009 Pearson Education, Inc. Slide 1- 23 Systematic Samples (cont.) Systematic sampling can be much less expensive than true random sampling. When you use a systematic sample, you need to justify the assumption that the systematic method is not associated with any of the measured variables.

24. Copyright © 2009 Pearson Education, Inc. Slide 1- 24 Ask specific rather than general questions. Ask for quantitative results when possible. Be careful in phrasing questions. A respondent may not understand the question or may understand the question differently than the way the researcher intended it. Even subtle differences in phrasing can make a difference. Be careful in phrasing answers. It’s often a better idea to offer choices rather than inviting a free response. The Valid Survey (cont.)

25. Copyright © 2009 Pearson Education, Inc. Slide 1- 25 What Can Go Wrong?—or, How to Sample Badly Sample Badly with Volunteers: In a voluntary response sample, a large group of individuals is invited to respond, and all who do respond are counted. Voluntary response samples are almost always biased, and so conclusions drawn from them are almost always wrong. Voluntary response samples are often biased toward those with strong opinions or those who are strongly motivated. Since the sample is not representative, the resulting voluntary response bias invalidates the survey.

26. Copyright © 2009 Pearson Education, Inc. Slide 1- 26 What Can Go Wrong?—or, How to Sample Badly (cont.) Sample Badly, but Conveniently: In convenience sampling, we simply include the individuals who are convenient. Unfortunately, this group may not be representative of the population. Convenience sampling is not only a problem for students or other beginning samplers. In fact, it is a widespread problem in the business world—the easiest people for a company to sample are its own customers.

27. Copyright © 2009 Pearson Education, Inc. Slide 1- 27 What Else Can Go Wrong? Watch out for nonrespondents. A common and serious potential source of bias for most surveys is nonresponse bias. No survey succeeds in getting responses from everyone. The problem is that those who don’t respond may differ from those who do. And they may differ on just the variables we care about.

28. Copyright © 2009 Pearson Education, Inc. Slide 1- 28 What Else Can Go Wrong? (cont.) Work hard to avoid influencing responses. Response bias refers to anything in the survey design that influences the responses. Make sure the question wording is neutral. Many surveys, especially those conducted by special-interest groups, present one side of an issue before the question itself.

29. Copyright © 2009 Pearson Education, Inc. Slide 1- 29 What have we learned? A representative sample can offer us important insights about populations. It’s the size of the sample, not its fraction of the larger population, that determines the precision of the statistics it yields. There are several ways to draw samples, all based on the power of randomness to make them representative of the population of interest: Simple Random Sample, Stratified Sample, Cluster Sample, Systematic Sample, Multistage Sample

30. Copyright © 2009 Pearson Education, Inc. Slide 1- 30 What have we learned? (cont.) Bias can destroy our ability to gain insights from our sample: Nonresponse bias can arise when sampled individuals will not or cannot respond. Response bias arises when respondents’ answers might be affected by external influences, such as question wording or interviewer behavior.

31. Copyright © 2009 Pearson Education, Inc. Slide 1- 31 What have we learned? (cont.) Bias can also arise from poor sampling methods: Voluntary response samples are almost always biased and should be avoided and distrusted. Convenience samples are likely to be flawed for similar reasons. Even with a reasonable design, sample frames may not be representative. Undercoverage occurs when individuals from a subgroup of the population are selected less often than they should be.