Sampling: Surveys and How to Ask Questions Chapter 4.

Slides:



Advertisements
Similar presentations
Collecting Data Sensibly
Advertisements

Sampling: Surveys and How to Ask Questions
Sampling.
Sampling Methods.
Chapter 7: Data for Decisions Lesson Plan
Copyright ©2006 Brooks/Cole, a division of Thomson Learning, Inc. Sampling: Surveys and How to Ask Questions Chapter 4.
Drawing Samples in “Observational Studies” Sample vs. the Population How to Draw a Random Sample What Determines the “Margin of Error” of a Poll?
Copyright ©2011 Brooks/Cole, Cengage Learning Sampling: Surveys and How to Ask Questions Chapter 5 1.
Chapter 4 How to get the Data Part1 n In the first 3 lectures of this course we spoke at length about what care we should take in conducting a study ourselves.
CHAPTER twelve Basic Sampling Issues Copyright © 2002
Chapter 12 Sample Surveys
Section 5.1. Observational Study vs. Experiment  In an observational study, we observe individuals and measure variables of interest but do not attempt.
Copyright ©2005 Brooks/Cole, a division of Thomson Learning, Inc. How to Get a Good Sample Chapter 4.
Copyright © 2011 Pearson Education, Inc. Samples and Surveys Chapter 13.
5.1 Designing Samples.  Differentiate between an observational study and an experiment  Learn different types of sampling techniques  Use a random.
Chapter 5 Section 3 Part 1.  Often when we hear of a sample we do not know the truth behind the sampling process.  The whole truth about opinion polls.
Producing Data: Sampling BPS - 5th Ed.Chapter 81.
CHAPTER 8 Producing Data: Sampling BPS - 5TH ED.CHAPTER 8 1.
Statistics The science of collecting, analyzing, and interpreting data. Planning A Study Using The Statistical Problem Solving Process: Ask a question.
Copyright © 2009 Pearson Education, Inc. Publishing as Longman. The 1936 Literary Digest Presidential Election Poll Case Study: Special Topic Lecture Chapter.
 Sampling Design Unit 5. Do frog fairy tale p.89 Do frog fairy tale p.89.
Sample Surveys.  The first idea is to draw a sample. ◦ We’d like to know about an entire population of individuals, but examining all of them is usually.
Chapter 12 Designing Good Samples. Doubting the Holocaust? An opinion poll conducted in 1992 for the American Jewish Committee asked: Does it seem possible.
Measurements, Mistakes and Misunderstandings in Sample Surveys Lecture 1.
Population We are almost always interested in knowledge about a population. We would have little interest in samples if we could always ask everyone.
Copyright ©2005 Brooks/Cole, a division of Thomson Learning, Inc. How to Get a Good Sample Chapter 4.
DATA COLLECTION METHODS Sampling
Sampling Design Notes Pre-College Math.
Chapter 7: Data for Decisions Lesson Plan Sampling Bad Sampling Methods Simple Random Samples Cautions About Sample Surveys Experiments Thinking About.
Section 5.1 Designing Samples Malboeuf AP Statistics, Section 5.1, Part 1 3 Observational vs. Experiment An observational study observes individuals.
Chapter 41 Sample Surveys in the Real World. Chapter 42 Thought Question 1 (from Seeing Through Statistics, 2nd Edition, by Jessica M. Utts, p. 14) Nicotine.
Other Probability Sampling Methods
MAT 1000 Mathematics in Today's World. Last Time 1.Two types of observational study 2.Three methods for choosing a sample.
Lecture 4. Sampling is the process of selecting a small number of elements from a larger defined target group of elements such that the information gathered.
BPS - 5th Ed. Chapter 81 Producing Data: Sampling.
Chapter 15 Sampling and Sample Size Winston Jackson and Norine Verberg Methods: Doing Social Research, 4e.
DATA COLLECTION METHODS SAMPLING 1. Class Objective After this class, you will be able to - Use Simple Random Sampling (SRS) to collect data 2.
Section 5.1 Designing Samples AP Statistics
AP STATISTICS LESSON AP STATISTICS LESSON DESIGNING DATA.
AP STATISTICS Section 5.1 Designing Samples. Objective: To be able to identify and use different sampling techniques. Observational Study: individuals.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc.. Chap 7-1 Chapter 7 Sampling Distributions Basic Business Statistics.
Section 2.2. Census – obtaining information from an entire population Sample – obtaining information from a selected part of the population Bias – the.
CONFIDENCE STATEMENT MARGIN OF ERROR CONFIDENCE INTERVAL 1.
SECTION 4.1. INFERENCE The purpose of a sample is to give us information about a larger population. The process of drawing conclusions about a population.
I can identify the difference between the population and a sample I can name and describe sampling designs I can name and describe types of bias I can.
 An observational study observes individuals and measures variable of interest but does not attempt to influence the responses.  Often fails due to.
Chapter 3 Surveys and Sampling © 2010 Pearson Education 1.
Chapter 3 Producing Data. Observational study: observes individuals and measures variables of interest but does not attempt to influence the responses.
5.3: SAMPLING. Errors in Sampling Sampling Errors- Errors caused by the act of taking a sample. Makes sample results inaccurate. Random Sampling Error.
Chapter 7 Data for Decisions. Population vs Sample A Population in a statistical study is the entire group of individuals about which we want information.
Chapter 7 Introduction to Sampling Distributions Business Statistics: QMIS 220, by Dr. M. Zainal.
Chapter 5 Sampling and Surveys. Section 5.3 Sample Surveys in the Real World.
Designing Studies In order to produce data that will truly answer the questions about a large group, the way a study is designed is important. 1)Decide.
Copyright © 2014, 2011 Pearson Education, Inc. 1 Chapter 13 Samples and Surveys.
Copyright ©2011 Brooks/Cole, Cengage Learning Sampling: Surveys and How to Ask Questions 1.
Chapter 11 Sample Surveys. How do we gather data? Surveys Opinion polls Interviews Studies –Observational –Retrospective (past) –Prospective (future)
Sampling Chapter 5. Introduction Sampling The process of drawing a number of individual cases from a larger population A way to learn about a larger population.
Introduction/ Section 5.1 Designing Samples.  We know how to describe data in various ways ◦ Visually, Numerically, etc  Now, we’ll focus on producing.
Ten percent of U. S. households contain 5 or more people
MATH Section 6.1. Sampling: Terms: Population – each element (or person) from the set of observations that can be made Sample – a subset of the.
Chapter 5: Producing Data 5.1 – Designing Samples "An approximate answer to the right question is worth a good deal more than the exact answer to an approximate.
Stat 100 Jan. 21 Read Chapter 4, Try problems 1,2, 5,7,9,14,15,17
Essential Statistics Producing Data: Sampling
Chapter 4 Sampling Design.
Essential Statistics Producing Data: Sampling
MATH 2311 Section 6.1.
Chapter 5: Producing Data
Sample Surveys Idea 1: Examine a part of the whole.
Basic Practice of Statistics - 5th Edition Producing Data: Sampling
MATH 2311 Section 6.1.
Presentation transcript:

Sampling: Surveys and How to Ask Questions Chapter 4

2 The Beauty of Sampling Sample Survey: a subgroup of a large population questioned on set of topics. Special type of observational study. Less costly and less time than a census. With proper methods, a sample of 1500 can almost certainly gauge the percentage in the entire population who have a certain trait or opinion to within 3%.

3 The Margin of Error: The Accuracy of Sample Surveys The sample proportion and the population proportion with a certain trait or opinion differ by less than the margin of error in at least 95% of all random samples. Conservative margin of error = Add and subtract the margin of error to create an approximate 95% confidence interval.

4 The Margin of Error: The Accuracy of Sample Surveys A confidence interval is an interval of values computed from sample data that is likely to include the true population value.

5 Example: The Importance of Religion for Adult Americans Poll of n = 1003 adult Americans: “How important would you say religion is in your own life?” Very important65% Fairly important23% Not very important12% No opinion0% Conservative margin of error is 3%: Approx. 95% confidence interval for the percent of all adult Americans who say religion is very important: 65%  3% or 62% to 68%

6 Interpreting Confidence Interval The interval 62% to 68% may or may not capture the percent of adult Americans who considered religion to be very important in their lives. But, in the long run this procedure will produce intervals that capture the unknown population values about 95% of the time => called the 95% confidence level.

7 Confidence Level Confidence Level describes the chance (probability) that an interval actually contains the true population value Most of the time (quantified by the confidence level) intervals computed in this way will capture the truth about the population, but occasionally they will not. In any given instance, the interval either captures the truth or it does not, but we will never know which is the case. Therefore, our confidence is the procedure (it works most of the time) and the confidence level (or level of confidence) is the percent of time we expect it to work.

8 Advantages of a Sample Survey over a Census Sometimes a Census Isn’t Possible when measurements destroy units Speed especially if population is large Accuracy devote resources to getting accurate sample results

9 Bias: How Surveys Can Go Wrong Results based on a survey are biased if method used to obtain those results would consistently produce values that are either too high or too low. Selection bias occurs if method for selecting participants produces sample that does not represent the population of interest. Nonresponse bias occurs when a representative sample is chosen but a subset cannot be contacted or doesn’t respond. Response bias occurs when participants respond differently from how they truly feel.

10 Simple Random Sampling and Randomization Probability Sampling Plan: everyone in population has specified chance of making it into the sample. Simple Random Sample: every conceivable group of units of the required size has the same chance of being the selected sample.

11 Choosing a Simple Random Sample You Need: 1.List of the units in the population. 2.Source of random numbers. Portion of a Table of Random Digits:

12 Simple Random Sample of Students Class of 270 students. Want a simple random sample of 10 students. 1.Number the units: Students numbered 001 to Choose a starting point: Row 3, 2 nd column (10484…) 3.Read off consecutive numbers: (3-digit labels here) 104, 842, 461, 613, 466, 416, 180, 855, 118, 314, 577, 002, 896, … 4.If number corresponds to a label, select that unit. If not, skip it. Continue until desired sample size obtained.

13 Simple Random Sample of Students 5.Step 4 very inefficient. Can give each unit in population multiple labels. e.g. use 001 to 270 then 301 to 570, 601 to 870 so the second 3-digit number of 842 would correspond to unit with label 842 – 600 = 242. Using method in Step 4 selected units would be: 104, 180, 118, 002, etc. Using method in Step 5 selected units would be found more efficiently as: 104, 242, 161, 013, 166, 116, 180, 255, 118, 014.

14 Using a Table of Random Digits in a Randomized Experiment Randomization plays a key role in designing experiments to compare treatments. Completely randomized design = all units are randomly assigned to treatment conditions. Matched-pairs / Randomized Block design = randomize order treatments are assigned within pair/block.

15 Other Sampling Methods Not always practical to take a simple random sample, can be difficult to get a numbered list of all units. Example: College administration would like to survey a sample of students living in dormitories. Shaded squares show a simple random sample of 30 rooms.

16 Stratified Random Sampling Divide population of units into groups (called strata) and take a simple random sample from each of the strata. College survey: Two strata = undergrad and graduate dorms. Take a simple random sample of 15 rooms from each of the strata for a total of 30 rooms. Ideal: stratify so little variability in responses within each of the strata.

17 Cluster Sampling Divide population of units into groups (called clusters), take a random sample of clusters and measure only those items in these clusters. College survey: Each floor of each dorm is a cluster. Take a random sample of 5 floors and all rooms on those floors are surveyed. Advantage: need only a list of the clusters instead of a list of all individuals.

18 Systematic Sampling Order the population of units in some way, select one of the first k units at random and then every k th unit thereafter. College survey: Order list of rooms starting at top floor of 1 st undergrad dorm. Pick one of the first 11 rooms at random => room 3, then pick every 11 th room after that. Note: often a good alternative to random sampling but can lead to a biased sample.

19 Difficulties and Disasters in Sampling Using wrong sampling frame Not reaching individuals selected Self-selected sample Convenience/Haphazard sample Some problems occur even when a sampling plan has been well designed.

20 Using the Wrong Sampling Frame The sampling frame is the list of units from which the sample is selected. This list may or may not be the same as the list of all units in the desired “target” population. Example: using telephone directory to survey general population excludes those who move often, those with unlisted home numbers, and those who cannot afford a telephone. Solution: use random-digit dialing.

21 Not Reaching the Individuals Selected Failing to contact or measure the individuals who were selected in the sampling plan leads to nonresponse bias. Telephone surveys tend to reach more women. Some people are rarely home. Others screen calls or may refuse to answer. Quickie polls: almost impossible to get a random sample in one night.

22 Nonresponse or Volunteer Response The lower the response rate, the less the results can be generalized to the population as a whole. Response to survey is voluntary. Those who respond likely to have stronger opinions than those who don’t. Surveys often use reminders, follow up calls to decrease nonresponse rate. “In 1993 the GSS (General Social Survey) achieved its highest response rate ever, 82.4%. This is five percentage points higher than our average over the last four years.” GSS News, Sept 1993

23 Disasters in Sampling Responses from a self-selected group, convenience sample or haphazard sample rarely representative of any larger group. Example A Meaningless Poll “Do you support the President’s economic plan?” Results from TV quickie poll and proper study: Those dissatisfied more likely to respond to TV poll and it did not give the “not sure” option.

24 Example: The Infamous Literary Digest Poll of 1936 Election of 1936: Democratic incumbent Franklin D. Roosevelt and Republican Alf Landon Literary Digest Poll: Sent questionnaires to 10 million people from magazine subscriber lists, phone directories, car owners, who were more likely wealthy and unhappy with Roosevelt. Only 2.3 million responses for 23% response rate. Those with strong feelings, the Landon supporters wanting a change, were more likely to respond. (Incorrectly) Predicted a 3-to-2 victory for Landon.

25 How to Ask Survey Questions Deliberate bias: The wording of a question can deliberately bias the responses toward a desired answer. Unintentional bias: Questions can be worded such that the meaning is misinterpreted by a large percentage of the respondents. Desire to Please: Respondents have a desire to please the person who is asking the question. Tend to understate response to an undesirable social habit/opinion. Possible Sources of Response Bias in Surveys

26 Asking the Uninformed: People do not like to admit that they don’t know what you are talking about when you ask them a question. Unnecessary Complexity: If questions are to be understood, they must be kept simple. Some questions ask more than one question at once. Ordering of Questions: If one question requires respondents to think about something that they may not have otherwise considered, then the order in which questions are presented can change the results. Possible Sources of Response Bias in Surveys (cont)

27 Confidentiality and Anonymity: People will often answer questions differently based on the degree to which they believe they are anonymous. Easier to ensure confidentiality, promise not to release identifying information, than anonymity, researcher does not know the identity of the respondents. Possible Sources of Response Bias in Surveys (cont)

28 Be Sure You Understand What Was Measured: Words can have different meanings. Important to get a precise definition of what was actually asked or measured. E.g. Who is really unemployed? Some Concepts Are Hard to Precisely Define: E.g. How to measure intelligence? Measuring Attitudes and Emotions: E.g. How to measure self-esteem and happiness?

29 Open or Closed Questions: Should Choices Be Given? Open question = respondents allowed to answer in own words. Closed question = given list of alternatives, usually offer choice of “other” and can fill in blank. If closed are preferred, they should first be presented as open questions (in a pilot survey) for establishing list of choices. Results can be difficult to summarize with open questions.

30 Example: No Opinion of Your Own? Let Politics Decide 1978 Poll, Cincinnati, Ohio: people asked whether they “favored or opposed repealing the 1975 Public Affairs Act.” No such act, about one-third expressed opinion Washington Post Poll: 1000 randomly selected people asked “Some people say the 1975 Public Affairs Act should be repealed. Do you agree or disagree that it should be repealed?” 43% expressed opinion, 24% agreeing should be repealed.

31 Example: No Opinion of Your Own? Let Politics Decide (cont) Second 1995 Washington Post Poll: polled two separate groups of 500 randomly selected adults. Group 1: “President Clinton [a Democrat] said that the 1975 Public Affairs Act should be repealed. Do you agree or disagree?” Of those expressing an opinion: 36% of the Democrats agreed should be repealed, 16% of the Republicans agreed should be repealed. Group 2: “The Republicans in Congress said that the 1975 Public Affairs Act should be repealed. Do you agree or disagree?” Of those expressing an opinion: 36% of the Republicans agreed should be repealed, 19% of the Democrats agreed should be repealed.