Surveys and Sampling

Midpoint/Don’t Know Midpoint- allows for neutral response Advantage- might be more accurate Advantage- might be more accurate Disadvantage- too easy? Disadvantage- too easy? Don’t know Advantage- cuts down on guessing, eliminates false attitudes Advantage- cuts down on guessing, eliminates false attitudes Disadvantages- missing data, not used equally by all groups Disadvantages- missing data, not used equally by all groups

Randomized Response Technique Good for getting more truthful answers on controversial issues. Randomly assign people to answer sensitive or random question (interviewer does not know) Look at question overall Compare rate of yes answers to see if it differs from expected yes answers for random question

Group 1 Imagine the US is preparing for an outbreak of an unusual Asian Disease which is expected to kill 600. Two alternative plans to deal with it have been proposed. If A is chosen, 200 People will be saved If B is chosen, there is a 1/3 Chance that 600 people will be saved and a 2/3 chance that no people will be saved

Group 2 Imagine the US is preparing for an outbreak of an unusual Asian Disease which is expected to kill 600. Two alternative plans to deal with it have been proposed. If Program A is adopted, 400 people will die If Program B is adopted, there is a 1/3 probability that no people will die and a 2/3 probability that 600 people will die

Results If A is chosen, 200 People will be saved If B is chosen, there is a 1/3 Chance that 600 people will be saved and a 2/3 chance that no people will be saved If Program A is adopted, 400 people will die If Program B is adopted, there is a 1/3 probability that no people will die and a 2/3 probability that 600 people will die

Equivalence frames A and B are numerically equivalent across conditions When framed in terms of death, people take risk (group 2) When Framed in terms of saving, people choose sure thing Take home point: Little differences in wording can make a big difference

Bad Questions Ambiguous Double Barreled Leading/Loaded Questions

Ambiguous Questions What is your income? How often do you participate in community events? Always Always Sometimes Sometimes Rarely Rarely Never Never

Double Barreled Questions How much to do agree with the following statement: There is too much violent crime. Citizens should be able to carry handguns for personal protection. Irish, Italian, Jewish, and other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.

Leading Questions Do you believe that the government should take action to stop the flood of illegal immigrants coming into our country? Do you intend to vote for candidate A who will bring you peace and prosperity, or candidate B who will bring you war and crippling taxes?

Challenge- What Wording Some cases clearly wrong Other’s less clear cut 75% agree with the statement: The federal government should spend more money on aid to the poor 75% oppose the statement: The federal government should spend more money on welfare.

Formatting a Survey Order Matters Generally, put open ended questions first Put sensitive measures first Randomize where possible

Challenges to Administration Refusals Non- completion Interceding Events Quality of interviewer Race/Gender of Interviewer

Sampling Can’t talk to everybody Select some members of population of interest If sample is “representative” can generalize findings

Sampling Bigger is generally preferable to smaller Quality trumps quantity Margin of Error 4000 ± ± ± ± ± ± ± 11

Sampling/Margin of Error Poll shows prefer Bush Sample of 1000 ± 4 Could be Could be 57-43

Sampling Errors Literary Digest 1936 Poll Draw on Approx 4 million respondents Predict Landon Victory What went wrong? Bad sample Auto Registrations Phones Readers of Literary magazine