SURVEY RESEARCH

Types of Surveys Telephone Face-to-face Mail Internet (web/ ) Administered Self-administered

Key Concerns/Issues Sampling issues Response rate Clarity of concepts/constructs/questions Non-attitudes (door-stop answers) social desirability bias

Guidelines for Asking Questions Respondents must be willing to answer. Questions should be relevant. Short items are best. Avoid negative items. Avoid biased items and terms.

Guidelines for Questionnaire Construction One question per line. Use contingency questions when necessary. Format matrix questions so they are easily answered.

Guidelines for Questionnaire Construction Be aware of issues with ordering items. Include instructions for the questionnaire. Pretest all or part of the questionnaire.

Question/Response Wording Likert Scales Thermometer ratings Agree/Disagree statements. Ordered responses. Open responses. Check all that apply.

–open-ended: allows subjects more leeway, flexibility –“What is your primary ethnic/cultural background?” –requires a content analysis of responses –close-ended ties respondents’ hands somewhat –easy to code the data –always include an “other_____________” category

Phrasing questions Avoid evaluative language –Phrasing of questions should not imply approval, disapproval –Follow-ups should not suggest surprise, liking, disliking, etc. Bad example: Do you think the Democratic dominated Congress should lift the harsh restrictions on stem cell research? Bad example: Despite its poor track record in crisis intervention, do you think the United Nations should intervene in Darfur, Sudan?

Acceptable Response Rates 50% - adequate for analysis and reporting 60% - good 70% - very good This is a rule of thumb that may change depending on time period, method, and the book you are looking at

11 Other Issues Pre-testing Follow-up mailings or telephone calls Training of interviewers CATI Legal issues –Minors –Autodialing of cell phone numbers

Guidelines for Survey Interviewing Dress in a similar manner to the people who will be interviewed. Study and become familiar with the questionnaire. Follow question wording exactly. Record responses exactly. Probe for responses when necessary.

additional considerations gender of researcher/respondent can influence responses ethnicity of researcher/respondent can influence responses medium can influence responses –face-to-face interview, telephone survey, mail survey –anonymous versus non-anonymous questionnaire fatigue: minimize length of surveys

Strengths of Survey Research Useful in describing the characteristics of a large population. Make large samples feasible. Flexible - many questions can be asked on a given topic.

Weaknesses of Survey Research Can seldom deal with the context of social life. Inflexible in some ways. Subject to artificiality. Weak on validity.

Survey Problems Literary Digest presidential survey of 1936; post cards. Over 2 million respondents. Predicted Alf Landon over FDR Survey predicted the outcomes of the 1916, 1920, 1924, 1928, and 1932 What went wrong? –Based on telephone and automobile registration –Response rate

Survey Problems Hite Sexuality Survey – 70% of women married 5 years or more are having sex outside of marriage (4,500 completed surveys out of 100,000). Recorded phone surveys. Push & Entertainment polling: political telemarketing masquerading as a poll

Bush campaign in South Carolina asked the following: "John McCain calls the campaign finance system corrupt, but as chairman of the Senate Commerce Committee, he raises money and travels on the private jets of corporations with legislative proposals before his committee. In view of this, are you much more likely to vote for him, somewhat more likely to vote for him, somewhat more likely to vote against him or much more likely to vote against him?"