MSS 905 Methods of Missiological Research Survey Research

Survey Research The most widely used data gathering technique in the social sciences Can be well made and appropriate, but can also be poorly constructed and used inappropriately Produces information that is inherently statistical in nature Appropriate for self-reported beliefs or behaviors (see box 10.1, p. 273); not “why?” questions

Types of Questionnaires See Adler and Clark (2008), Chapter 9, p. 232-239

Good Question Writing Good surveys are smooth, non-threatening, unambiguous, clear Question writing is more an art than just a science

10 Things to Avoid Avoid jargon, slang and abbreviations Use the language of TV and newspapers Avoid ambiguity, confusion and vagueness “What is your income?” Monthly? Annual? Before or after taxes? From salary or all sources? Accurate response categories

10 Things to Avoid Avoid emotional language and prestige bias Some words carry emotional baggage, “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving citizens?” Prestige bias: associating a statement with a person or group that enjoy prestige, “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”

10 Things to Avoid Avoid double-barreled questions “Do you support the president’s policy regarding Uzbekistan?” (very few have heard of Uzbekistan and will respond on the basis of their attitude toward the president) Avoid double-barreled questions Two or more questions in the same question “Do you have pension and health insurance benefits at your job?”

10 Things to Avoid Avoid leading (“loaded”) questions Suggesting the answer in the way you pose the question “You go to church, don’t you?” “Should the church spend even more tithing income on interior decorating?” “Should the church allocate more of tithing income to save lost souls in our community?”

10 Things to Avoid Avoid asking questions that are beyond people’s capabilities People can’t always recall past details “How did you feel about your Sunday School teacher when you were six years old?” Asking church visitors, “What percentage of our tithing income should we be investing in our building fund?”

10 Things to Avoid Avoid false premises You cannot begin with a premise which is debatable, and then ask someone a question about it “Our sanctuary is already open too early on a Sunday morning. Do you want it to be open an hour earlier or an hour later?” Avoid asking about future, distant intentions “If we opened a branch church in the north, would you attend it?” Stay in the present

10 Things to Avoid Avoid double negatives If you disagree with this question, “Students should not be required to take a comprehensive exam at the end of the course”, then you don’t agree they should not Avoid overlapping or unbalanced response categories Categories should be mutually exclusive “Are you working or unemployed?”

10 Things to Avoid Unbalanced: “Which of the Democratic candidates do you support, Hillary Clinton or one of the others?”

Some Tips Use a Likert-scale to gain responses I prefer a 5-pointer “Anchor” the two extreme positions (1 and 5) with clear opposites “To a great extent” vs “Not at all” “Strongly agree” vs “Strongly disagree” In the question, make a clear statement to which the person needs to respond Mix the response directions

Getting Good Answers Be aware of social desirability bias Respondents give the socially desirable response (politically, ethically, etc.) Avoid questions that test knowledge Large majority cannot answer elementary geography questions or id important political documents Respondents do not want to appear ignorant

% Very Topic Uneasy

Getting Good Answers Make use of contingency questions “if – then” Saves time of those who do not qualify Open-ended versus closed questions Open is unstructured, respondents can give any answer they wish Closed restricts the options of the respondent Both have benefits and disadvantages (box 10.5)

Types of Surveys: Mail and Self-administered Cheap No need for presence of researcher Can be completed when convenient Anonymity Avoid interviewer bias Low response rates Cannot control conditions when questionnaire is completed No one to clarify Someone else may complete Incomplete questions Ill-suited for illiterate

Types of Surveys: Telephone Interviews 95% can be reached Can be done long distance as well High response rates Cheaper than face-to-face, same benefits Computer-assisted telephone interview (CATI) & Interactive Voice Response (IVR) Higher cost Limited interview length Not all have telephones Calling at inconvenient times Open-ended questions are difficult to use

Types of Surveys: Face-to Face Interviews Highest response rates Permit longest format Interviewer can observe non-verbals Can use visual aids All types of questions, complex issues, extensive probing possibilities Highest cost Training, travel, supervision, personnel deployment Interviewer bias potential

Types of Surveys: Web Surveys By 2004 75% of US homes connected to Internet Very fast, inexpensive Flexible: visual or audio inputs Cuts out the “data entry” step (and costs) International potential Could exclude older, less-educated, lower SES and rural people Sampling problems are formidable Privacy and security Design issues: question format, drop-down menus, progress indicators, move back