1. MSS 905 Methods of Missiological Research
Survey Research

2. Survey Research
The most widely used data gathering technique in the social sciences
Can be well made and appropriate, but can also be poorly constructed and used inappropriately
Produces information that is inherently statistical in nature
Appropriate for self-reported beliefs or behaviors (see box 10.1, p. 273); not “why?” questions

3. Types of Questionnaires
See Adler and Clark (2008), Chapter 9, p

4. Good Question Writing
Good surveys are smooth, non-threatening, unambiguous, clear
Question writing is more an art than just a science

5. 10 Things to Avoid Avoid jargon, slang and abbreviations
Use the language of TV and newspapers
Avoid ambiguity, confusion and vagueness
“What is your income?”
Monthly? Annual? Before or after taxes? From salary or all sources?
Accurate response categories

6. 10 Things to Avoid Avoid emotional language and prestige bias
Some words carry emotional baggage, “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving citizens?”
Prestige bias: associating a statement with a person or group that enjoy prestige, “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”

7. 10 Things to Avoid Avoid double-barreled questions
“Do you support the president’s policy regarding Uzbekistan?” (very few have heard of Uzbekistan and will respond on the basis of their attitude toward the president)
Avoid double-barreled questions
Two or more questions in the same question
“Do you have pension and health insurance benefits at your job?”

8. 10 Things to Avoid Avoid leading (“loaded”) questions
Suggesting the answer in the way you pose the question
“You go to church, don’t you?”
“Should the church spend even more tithing income on interior decorating?”
“Should the church allocate more of tithing income to save lost souls in our community?”

9. 10 Things to Avoid
Avoid asking questions that are beyond people’s capabilities
People can’t always recall past details
“How did you feel about your Sunday School teacher when you were six years old?”
Asking church visitors, “What percentage of our tithing income should we be investing in our building fund?”

10. 10 Things to Avoid Avoid false premises
You cannot begin with a premise which is debatable, and then ask someone a question about it
“Our sanctuary is already open too early on a Sunday morning. Do you want it to be open an hour earlier or an hour later?”
Avoid asking about future, distant intentions
“If we opened a branch church in the north, would you attend it?”
Stay in the present

11. 10 Things to Avoid Avoid double negatives
If you disagree with this question, “Students should not be required to take a comprehensive exam at the end of the course”, then you don’t agree they should not
Avoid overlapping or unbalanced response categories
Categories should be mutually exclusive
“Are you working or unemployed?”

12. 10 Things to Avoid
Unbalanced: “Which of the Democratic candidates do you support, Hillary Clinton or one of the others?”

14. Some Tips Use a Likert-scale to gain responses I prefer a 5-pointer
“Anchor” the two extreme positions (1 and 5) with clear opposites
“To a great extent” vs “Not at all”
“Strongly agree” vs “Strongly disagree”
In the question, make a clear statement to which the person needs to respond
Mix the response directions

15. Getting Good Answers Be aware of social desirability bias
Respondents give the socially desirable response (politically, ethically, etc.)
Avoid questions that test knowledge
Large majority cannot answer elementary geography questions or id important political documents
Respondents do not want to appear ignorant

16. % Very Topic Uneasy

17. Getting Good Answers Make use of contingency questions
“if – then”
Saves time of those who do not qualify
Open-ended versus closed questions
Open is unstructured, respondents can give any answer they wish
Closed restricts the options of the respondent
Both have benefits and disadvantages (box 10.5)

18. Types of Surveys: Mail and Self-administered
Cheap
No need for presence of researcher
Can be completed when convenient
Anonymity
Avoid interviewer bias
Low response rates
Cannot control conditions when questionnaire is completed
No one to clarify
Someone else may complete
Incomplete questions
Ill-suited for illiterate

19. Types of Surveys: Telephone Interviews
95% can be reached
Can be done long distance as well
High response rates
Cheaper than face-to-face, same benefits
Computer-assisted telephone interview (CATI) & Interactive Voice Response (IVR)
Higher cost
Limited interview length
Not all have telephones
Calling at inconvenient times
Open-ended questions are difficult to use

20. Types of Surveys: Face-to Face Interviews
Highest response rates
Permit longest format
Interviewer can observe non-verbals
Can use visual aids
All types of questions, complex issues, extensive probing possibilities
Highest cost
Training, travel, supervision, personnel deployment
Interviewer bias potential

21. Types of Surveys: Web Surveys
By % of US homes connected to Internet
Very fast, inexpensive
Flexible: visual or audio inputs
Cuts out the “data entry” step (and costs)
International potential
Could exclude older, less-educated, lower SES and rural people
Sampling problems are formidable
Privacy and security
Design issues: question format, drop-down menus, progress indicators, move back