Composing Questions

Goals for Effective Survey Questions Focused Concise Clear

Source of Bias and Error Instrumentation The way the questions are written, asked, or set up Response Bias The way the respondent chooses to answer the question(s) asked

Ways to Avoid Bias Basic ‘core’ vocabulary should be used Avoid jargon, technical language, abbreviation Are you cognizant of the variables influencing your decision? Vs. What variables influenced your decision?

Ways to Avoid Bias Stay in an “active” versus passive voice Avoid ‘leading’ questions Don’t you believe SUVs are good for off-road driving? vs. Do you believe SUVs are good vehicles for off-road driving? Even instructions can be leading: Consider “Indicate your agreement to the following questions”

Ways to Avoid Bias Avoid double barreled questions What do you think about the image and advertising for the Pontiac Aztek? Avoid ambiguity, be specific as necessary How would you categorize your public television usage? Sometimes, often, regular, irregular, infrequent Responses might be re-framed with time boundaries that define the researchers perceptions of each descriptor.

Ways to Avoid Bias Avoid making assumptions about the respondent’s state of mind Explain what you like about PBS… vs. What, if anything, do you like about PBS?

Ways to Avoid Bias Loaded questions Loaded questions can also generate Increases the possibility for responses that makes self look more socially desirable Loaded questions can also generate Acquiescence Yea-/Nay saying Threat defense responses Hostility responses Extremity responses

Ways to Avoid Bias Loaded words (over-dramatized, inflammatory language) To what extent do you believe global warming affects the planet’s environment? Vs. To what extent do you believe climate change affects the planet’s environment?

Ways to Avoid Bias Burdensome Questions (overtaxing to the memory) How often have you gone to a sporting event since the last Olympic Games? How often did you watch sports on ESPN last week?

Ways to Avoid Bias Avoid negatively phased questions Do you agree that it is not the job of the government to make decisions about social security? Vs. To what extent do you agree or disagree with the idea that the government should make decisions about social security?

Ways to Avoid Bias Order bias Responses influenced by the order of question presentation Solution Funnel questions - Asking brand specific questions before general questions

Ways to Avoid Bias Filter [or qualifying] questions Maximizes the chance of asking appropriate questions for the respondent in question Which Bank of America location has the best customer service? Vs. Are you familiar with Bank of America’s customer service?

Ways to Avoid Bias Alternatives should be exhaustive Options should avoid response bias Response alternatives should be appropriate [typical] for the behavior, attitude in question