Guidelines for Writing Surveys A number of resources available ◦ Survey Research by Backstrom and Hursh- Casar ◦ Survey Research Methods by Fowler

Survey Terms closed question: A survey question that offers response categories. context effects: The effects that prior questions have on subsequent responses. open question: A survey question that does not offer response categories. recency effect: Over reporting events in the most recent portion of a reference period, or a tendency to select the last-presented response alternative in a list. response effects: The effects of variations in question wording, order, instructions, format, etc. on responses. screening questions: Questions designed to identify specific conditions or events.

General Guidelines Keep the survey as short as possible Ask only what you need The more white space on the survey instrument the better Typically, surveys begin with demographic items then get into specific research areas Save the more “delicate” questions for the end of the survey

Consistency Instructions should be provided for the survey and unique types of items Provide definitions if necessary The survey instrument should have like types of items grouped together Keep scales in the same direction

Closed Response Questions Agree–disagree Forced choice Ordered response categories or scales

Question Order Question order changes the context in which a particular question is asked. Prior questions can influence answers to subsequent questions through several mechanisms. ◦ For example, an obscure “monetary control bill” was more likely to be supported when a question about it appeared after questions on inflation, which presumably led respondents to infer the bill was an anti-inflation measure.

Terminology “Avoid ambiguity” is a truism of questionnaire design. ◦ Language is inherently ambiguous, and seemingly simple words may have multiple meanings. ◦ Research by Belson and others demonstrates that ordinary words and phrases, such as “you,” “children,” and “work”are interpreted very differently by different respondents.

Terminology In a national sample, respondents were randomly assigned to be asked one of two questions: 1. “Do you think the United States should allow public speeches against democracy?” 2. “Do you think the United States should forbid public speeches against democracy?” ◦ Support for free speech is greater–by more than 20 percentage points–if respondents answer question 2 rather than question 1. ◦ That is, more people answer “no” to question 2 than answer “yes” to question 1; “not allowing” speeches is not the same as “forbidding” them, even though it might seem to be the same.

Leading Questions A leading question is simply one that, either by its form or content, suggests to the subject what answer is desired or leads him/her to the desired answer (Loftus & Palmer, 1974). How fast were the cars going when they hit each other?“ ◦ In this question the word hit would be replaced with other words such as smashed, collided, bumped, and contacted. ◦ When the word smashed was used, results showed that the subject estimated the car to be traveling at a faster speed then when the word bumped was used.

Don’t Know Giving “don’t know” as an explicit response option? On one hand, this has been advocated as a way of filtering out respondents who do not have an opinion and whose responses might therefore be meaningless. On the other hand, it increases the number of respondents who say “don’t know,” resulting in loss of data.

Likert Scales There are correct and incorrect ways of setting up a Likert Scale Used to collect data about respondents' feelings or attitudes The number at one end of the scale represents least agreement, or "Strongly Disagree," and the number at the other end of the scale represents most agreement, or "Strongly Agree." Ends of a Likert Scale are opposites Label each scale item (Don’t place one at the top of the page and require the respondent to refer back to it) The minimum number of points on the scale with descriptors should be both ends and the middle (If an odd number scale)

Odd Versus Even Number Likert Scales When using an odd number of points on the scale (ie: 5 point scale), ends are opposite and the middle is “neutral” ◦ Don’t know or NA is not neutral When using an even number of points on the scale (ie: 4 point scale), end points are opposite. Even numbered scales force the respondents to make decisions.

Available Choices When asking a question with boxes to check, provide instructions as to “check one box”, check all that apply, etc. Be sure to eliminate overlapping categories ◦ How many years have your worked here? ◦ 0-1 year 1-3 years 3-5 years No possible answer ◦ How many years have your worked here? ◦ 0-1 year 1-3 years 3-5 years ◦ What does someone check if they worked 7 years?

Thank You and Return Instructions Thank the participant Provide instruction on how to return the survey

Pilot Testing Before a survey instrument is ever used in a real study, it must be pilot tested!!