STAT 472 Survey Design Constructing the Questionnaire

 A good questionnaire forms an integrated whole.  The researcher weaves questions together so they flow smoothly.  He or She includes Introductory Remarks, Instructions for clarification and measure each variable with one or more survey questions

 There are two keys principles: - Avoid confusion (Keep it clear and simple) - Keep the respondent’s perspective in mind  Good survey questions give the researcher Valid and reliable measures Help respondents feel that they understand the question and that their answers are meaningful Do not mesh with a respondent's view point

Question writing is more of an art than a science. It takes skill, practice, patience, and creativity  A survey researcher must exercise extra care if the respondents are come from different life situations  Researchers face a dilemma All respondents hear exactly the same questions, equally clear, relevant, meaningful and exact wording If respondents have diverse back grounds and frames of reference, the exact same wording may not have the same meaning. Yet, tailoring question wording to each respondent makes comparisons almost impossible

(1) Avoid jargon, slang, and abbreviations: (unless a specialized population is being surveyed) Jargons and technical terms come in many forms: Plumbers talk about snakes Slang: is a kind of jargon Skiers about a hotdog Abbreviations: for example NATO means North Atlantic Treaty Organization North African Tea Office National Auto Tourist Organization Native Alaskan Trade Orbit

(2) Avoid ambiguity, confusion and vagueness Ambiguity and vagueness plague most question writers For example: What is your income? Could mean, weekly, monthly, or annual; family or personal; before tax or after; for this year or last year; from salary or from all sources "Do you jog regularly? Yes / No”  Use of indefinite word regularly or response categories “hinges on the meaning of the word regularly. It may define as every day, others as once a week” Good Question: “To reduce respondent confusion and get more information” Do you jog "about once a day”, “a few times a week”, “once a week”?

The question wording is an art that may improve with practice, patience, and pilot testing Here are three survey questions written by experienced professional researchers. They revised the original wording after a pilot test discover that 15% of respondents asked for clarification or gave inadequate answers (e.g., don't know)

Q1. Do you exercise or play sports regularly? Original question Problem: What counts as exercise? Response to question: 48% saying “YES” % asking for clarification: 5% Q1. Do you do any sports or hobbies, physical activities, or exercise, including walking, on a regular basis? Revised question Response to question: 60% saying “YES” % asking for clarification: 0%

Q2. What is the average number of days each week you have butter? Original question Problem: Does margarine count as butter ? Response to question: 33% saying “none” % asking for clarification: 18% Q2. The next question is just about butter-not including margarine. How many days a week do you have butter? Revised question Response to question: 55% saying “none” % asking for clarification: 13%

Q3. [Following question on eggs] What is the number of servings in a typical day? Original question Problem : How many eggs is a serving? What is a typical day? Response to question: 80% saying “one” % asking for clarification: 33% Q3. On days when you eat eggs, how many eggs do you usually have? Revised question Response to question: 33% saying “one” % asking for clarification: 0%

(3) Avoid emotional language and prestige bias murderousthreaten freedompeace-loving Example ( emotional ): What do you think about a policy to pay murderous terrorists who threaten to steal the freedom of peace-loving people? Is full of emotional words (murderous, freedoms, steal, and peace) Problem: respondents may react to the emotionally laden words rather than to the issue

Example 1 (Prestige bias) Most doctors say that cigarette smoke causes lung disease for those near a smoker, do you agree? Problem: tends to provoke "yes" answers because people trust doctors Example 2 (Prestige bias) "Do you support the president's policy on Zimbobutu? P roblem: tends to provokes "yes" answers because people trust president

(4) Avoid double-barreled questions Double-barreled questions consists of two or more questions joined together. Example : How do you evaluate the work of the Jordanian parliament and prime minister ? The two variables are: Jordanian parliament : prime minister

wrong The wrong way is to ask one question: How do you evaluate the work of the Jordanian parliament and prime minister ? correct The correct way is to ask two questions: 1- How do you evaluate the work of the Jordanian parliament ? 2- How do you evaluate the work of the prime minister ?

Example: Tell me whether you like summer and winter in Irbid or not? correct The correct way is to ask two questions: 1- Do you like summer in Irbid? 2- Do you like winter in Irbid?

(5) Avoid leading (Bias) Questions Make respondents feel that all responses are legitimate. Do not let them become aware of an answer that the researcher wants. A leading question is one that leads the respondent to choose one response over another by its wording. Example: The government should force you to pay higher taxes. No one likes to be forced, and no one likes higher taxes. Alternative: The government should increase taxes, or the government needs to increase taxes.

Example: Don’t you think that suffering terminal cancer patients should be allowed to be released from their pain? Researchers should never try to make one response option look more suitable than another. Alternative: The suffering terminal cancer patients should be allowed to be released from their pain, do you agree? Example: Wouldn’t you like to receive our free brochure? Example: You don't smoke, do you?

(6) Avoid asking questions that are beyond respondents capabilities. People have cognitive limitations, especially when it comes to memory of past events Example: (asking adult) How do you feel about your brother when you were 6 years old? is probably useless It is pointless to ask people about things that are not natural ways for them to think. Example: How many gallons of gasoline did you buy for your car last year? "Yet, respondents may be able to answer a question about gasoline purchases for a typical week, which the researcher can multiply by 52 to estimate annual purchases