1. Designing the Questionnaire or Observation Form

2. Procedure for Developing a Questionnaire Step 1 Step 2 Step 3 Step 5 Step 4 Step 6 Step 7 Step 8 Step 9 Specify what information will be sought Determine type of questionnaire and method of administration Determine content of individual questions Determine form of response to each question Determine wording of each question Determine question sequence Determine physical characteristics of questionnaire Reexamine steps 1-7 and revise if necessary Pretest questionnaire and revise if necessary

3. Open-Ended Questions 1.How old are you? ______ 2.How do you feel when you drive your Toyota minivan? Why do you feel that way? 3.Can you name three sponsors of the Monday night football games? _____ 4.Do you intend to purchase an automobile this year? _____ 5.Why did you purchase a Sony brand color television set? _____ 6.Do you own a VCR? _____

4. A fixed-alternative question in which respondents are asked to choose the alternative that most closely corresponds to their position on the subject. Example How old are you? ____ Less than 20 ____20-29 ____30-39 ____ 40-49 ____50-59 ____60 or over Multichotomous Questions

5. A fixed-alternative question in which respondents are asked to indicate which of the two alternative responses most closely corresponds to their position on the subject. Example Do you think laws requiring passengers in motor vehicles to wear seat belts are needed? Yes No Dichotomous Questions

6. A technique used to combat response bias in which one phrasing is used for a question in one-half of the questionnaires while an alternate phrasing is used in the other one-half of the questionnaires. Split Ballot Example Do you think gasoline will be more expensive or less expensive next year than it is now? More expensive Less expensive Do you think gasoline will be less expensive or more expensive next year than it is now? More expensive Less expensive

7. A source of non-sampling error that arises when a respondent agrees to an interview but refuses, or is unable, to answer specific questions. Item Non-Response

8. Flame-broiled or Fried? Original Question: “Do you prefer your hamburgers flame-broiled or fried?” Result:flame-broiling (Burger King) beat frying (McDonald’s) by a 3-1 margin Revised Question #1: “Do you prefer a hamburger that is grilled on a hot stainless-steel grill or cooked by passing the raw meat through an open gas flame?” Result:53% preferred the grill (McDonald’s) Revised Question #2: “The chain that grills on a hot stainless-steel griddle serves its cooked hamburgers at the proper temperature without having to use a microwave oven. And the chain that uses the gas flame puts the hamburgers after they are cooked into a microwave oven before serving them. Just knowing this, from which of these two chains would you prefer to buy a hamburger?” Result:the grill (McDonald’s) beat the flame (Burger King) by a 5.5 to 1 margin

9. A question framed so as to give the respondent a clue as to how he or she should answer. Example Do you feel that limiting taxes by law is an effective way to stop the government from picking your pocket every payday? Yes No Undecided Leading Questions

10. A problem that occurs when an important alternative or response option is not included in a question (or response categories). Implicit Alternative Example Would you like to have a job, if this were possible? Would you prefer to have a job, or do you prefer to do just your housework? YesNo YesNo

11. A problem that occurs when a question is not framed so as to explicitly state the consequences, and thus it elicits different responses from individuals who assume different consequences. Implicit Assumption Example Would you like to double the number of job offers you receive as a senior? Would you like to double the number of job offers you receive as a senior if that means devoting an additional 10 hours per week to studying so as to raise your grade point? YesNo YesNo

12. A question that calls for two responses and thereby creates confusion for the respondent. Double-Barreled Question Example Are the food and service at the local Pizza Hut good? Yes No Key: Be on the lookout for “and” and “or” in your items!

13. Use simple words and questions Avoid ambiguous words and questions Avoid leading questions Avoid implicit alternatives Avoid implicit assumptions Avoid generalizations and estimates Avoid double-barreled questions Guidelines for Question Wording

14. Use simple, interesting opening questions Use the funnel approach, asking broad questions first Carefully design branching questions Ask for classification information last Place difficult or sensitive questions near the end Guidelines for Question Sequencing

15. Funnel Approach An approach to question sequencing that gets its name from its shape, starting with broad questions and progressively narrowing down the scope. Example How would you rate your last experience when eating at ABC restaurant? Extremely Poor Poor So-So Good Good How would you rate the quality of the service? Extremely Poor Poor So-So Good Good Guidelines to Question Sequencing (Cont)

16. Example of Question Order Bias Demand estimation for new product Key question: buying interest (5-point scale, “not at all interested-very interested”) Questions asked before buying % very interest questioninterested None 3 Asked about advantages17 Asked about disadvantages 0 Asked about advantages and disadvantages 6

17. Branching Question A technique used to direct respondents to different places in a questionnaire, based on their response to the question at hand. Guidelines to Question Sequencing (Cont)

18. Branching Question Example 1. Please answer the following question Yes or No. Did you, personally, give money to Canopy of Care last year? Y or N {If Yes, go to #3.} 2.We are interested in why people do not contribute. The following is a list of answers others have given. Please tell me which, if any, apply to you. a. ___ Someone else in my household had already contributed. b. ___ I did not have the money at the time. c. ___ I gave to other charities. d. ___ I volunteered my services to Canopy of Care instead of contributing money. e. ___ I volunteered my services to other charities instead of contributing to Canopy of Care. f. ___ I did not give because Canopy of Care spends its money inefficiently. g. ___ None of the above. 3.To how many different charities do you think Canopy of Care gives money? a. ___ 0-20 d. 81-100 b. ___ 21-40e. More than 100 c. ___ 41-80f. Don’t know. Guidelines to Question Sequencing (Cont)

19. Use of a questionnaire (or observation form) on a trial basis in a small pilot study to determine how well the questionnaire (or observation form) works. Pretest ALWAYS PRETEST DATA COLLECTION FORMS!!!!!

20. Handling Sensitive Questions Don’t ask unless absolutely necessary! Give broad response categories (e.g., income, age) Place near end of questionnaire “Hide” them in less sensitive questions Use counterbiasing statement Use randomized response technique

21. Use of Counterbiasing Statement Recent studies have shown that a high percentage of males use their wives’ cosmetics to hide blemishes. Have you used your wife’s cosmetics in the past week?

22. Randomized Response Model An interviewing technique in which potentially embarrassing and relatively innocuous questions are paired, and the question the respondent answers is randomly determined but is unknown to the interviewer.

23. Randomized Response Technique (example from Weiers 1988) Initial, “benchmark” study with nonsensitive question: “Do you likepepperoni pizza?” (65% say “yes.”) Second survey, involving 400 respondents. Each respondent flips coin, with flip result not revealed to researcher. Heads Tails Respondent answers question 1: “Do you like pepperoni pizza?” Respondent answers question 2: “Have you ever been in jail?” 160 “yes” responses 240 “no” responses

24. Randomized Response Technique (example from Weiers 1988) 160 “yes” responses 240 “no” responses 130 (.65x200) 70200 (.5x400) 30 (160-30) 170200 Pizza Jail “Yes” “No” 160 240 Conclusion: Estimate that 30/200, or 15% of population has “been in jail.”

25. Physical Characteristics of Questionnaire: Suggestions  questionnaire should appear simple to complete (white space is your friend!)  minimize number of pages (smaller fonts are OK, provided form appears simple)  mix-up response formats occasionally (avoids response set bias and breaks monotony)  use directions as necessary for each group of items, but keep them short and simple  number items within each section  have respondents check boxes rather than lines  use shading, boxes, lines, etc., to keep it interesting

26. Other Issues  Pretest!  Know exactly how each piece of information will be used  NO “nice-to-know” questions!  Back translation for international questionnaires  Pre-coding