Ed Blair University of Houston Questionnaire Design Ed Blair University of Houston

Broad issues in writing questions: Have you asked the right question? Do respondents understand the question? Do they know the answer? Are they willing to answer accurately?

Have you asked the right question? In marketing research, we ask questions because the answers will help improve marketing decisions. A question can be technically perfect, yet still be a bad question because it doesn't properly serve our marketing purpose. Is this a good question? 1. Have you heard of any physician referral services that you can call if you need help in finding a doctor? Yes . . . . . . 1 (ASK Q1A) No . . . . . . 2 (GO TO Q2) (IF YES) 1A. Can you name any of these services or tell me their telephone numbers? (PROBE: Any others?)

Making sure it’s the right question Ask yourself: Is this question consistent with my theory about how the market works? Do I know, specifically, how I plan to use the answers to this question to draw conclusions about the market?

Do respondents understand the question? Q: Does your home have power conditioning equipment to protect your critical equipment against power fluctuations? Q: (en Español?) Do all respondents interpret the question the same way? Q: What is your income? Do respondents interpret the question the same way you do? Q: What are all the reasons you bought your groceries at ValueLand rather than some other store? (PROBE: Any others?)

Minimizing problems from misunderstanding Ask questions one at a time. BAD Do you consider a backup generator or a surge suppressor to be necessary equipment for your home? Use simple, specific language, and avoid words with more than one meaning. BAD? Do you exercise regularly? Specify who, what, when, where, how, and the desired form of response. Which of the following categories best describes your family’s total income last year before taxes? Before doing the project, have someone read you the questions. Also consider pre-testing your questionnaire to see whether respondents have trouble understanding some of the questions, and how they interpret key questions.

Do respondents know the answer? Were they there? Q: How satisfied were you with the service you received from the hospital's business office? Q: Why did your company…? Can they remember? Q: In the past thirty days, about how many telephone calls did you make? Do they have an opinion? Q: I have a list of hospitals in our area. As I read each one, please tell me how you would rate that hospital as a place to receive care: excellent, good, fair, or poor. How about . . . Are intentions meaningful? Q: If there was a local magazine like Consumer Reports that evaluated restaurants, auto repair shops, stores, plumbers, and other local businesses, would you subscribe? Q: How many small batteries do you intend to buy during the next six months?

Minimizing knowledge problems (a) Consider qualifying respondents for knowledge or allowing for “no opinion” or “not sure” or “don’t know.” Use the right time frame for your question. Event Reasonable time frame Hospital stay, car purchase One year Appliance purchase Three months Grocery purchase One week Food consumption One day Measure behavior over intentions/attitudes (e.g., previous battery purchases over intent to purchase, actual referrals over intent to refer/satisfaction). Conjoint, anyone?

Minimizing knowledge problems (b) Anchor subjective measures with comparisons across time, objects, or people. 78% of patients say they were "very satisfied" overall with the food at your hospital. Is this good or bad? - What if the number was 84% last year? - What if the number is 84% at another hospital? - What if the number is 90% for maternity patients, 71% for oncology patients? Use previous questions wherever possible! Interpret and present self-report data as if they are approximate, not exact, and focus on patterns in the data rather than specific tabulations.

Are respondents willing to answer accurately? Is the behavior or attitude socially desirable or undesirable? Q: During the past month, have you read any books other than for work or school? Q: During the past month, since (DATE) , did you drink beer even once? (IF YES) A. About how often did you drink beer during the past month? Once . . . . . . . . . . . . . . ……….1 2-3 times . . . . . . . . . . . ……....2 About once a week . . . . . . … .3 2-3 times a week . . . . . . . ……4 4-6 times a week . . . . . . . . …..5 Once a day or more . . . . . . . …6 Do respondents think you care which answer they give? Q: If there was a local magazine like Consumer Reports that evaluated restaurants, auto repair shops, stores, plumbers, and other local businesses, would your family subscribe to this magazine?

Minimizing problems with unwillingness (a) Professionalism is a must, to convince respondents that you care whether they answer but not how they answer. This means: - Professional appearance for questionnaires that respondents will see. - A proper and professional introduction. - Formal language in the questionnaire (no slang, please) - Business clothing for interviewers if respondents will see them - Proper training so that interviewers seem to know what they are doing Use choice measures or previous behavior rather than intention measures. Check for loading. Don't ask subjective questions (attitudes, opinions, etc.) in in a "Yes-No" format, and avoid “angel" or "devil" words (e.g., reasonable, forbid), absolute terms (always, never), and appeals to the norm (“most people think")

Minimizing problems with unwillingness (b) Don't reveal sponsor identity unless necessary to convince respondents that their answers will have a legitimate use. Consider less personal methods (mail, online) for sensitive topics. Use open questions for sensitive quantitative questions (except income) Use longer questions to reduce the social stigma attached to certain answers. Q: Many people say that they just don't have time for reading anymore. In the past month, have you read any books other than for work or school?

What’s wrong with this questionnaire? (a) (Fan survey to be administered at an Astros game) 1. In what city do you live? ____________________ (IF HOUSTON) A. In what part of Houston do you live?  Inside the Loop  South  North  Southwest  Northeast  West  East  Northwest  Southeast 2. How many people are in your group at tonight's game?  1  2-3  4-5  6 or more

What’s wrong with this questionnaire? (b) 3. How many of them are kids?  1  2-3  4-5  6 or more 4. How many people in your group, including yourself, are Astros season ticket holders? 5. How many games have you, yourself attended this season? 6. What four radio stations do you listen to most often? __________________ _________________

What’s wrong with this questionnaire? (c) 7. Why did you come to tonight's Astros game? __________________________________________ We need your name and address for our prize drawing. NAME: ________________________________________ ADDRESS: ________________________________________ CITY, STATE, ZIP ___________________________________

Revised questionnaire (a) 1. Are you attending tonight's game on a season ticket or a single game ticket?  Season ticket  Single game ticket 2. Counting tonight's game, how many Astros games have you attended this season?  1  2-3  4-5  6 or more 3. Counting yourself, how many people are in your group at tonight's game? 4. How many of the people in your group are less than 13 years of age?  0  1  2-3  4-5  6 or more

Revised questionnaire (b) 5. When did you decide to attend tonight's game? Was it today, before today but during the past week, or earlier than the past week?  Today  Past week  Earlier 6. What promotions, if any, might influence you to attend an Astros game? (CHECK ALL THAT APPLY)  $2 off tickets  Prize drawings  Family discounts  Country music concerts  Free parking  Rock music concert  Free Astros mug  Tailgate parties  Free kid's cap  Home run derby

Revised questionnaire (c) 7. What is your favorite radio station? (FILL IN) _____________________ We need your name and address for our prize drawing. │ NAME: ________________________________________ ADDRESS: ________________________________________ CITY, STATE, ZIP ____________________________________