The Questionnaire Development Process MKTG 3342 Fall 2008 Professor Edward Fox

Questionnaire Definition – A set of questions designed to generate the information necessary to meet research objectives Characteristics Elicits information from respondents Results can be tabulated Standardized across respondents Understandable to respondents A good questionnaire must: provide the necessary information be considerate of respondents meet coding and data processing requirements

The Questionnaire Design Process Implementation Determine Data Collection Methods Determine Question Response Format Prepare Final Copy Determine Survey Objectives and Constraints Decide on Question Wording Pretest and Revise Obtain Approval from all Relevant Parties Establish Questionnaire Flow and Layout Evaluate the Questionnaire and Layout

Step 1: Determine Survey Objectives Translates research objectives into information requirements Survey objectives should be spelled out as clearly and precisely as possible Rule 1 - Let the research objectives dictate what questions to ask Rule 2 - Avoid unnecessary questions

Step 2: Determine the Data Collection Method We previously discussed the variety of ways that survey data can be gathered Personal Telephone Electronic Mail Self administered Each method has a different impact on survey design

Step 3: Determine the Question Response Format Open-Ended Questions Open-ended questions are those in which the respondent can reply in his/her own words Example of an open-ended question: What are the advantages, if any, of ordering from a mail order catalog compared with buying at a nearby retail outlet?

Open-Ended Questions Open-ended questions are useful: to interpret closed-ended responses when there are too many alternatives to list when alternatives vary between respondents or are not clear when intensity of feeling is important when verbatim response is helpful for some sensitive issues (e.g. illness)

Open-Ended Questions Advantages More information Wide range of responses Maybe more accurate description Limitations Respondent articulateness can lead to error/variation Time consuming for respondent Coding and analysis is difficult

Step 3: Determine the Question Response Format Closed-Ended Questions A closed-ended question is one that requires the respondent to make a selection from a list of options The primary advantage of closed-ended questions is avoiding the problems of open-ended questions (such as subjectivity, coding, etc.) May be dichotomous or multiple choice

Closed-Ended Questions Types Dichotomous – Respondent chooses between two alternatives Multiple-Choice – Respondent selects from among several alternatives Example of a multiple-choice question: Who do you think will win the World Series? A. Tampa Bay Devil Rays B. Boston Red Sox C. Philadelphia Phillies D. Los Angeles Dodgers

Step 3: Determine the Question Response Format Scaled-Response Questions Question format that permits measurement of the “intensity” of a respondent’s answers Example of a scaled-response question: Now that you have tried the new sugar-free Slurpee, would you say that you would… (Check one) definitely buy it probably buy it might or might not buy it probably not buy it definitely not buy it

Closed-Ended Questions Advantages Easier to answer Reduce measurement error by overcoming respondent articulateness and possible interviewer bias Easy to code and analyze More likely to respond for some personal data (e.g. income, age) Disadvantages Information is lost Answers may not be precise Not able to develop alternatives

Step 4: Question Wording Rule 3: Use simple words e.g., not “marital status” but whether “married,” “single” or “divorced” Rule 4: Use unambiguous words (words that are easily and clearly understood) e.g. not “where do you usually buy school supplies?” but “from which retailer(s) did you buy school supplies this year?” “where” and “usually” are often ambiguous

Question Wording Rule 5: Avoid “double-barreled” questions (two questions in one) e.g., “What is your opinion about the price and convenience of DART?” Rule 6: Avoid leading questions (questions that imply an answer) e.g. “Do you agree that the government should cut taxes so that we can lead better lives?”

Question Wording Rule 7: Avoid “loaded” questions (framing questions so that they are leading or emotionally charged) e.g., “Do you think chemical companies are doing everything possible to control pollution?” Rule 8: Avoid implicit assumptions or alternatives – State them clearly e.g., “Do you favor a law mandating methanol-free gasoline” [if it means an incras in gas prices]

Question Wording Rule 9: Avoid burdensome questions (questions that tax memory or skills) e.g., “How many different brands of breakfast cereal have you bought in the last twelve months?” Rule 10: Clarify references (how the should respondent answer) e.g., “State your annual income” [personal income or household income?]

Question Wording Rule 11: Alternatives should be mutually exclusive,collectively exhaustive, and have reasonable range e.g., What is your annual household income? __ $0 - $10,000 __ $10,000 - $15,000 __ $15,000 - $20,000 __ $40,000 - $60,000 What is wrong with this question and how would you correct it?

Question Wording – Scaled Responses Rule 12: Use appropriate phrases e.g., B. Are you: (1) amazingly happy, (2) middling happy or (3) kind of unhappy (check one) A. Are you: (1) very happy, (2) happy or (3) unhappy (check one) Rule 13: Avoid order bias e.g., should you start with (1) as “very happy” or (1) as “unhappy?”

Question Wording - Sensitive / Embarrassing Questions Questions about borrowing money, personal hygiene, sexual activity, criminal history, etc. must be phrased carefully to minimize measurement error Suggestions/Techniques - Not Rules Ask non-threateningly and mix with other questions e.g., How many times each day do you brush your teeth? Frame question to prompt more honest responses e.g., Many people find it difficult to brush their teeth twice a day. How often do you brush your teeth?

Step 5: Questionnaire Flow and Layout Use screening questions to identify qualified respondents After identifying qualified respondent, begin with a question that piques the respondent’s interest Ask general questions first Position sensitive, threatening, and demographic questions as the end Insert “prompters” at strategic points Ask questions that require “work” in the middle of the questionnaire

Step 5: Questionnaire Flow and Layout Rule 14: Opening questions should be simple and interesting – begin with open- ended question if warm-up is necessary Rule 15: Use “funnel approach” – ask most general questions first, then increasingly specific questions Rule 16: Questions should flow smoothly and logically

Step 5: Questionnaire Flow and Layout Rule 17: Ask “screener” questions up front to assess respondent qualifications Rule 18: Ask sensitive questions near the end Rule 19: Ask for demographic information at the end Rule 20: The questionnaire’s appearance should be attractive and professional

Step 6: Evaluate the Questionnaire Is each question necessary? Is the questionnaire too long? Will the questions provide the desired information to accomplish the research objectives? Is the layout attractive and professional?

Step 8: Pretest and Revise Rule 21: Always pretest your questionnaire Pretesting can be done by giving the questionnaire to a few friends, “experts”, and potential respondents (may be 5-10 persons total), asking them to fill out, and obtaining their feedback

Step 8: Pretest and Revise Pretest the questionnaire for: wording of individual questions flow of the questionnaire skip patterns length respondent interest and attention

Step 9: Prepare Final Copy Even the final copy phase does not allow the researcher to relax. Precise typing instructions, spacing, numbering, and precoding must be set up, monitored, and proofread, before … Step 10: Implement the Survey