1. The Questionnaire Development Process
MKTG 3342
Fall 2008
Professor Edward Fox

2. Questionnaire
Definition – A set of questions designed to generate the information necessary to meet research objectives
Characteristics
Elicits information from respondents
Results can be tabulated
Standardized across respondents
Understandable to respondents
A good questionnaire must:
provide the necessary information
be considerate of respondents
meet coding and data processing requirements

3. The Questionnaire Design Process
Implementation
Determine Data
Collection Methods
Determine Question
Response Format
Prepare Final Copy
Determine Survey
Objectives and
Constraints
Decide on Question
Wording
Pretest and Revise
Obtain Approval
from all Relevant
Parties
Establish
Questionnaire Flow
and Layout
Evaluate the
Questionnaire and
Layout

4. Step 1: Determine Survey Objectives
Translates research objectives into information requirements
Survey objectives should be spelled out as clearly and precisely as possible
Rule 1 - Let the research objectives dictate what questions to ask
Rule 2 - Avoid unnecessary questions

5. Step 2: Determine the Data Collection Method
We previously discussed the variety of ways that survey data can be gathered
Personal
Telephone
Electronic
Mail
Self administered
Each method has a different impact on survey design

6. Step 3: Determine the Question Response Format
Open-Ended Questions
Open-ended questions are those in which the respondent can reply in his/her own words
Example of an open-ended question:
What are the advantages, if any, of ordering from a mail order catalog compared with buying at a nearby retail outlet?

7. Open-Ended Questions Open-ended questions are useful:
to interpret closed-ended responses
when there are too many alternatives to list
when alternatives vary between respondents or are not clear
when intensity of feeling is important
when verbatim response is helpful
for some sensitive issues (e.g. illness)

8. Open-Ended Questions Advantages More information
Wide range of responses
Maybe more accurate description
Limitations
Respondent articulateness can lead to error/variation
Time consuming for respondent
Coding and analysis is difficult

9. Step 3: Determine the Question Response Format
Closed-Ended Questions
A closed-ended question is one that requires the respondent to make a selection from a list of options
The primary advantage of closed-ended questions is avoiding the problems of open-ended questions (such as subjectivity, coding, etc.)
May be dichotomous or multiple choice

10. Closed-Ended Questions
Types
Dichotomous – Respondent chooses between two alternatives
Multiple-Choice – Respondent selects from among several alternatives
Example of a multiple-choice question:
Who do you think will win the World Series?
A. Tampa Bay Devil Rays
B. Boston Red Sox
C. Philadelphia Phillies
D. Los Angeles Dodgers

11. Step 3: Determine the Question Response Format
Scaled-Response Questions
Question format that permits measurement of the “intensity” of a respondent’s answers
Example of a scaled-response question:
Now that you have tried the new sugar-free Slurpee, would you say that you would… (Check one)
definitely buy it
probably buy it
might or might not buy it
probably not buy it
definitely not buy it

12. Closed-Ended Questions
Advantages
Easier to answer
Reduce measurement error by overcoming respondent articulateness and possible interviewer bias
Easy to code and analyze
More likely to respond for some personal data (e.g. income, age)
Disadvantages
Information is lost
Answers may not be precise
Not able to develop alternatives

13. Step 4: Question Wording
Rule 3: Use simple words
e.g., not “marital status” but whether “married,” “single” or “divorced”
Rule 4: Use unambiguous words (words that are easily and clearly understood)
e.g. not “where do you usually buy school supplies?”
but “from which retailer(s) did you buy school supplies this year?”
“where” and “usually” are often ambiguous

14. Question Wording
Rule 5: Avoid “double-barreled” questions (two questions in one)
e.g., “What is your opinion about the price and convenience of DART?”
Rule 6: Avoid leading questions (questions that imply an answer)
e.g. “Do you agree that the government should cut taxes so that we can lead better lives?”

15. Question Wording
Rule 7: Avoid “loaded” questions (framing questions so that they are leading or emotionally charged)
e.g., “Do you think chemical companies are doing everything possible to control pollution?”
Rule 8: Avoid implicit assumptions or alternatives – State them clearly
e.g., “Do you favor a law mandating methanol-free gasoline” [if it means an incras in gas prices]

16. Question Wording
Rule 9: Avoid burdensome questions (questions that tax memory or skills)
e.g., “How many different brands of breakfast cereal have you bought in the last twelve months?”
Rule 10: Clarify references (how the should respondent answer)
e.g., “State your annual income” [personal income or household income?]

17. Question Wording
Rule 11: Alternatives should be mutually exclusive,collectively exhaustive, and have reasonable range
e.g., What is your annual household income?
__ $0 - $10,000
__ $10,000 - $15,000
__ $15,000 - $20,000
__ $40,000 - $60,000
What is wrong with this question and how would you correct it?

18. Question Wording – Scaled Responses
Rule 12: Use appropriate phrases
e.g.,
B. Are you: (1) amazingly happy, (2) middling happy or (3) kind of unhappy (check one)
A. Are you: (1) very happy, (2) happy or (3) unhappy (check one)
Rule 13: Avoid order bias
e.g., should you start with (1) as “very happy” or (1) as “unhappy?”

19. Question Wording - Sensitive / Embarrassing Questions
Questions about borrowing money, personal hygiene, sexual activity, criminal history, etc. must be phrased carefully to minimize measurement error
Suggestions/Techniques - Not Rules
Ask non-threateningly and mix with other questions
e.g., How many times each day do you brush your teeth?
Frame question to prompt more honest responses
e.g., Many people find it difficult to brush their teeth twice a day. How often do you brush your teeth?

20. Step 5: Questionnaire Flow and Layout
Use screening
questions to
identify qualified
respondents
After identifying
qualified respondent,
begin with a question
that piques the
respondent’s interest
Ask general
questions first
Position sensitive,
threatening, and
demographic
questions as the
end
Insert “prompters”
at strategic points
Ask questions that
require “work” in
the middle of the
questionnaire

21. Step 5: Questionnaire Flow and Layout
Rule 14: Opening questions should be simple and interesting – begin with open- ended question if warm-up is necessary
Rule 15: Use “funnel approach” – ask most general questions first, then increasingly specific questions
Rule 16: Questions should flow smoothly and logically

22. Step 5: Questionnaire Flow and Layout
Rule 17: Ask “screener” questions up front to assess respondent qualifications
Rule 18: Ask sensitive questions near the end
Rule 19: Ask for demographic information at the end
Rule 20: The questionnaire’s appearance should be attractive and professional

23. Step 6: Evaluate the Questionnaire
Is each question necessary?
Is the questionnaire too long?
Will the questions provide the desired information to accomplish the research objectives?
Is the layout attractive and professional?

24. Step 8: Pretest and Revise
Rule 21: Always pretest your questionnaire
Pretesting can be done by giving the questionnaire to a few friends, “experts”, and potential respondents (may be 5-10 persons total), asking them to fill out, and obtaining their feedback

25. Step 8: Pretest and Revise
Pretest the questionnaire for:
wording of individual questions
flow of the questionnaire
skip patterns
length
respondent interest and attention

26. Step 9: Prepare Final Copy
Even the final copy phase does not allow the researcher to relax.
Precise typing instructions, spacing, numbering, and precoding must be set up, monitored, and proofread,
before …
Step 10: Implement the Survey