Designing a Questionnaire

Strategic Concerns in Instrument Design What type of questions (“scales”) are needed? What communication approach will be used? Should the questions be structured? Should the questioning be disguised?

Major Function of a Questionnaire Translating the research objectives into specific questions Iterative Process A good questionnaire is not written, it is rewritten.

Sections in a Questionnaire 1. Introduction or Cover Letter 2. Opening Questions (warm up or screening, or both) 3. Core Questions (always correspond to objectives) 4. Classification data

1. Introduction or a Cover Letter Identification of surveyor or sponsor Purpose of the survey Explanation of respondent selection Request for participation and specifying incentive, if any Screening respondents (not always necessary) Identify the specific response context (not always necessary) Assure either anonymity or confidentiality (even if in-built, may not always be obvious) Specify what to do after the survey is filled

Example of Introduction: The American Airline Survey Dear Customer: Welcome aboard American Airline. We appreciate your business and value your opinions. You have been randomly selected to receive this survey. This survey pertains to the flight you boarded today. Please take a few minutes to answer the following questions, then return the completed survey to the flight attendant.

Example: Toothpaste Survey To determine the purchase frequency of toothpaste To ascertain the favorite store format for buying a toothpaste To gauge the brand loyalty among toothpaste users To determine the most important attributes in toothpaste To explore gender differences, if any, in desired attributes To determine the amount of money a student is willing to spend on toothpaste

Example of Translating Objectives to Questions Beer Survey for the current patrons, who drink at least 3 times a week To ascertain the favorite retail place of buying beer To find out the frequency of beer drinking To find out the place/time of beer drinking To find out the preferred brand of beer To find out the satisfaction level with the favorite brand of beer To uncover the most important attributes in beer To estimate the average purchase price of beer

Requirements for Getting Good Results from Questions Are you asking the right questions? Do they understand the question as intended? Does everyone interpret the question same way? Do they know the answer? Are they willing and able to respond accurately? Other concerns

1. Are you asking the right questions? Does it fit the objective? If it does, then you are asking the right questions.

2. Do they understand the question as intended? Version 1 Version 2 As Uber driver, how do you communicate with the platform managers? Synchronous Communication Asynchronous Communication This (platform, synchronous, and asynchronous) is jargon, please use simple language As Uber driver, how do you communicate with the Uber managers? Email Text Phone Call Something Else ______________ (Please specify)

3. Does everyone interpret the question same way? Version 1 Version 2 What is your income? In year ****, about what was your household income before taxes?

Asking “Toyota-related questions” to someone who never drove it 4. Do they know the answer? Asking “Toyota-related questions” to someone who never drove it Can they remember? how much did you pay for the breakfast bar the last time you bought it? Are intentions meaningful? how many AA batteries will you be buying in next 6 months?

Are they willing and able to respond? Willingness: social desirability bias (if the respondents feel there are social norms about which answers are “right”, you may not get accurate responses). Alcohol consumption (underreported) Reading (overreported)

Minimizing social desirability bias Casual approach do you happen to have murdered your wife? The numbered card “Would you please read off the card what became to your wife?” (hand off the card) Natural death A car ran over her She jumped off the roof I killed her

Minimizing social desirability bias The everybody approach “As you know, many people have been killing their wives these days. Do you happen to have killed yours?” The other people approach Do you know any people who murdered their wives? How about yourself?

Minimizing social desirability bias Sealed ballot question Put the question at the end of the interview

Minimizing other biases Asking loaded questions Double barrelled questions

Categories must be exhaustive Categories must be mutually exclusive 5. Other concerns Categories must be exhaustive Categories must be mutually exclusive

Guidelines for Question Sequencing General to Specific Interesting topics early Simple before complex Group questions by topic Transition between topics The question process must quickly awaken interest and motivate the participant to participate in the interview. More interesting topical target questions should come early. Classification questions that are not used as filters or screens should come at the end of the survey. The participant should not be confronted by early requests for information that might be considered personal or ego-threatening. Buffer questions are neutral measurement questions designed to establish rapport with the participant. These can be used prior to sensitive questions. The questioning process should begin with simple items and then move to the more complex, as well as move from general items to the more specific. Taxing and challenging questions later in the questioning process. Changes in the frame of references should be small and should be clearly pointed out. Use transition statements between different topics of the target question set. An example of a transition is provided in Exhibit 13-10. Sensitive questions later Classification questions last

Insert instructions whenever necessary Insert instructions whenever necessary. Make sure the instructions are clear, comprehensive, and as short as possible Make sure you use 12-font size new roman check for spelling, grammar, alignment etc. Let the layout clarify what the respondents are supposed to do