5.4 Creating Survey Questions

Surveys Very common for data collection Types: –Face-to-face interviews –Telephone surveys –Internet surveys –Mail-in surveys All have questions to answer

Creating Survey Questions Difficult! Must pilot/test draft questionnaire –with people who will not form part of survey sample Primary data: –Collected by researcher Secondary data: –Collected from other sources (e.g. internet)

Creating Survey Questions Should be –Relevant to the study –Readable (use familiar words) –Specific –Simple

Things to Avoid Jargon (slang) Abbreviations or acronyms –Abbreviation, not abbrev. –Jarvis Collegiate Institute, not J.C.I. Negatives –“Do you think we should we do this”, not “Do you think we should not do this?”

Things to Avoid Leading respondents –“Should the office continue its heinous practice of forcing students to come to detentions?” –“Given the amount of CO 2 they produce, rate your opinion of SUVs on a scale from 1 to 10.” Double-barrelled questions (asking two things at once) –“Do you walk or take the bus?” Insensitivity: belittling or insulting

Question Styles Open questions –Respondents answer in their own words –E.g.: “What changes should be made in the school to ensure better student achievement?” Closed questions –Respondents given limited number of responses to chose from –Response categories should be Mutually exclusive: should not be able to choose more than one unless you can “choose as many as apply” Exhaustive: all possibilities listed

Open vs. Closed Open Pros –Respondent can interpret and answer question any way they choose Open Cons: –Not necessarily consistent –Difficult to analyse/interpret answers –Wide variety of possibilities Closed Pros –Faster and easier to answer –Faster and easier to code and analyse –Provides consistency Closed Cons –Restricts answers to choices given –Options may bias choice

Ex: Inconsistent Answers One orange & a little water and sugar 25 % orange juice, 75% carbonated water Juice of ½ dozen oranges 3 oz Full strength ¼ c. orange juice None Not much Don’t know A pint Most of it About 1 ½ glass Difficulties with open questions: “How much orange juice do you think this bottle contains?”

Ex: Inconsistent Answers For a uniform response, sometimes you need to state what kind of response is needed. “This bottle contains 250 mL orange drink. How many mL of this drink would you say are orange juice?” millilitres

Question Styles 4 main types: –Information –Checklist –Ranking –Rating

Information Questions Solicit information Usually fill-in-the-blank & open What kind of car do you own? Model _________________ Make __________________Year _____

Checklist Questions Closed Exhaustive Note that these are mutually exclusive response categories. What level of license do you hold?  G2 □ G1 □ G □ None

Another Checklist Question Problems? Unclear if should only choose one What if your favourite is not up there or you don’t have a favourite (not exhaustive)  Other (please state): ________________  None of the above/I don’t have a favourite (Check only one) (Check all that apply) Which of the following brand names is your favourite?  Honda □ Toyota □ Nissan □ GM

Ranking Questions Orders preferences Rank the following car concerns in order of importance from 1 to 4: ____ price____ appearance ____ maintenance ____ gas costsconsumption Is 1 high or low? (1 is low, 4 is high)

Rating Questions How satisfied are you with your car?  Very satisfied  Satisfied  Dissatisfied  Very Dissatisfied Can also assign a numerical value On a scale from 0 to 10, 0 being very dissatisfied and 10 being very satisfied, how satisfied are you with your car? _____ Could use a line:

Creating a Questionnaire Introduction –Informative –Stimulates respondent interest –Should answer: “Why should I answer this?” –State what will be done with the results Begin with easy and interesting questions Questions should flow and read well

Testing a Questionnaire Fantastically important! Do not skip! Identifies problems in survey Discovers poor wording/ordering –People might be unwilling/unable to answer a question Identifies errors in layout and instructions

Testing a Questionnaire Suggests additional response categories that can be pre-coded –If 10 people suggest that Ford should be added to favourite car… Provides preliminary idea of length of interview Provides preliminary idea of refusal problems –Should we adjust survey size to account for non- response?