Download presentation
Presentation is loading. Please wait.
Published byValerie Taylor Modified over 8 years ago
1
Surveys Alfred Kobsa University of California, Irvine
2
When to chose a survey Scope of problem very clear Questions can be clearly formulated at a concrete level Complex/holistic answers not needed. Checking pre- formulated answers is sufficient (plus occasionally one or two free-text sentences). Mailing lists with current addresses available Specific location not required Answers from many users needed Answers needed quickly Broad geographic distribution of respondents needed Anonymity required
3
Steps Identify the objectives of the study (in writing) Select the target audience Select the form of announcement and response collection (email, physical mail, web [ ☛ survey websites], in person, phone) Decide how to analyze the data (software, statistical tests if any) Brainstorm questions (closed questions, open questions) Formulate questions (mind double-barreled, double negatives, leading or loaded questions, and questions with self-image, acquiescence and social desirability bias) Use existing surveys (e.g. System Usability Scale SUS ) Consider capabilities of survey site Reduce question set Pilot test and revise your questions!
4
Problems to consider Non-representative sample population ( ☛ compare demographics of respondents with demographics of names in mailing list and normalize answers accordingly) Low response rate ( ☛ pre-announcement, personalized cover letter, reward, short survey, few open-ended questions, self- addressed envelope w/ stamp, multi-mode reminders) Self-selection bias of respondents based on topic ( ☛ hide your study goal among more general questions) Attention, honesty ( ☛ reverse-code questions, check consistency, discard data) Self-incrimination ( ☛ emphasize“harmlessness”, impunity, anonymity) Social desirability bias, self-image bias ( ☛ phrase questions carefully, use several questions for same concept, go for facts and not habits, attitudes or intentions) Sensitive questions ( ☛ explain their purpose, do not make answers mandatory, offer broad answer bins)
5
Response format Closed-ended questions Single-choice or multiple-choice (include “other” + “explain”) Ratings (“on a scale from 1 to 5, how would you rate…”) Likert* scale: [1 … n] or [-n -n+1…. n-1 n] ( typically n = 5, 6 or 7) with mid-point or without (forces respondents to take a stance) with verbal “anchors” for first and last value only, or for all/some values including first and last** Ranking scales (e.g., sorting by priority) Open-ended questions Should be kept to a minimum and optional since they may cause users not to fill in or abandon a questionnaire if users answer them, they only provide brief answers answers are hard to evaluate *) pronounced like “ike” in the US, but like “lickert” elsewhere (also by Mr Likert himself and his son) **) First and last value anchors should be “extreme”
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.