1. 1 Chapter 11: Survey Research Summary page 343 Asking Questions Obtaining Answers Multi-item Scales Response Biases Questionnaire Design Questionnaire Administration Survey Data Archives

2. 2 Asking Questions Open and Closed-Ended Question Wording –Use participants’ language –Avoid negatives –Ask one at a time –Avoid leading and loaded Qs –Be specific –Don’t make assumptions –Address sensitive topics sensitively

3. 3 Purposes and Types Two purposes…what are they? –Estimate population parameters –Test hypotheses –Which type can be tested? Corr or Experimental? Both! But how would you conduct an experiment? Two types of recruiting Ps are…? –Sample surveys –Convenience surveys

4. 4 Before Asking Questions Establish purpose(s) –“The purpose of this survey is to determine…” Specifics ….e.g. Ps attitudes toward, intentions, etc. –Characteristics of Ps (attributes) –Beliefs (cognitive) –Feelings (affect) –Behavioral intentions –Past behaviors Objectives (Explicit) –Specify what findings will indicate the objectives were met –Anticipate what analyses will need to be conducted (ahead of time!)

5. 5 Before Asking Questions Determine sample demographics needed –Grouping variables E.g. race, gender, age, occupation, dept, geographical Determine constructs –Established measures E.g. job stress –Self-developed measures E.g. Attitudes toward drug therapy –Single item variables E.g. intention to continue or quit Determine topic and question order

6. 6 Asking Questions: Open v. Closed Ended Open Qs when: –For some sensitive Qs –When don’t know all options Esp. exploratory study (use interview/focus group) –Certain types of judgments Caution: –Open ones have to use content analysis

7. 7 Asking: Question Wording Use Ps language –To hell with grammatically correct –Clarity takes precedence Avoid Negatives (double ones) –But counterbalance to avoid acquiescence One at a time –No double barreled “would you like to pass this course and get an A?”

8. 8 Asking: Question Wording (con’t) No leading questions –“Don’t you agree that Dr. Mitchell is an excellent professor?” Be specific –“Do you like your instructors here at UB?” Don’t make assumptions –“How long have you been a thug?” Address sensitive topics sensitively –“Even the best students get irritated with professors sometimes”

9. 9 Obtaining Answers Levels of measurement –Consider the construct –Consider precision needed Information content Statistical tests –Decide ahead of time! Ecological Validity –E.g. mock jury (interval) v. real jury (nominal)

10. 10 Response Format p 354 Choice depends on purpose –E.g. should each of a category be itemized? Comparative rating scales –Method of paired comparisons (all possible combos) Itemized rating scales (MC questions) –Make sure responses are mutually exclusive –Nominal, ordinal Graphic scales –Visual anchors Numerical rating scales –5,7,9 points with labels (interval level)

11. 11 Response Formats con’t Multi-item scales –Likert scales Consider direction: reverse some or not –Thurstone Not worth the trouble –Guttman scales Ordered attitude items with increasing levels of agreement –“marry 1. stranger, 2. cousin, 3. sibling. 4. ? –Semantic Differential Choose relevant adjectives

12. 12 Response Biases Question-related –Scale ambiguity (choose meaningful labels) –Category Anchoring (find right range) –Estimation biases (open ended) Retrospective is very unreliable –E.g. self-reported illegal drug use –Respondent interpretations of numerical scales E.g. “not at all successful?”

13. 13 Response Biases Person-related –Social desirability –Acquiescence –Extremity response rates (gender, age, race differences) –Halo & Leniency –Interpreting responses “Literal interpretation fallacy” –A good reason to calculate means

14. 14 Questionnaire Design Question order Question sequencing –Easy to difficult; related to purpose –General to specific within topic Context effects –An early question may influence response to a later one

15. 15 Questionnaire Design Layout –Use closed as much as possible –Consistent format depending upon the nature of constructs Whether or not you want to compare to norms Instructions –Used to motivate and clarity how to respond Use existing measures Response formats

16. 16 Methods of Data Collection Group (efficient) Mail (low response rate) Personal (good but expensive) Phone (efficient and provide anonymity) Focus groups –Confounding, conformity, all sorts of problems Computer (lots of advantages) –But identity confirmation, duplicate entry problems Compare methods (use table 11-12, p. 382)

17. 17 Survey Data Archives Great, but never seem to answer your research questions But start there

18. 18 Survey Research summary Asking Questions Obtaining Answers Multi-item Scales Response Biases Questionnaire Design Questionnaire Administration Survey Data Archives