1. STRUCTURED INTERVIEWS AND INSTRUMENT DESIGN PART II Lecture 8

2. 2 Question Response Logic: Categorical Responses Responses must be mutually exclusive  Example (bad): Where do you live?  Berkeley, San Fran, Bay Area, Other Responses must be exhaustive  Example (bad): What kind of computer do you have?  PC, Mac Use ‘Don’t Know’, ‘Other’, ‘Not Applicable’ when absolutely necessary

3. Mutual Exclusivity and Exhaustive Responses 3 How do you perceive communication between your department and other departments in the university? [Check one] (1) There is much communication (2) There is sufficient communication (3) There is litter communication (4) There is no communication (5) No basis for perception

4. 4 Question Sequence Static order for questions needs to have some rationale/logic  Grouping similar items together  Scattering similar items throughout survey Personal demographic questions work best at end of survey (response rate and completion) Randomization for all respondents

5. 5 Clear Wording / Leading Questions Clear Wording  Examples:  “What ISP do you use?  “If you have Internet service at home, what company or service provider do you use for Internet access?” Leading Questions  Example:  “Don’t you think we should support our troops in Iraq?”  “How strongly do you agree or disagree with the following question: ‘We should support our troops in Iraq’”

6. 6 Clear Formatting, Logic and Branching Not all questions apply to everyone  Example:  “How much do you spend on gas heat each month?” Branching is a possible solution  Example:  “Do you have gas heat? If yes, go to next question. If not, skip to question #3. Condense when possible to avoid unnecessary branching.  Example:  “How much do you spend on gas heat each month? (write 0 if you do not have gas heat)

7. ‘Unpacked’ Questions 7 Some people believe that the internet reduces face- to-face interaction and may cause ‘internet addiction’. Do you agree or disagree with these people? (1) Agree (2) Disagree (3) Do Not Know (4) Refuse to Answer

8. 8 Know your sample population Regional language and terminology Cultural differences How you conduct survey can influence your valid sample  Door to door?  Registered telephone directory?  Internet-based survey?

9. 9 Replication and Using Existing Survey Instruments ALWAYS a good idea to find other surveys that are used in your area of interest.  Especially with large, funded surveys the questions may have been tested for reliability.  Allows for comparisons between different samples if the question wording is the same.  If a question or set of questions is accepted as a good operationalization of the concept you are interested in, you don’t want to reinvent it unless you really intend to argue that your measure is more appropriate.

10. 10 Scales and Scaling

11. Indicators and Composite Measures 11 Single Indicator Scale  Assigns units of analysis to categories of a variable Composite Measure Scale  Cumulative index  Criterion referenced index  Norm referenced

12. 12 Common Scaling Techniques  Cumulative Index (Q1 + Q2 + Q3)  Guttman Scale (Q1 -  Q2  Q3)  Likert-style Scales [Q1, Q2, Q3]

13. 13 Replication and Using Existing Survey Instruments ALWAYS a good idea to find other surveys that are used in your area of interest.  Especially with large, funded surveys the questions may have been tested for reliability.  Allows for comparisons between different samples if the question wording is the same.  If a question or set of questions is accepted as a good operationalization of the concept you are interested in, you don’t want to reinvent it unless you really intend to argue that your measure is more appropriate.

14. 14 Pre-Testing and Pilots Pre-tests and Pilots are always necessary, unless the survey in its existing form has already been given before to the current population. Pre-testing and Pilot studies should have a large enough response rate so that you can actually find problems!  Example: You want to survey 100 undergrads for a small study. You may need to at least pre-test on a 20% sample from your population of undergraduates. However, you cannot use these pre-test participants in your full sample of 100 students.

15. 15 Testing your questions during pre-testing Behavior coding– interview some respondents as you give the survey questions and keep track of requests for clarification. Ask pretest respondents to rephrase your questions in their own words. Panels of ‘experts’: give your questions to groups of individuals for comments/suggestions.