STRUCTURED INTERVIEWS AND INSTRUMENT DESIGN PART I Lecture 7

“Continuum” of Interview Situations 2 UnstructuredStructured Open-ended interviews Interview Guides, still open-ended Identical Instrument (questions, etc) given to all participants

3 What is a Survey? To survey:  “act of looking or seeing, observing” Research Surveys  Qualitative interviews, focus groups  Specific, systematic, quantitative data-collection instruments

4

5 Example Surveys US Census General Social Survey Online Surveys

Structured Interview Formats 6 Face-to-face, self-administered ( , mail, etc), phone surveys Typically either cross-sectional or longitudinal Often a balance between  (1) standardization  (2) ability to reach sample  (3) available resources  (4) type of questions you are asking

Structured Interviewing 7

Standardization of Wording and Format in an Interview Instrument 8 Must have a logic for each question  Must have a logic for the question responses (if provided)  Must have a logic for the sequencing of questions Must have clear wording Must have clear formatting Must take into account the sample population that will actually take/use the survey instrument  Culture, language, interpretive ambiguity

Question Logic 9 Avoid asking unnecessary questions  Example: Survey on computer usage  Socio-demographic questions (age, gender)?  Risk behavior questions (drug use, etc)? Closed or Open-ended?

Closed versus Open Questions 10 (Q1a) What do you spend most of your time doing on the Internet? (Q1b) Which of the following things do you do most of the time on the Internet? (a) (b) Browse the web (c) Play games (d) Other

Question Logic 11 Avoid redundant questions unless you have a clear reason  Exception: Sometimes it is a good idea to ask two questions that tap same concept (first as a scale, then as a categorical decision)  Example (ranks):  First ask respondent to rate 1-10 how much they agree with several statements.  Next, ask respondents to rank the statements by how much they agree with them (i.e., 5 statements, rank them 1-5).

Question Response Logic: Scales versus Categories 12 As a basic rule, metric scales with more range are better than binary or categorical responses (all other things being equal).  Example: Age  Are you 18 or older (yes/no)?  How old are you (in years)? _________ With general scales and Likert-scales, consider having no “middle” category (neutral, no opinion).

13 Question Response Logic: Categorical Responses Responses must be mutually exclusive  Example (bad): Where do you live?  Berkeley, San Fran, Bay Area, Other Responses must be exhaustive  Example (bad): What kind of computer do you have?  PC, Mac Use ‘Don’t Know’, ‘Other’, ‘Not Applicable’ when absolutely necessary

14 Question Sequence Static order for questions needs to have some rationale/logic  Grouping similar items together  Scattering similar items throughout survey Personal demographic questions work best at end of survey (response rate and completion) Randomization for all respondents

15 Clear Wording / Leading Questions Clear Wording  Example:  “What ISP do you use?”  “If you have Internet service at home, what company or service provider do you use for Internet access?” Leading Questions  Example:  “Don’t you think we should support our troops in Iraq?”  “How strongly do you agree or disagree with the following question: ‘We should support our troops in Iraq’”

16 Clear Formatting, Logic Not all questions apply to everyone  Example:  “How much do you spend on gas heat each month?” Branching is a possible solution  Example:  “Do you have gas heat? If yes, go to next question. If not, skip to question #3. Condense when possible to avoid unnecessary branching.  Example:  “How much do you spend on gas heat each month? (write 0 if you do not have gas heat)

17 Know your sample population Regional language and terminology Cultural differences How you conduct survey can influence your valid sample  Door to door?  Registered telephone directory?  Internet-based survey?

18 Replication and Using Existing Survey Instruments ALWAYS a good idea to find other surveys that are used in your area of interest.  Especially with large, funded surveys the questions may have been tested for reliability.  Allows for comparisons between different samples if the question wording is the same.  If a question or set of questions is accepted as a good operationalization of the concept you are interested in, you don’t want to reinvent it unless you really intend to argue that your measure is more appropriate.