© 2001 Dr. Laura Snodgrass, Ph.D.1 Non-experimental Methods Observation and Survey Research

© 2001 Dr. Laura Snodgrass, Ph.D.2 Observational Methods Naturalistic Observation Roots in Anthropology Field Research –settings

© 2001 Dr. Laura Snodgrass, Ph.D.3 Observational Data Qualitative –video or audio tape –immerse in situation –describe –interpret –explanatory themes and frameworks for understanding

© 2001 Dr. Laura Snodgrass, Ph.D.4 Observational data Quantitative –coding of events –develop coding systems –e.g. SYMLOG –often requires expectations –limits and directs observations –easier to analyze

© 2001 Dr. Laura Snodgrass, Ph.D.5 Issues in Observational Methods Participation –observation vs. participation –degree of objectivity –degree of direct experience Concealment –reactivity –ethics –nature of group and setting

© 2001 Dr. Laura Snodgrass, Ph.D.6 Issues continued Scope –What behaviors are important? –Negative case analysis events that do not fit explanatory themes –sampling of subsets of behavior

© 2001 Dr. Laura Snodgrass, Ph.D.7 Issues continued The Observer –biased observations expectations pre-existing feelings about groups –reliability training measures of inter-rater reliability

© 2001 Dr. Laura Snodgrass, Ph.D.8 Survey Techniques Questionnaires –individuals may not understand or take seriously –group administration is fast –mail surveys can give low response rates expect about 20%returns –internet surveys - control over users

© 2001 Dr. Laura Snodgrass, Ph.D.9 Survey Techniques Interviews –rapport –interviewer bias –face-to-face –telephone –focus groups

© 2001 Dr. Laura Snodgrass, Ph.D.10 Problems with Answers Response set –social desirability –oppositional people Lack of variation –all one end –all the middle

© 2001 Dr. Laura Snodgrass, Ph.D.11 Addressing the problems Emphasize seriousness and value of data Anonymity of respondent Multiple questions on same issues and direction of answers

© 2001 Dr. Laura Snodgrass, Ph.D.12 Constructing Questions Before you start Types of questions Wording Responses Pretest

© 2001 Dr. Laura Snodgrass, Ph.D.13 Before You Start Keep objective clear –no extraneous questions Plan data analysis first –will the data answer your questions –how will each question be analyzed

© 2001 Dr. Laura Snodgrass, Ph.D.14 Types of Questions Attitudes and Beliefs Facts and Demographics Behaviors

© 2001 Dr. Laura Snodgrass, Ph.D.15 Wording Keep it simple –grade school level –know your population Avoid “double-barreled” questions Avoid “loaded” questions –emotional wording Avoid negative wording

© 2001 Dr. Laura Snodgrass, Ph.D.16 Responses Open- vs. closed-ended –ease of analysis –missing options (choices) Number of response alternatives –usually 5 to 9 Rating scales –e.g. strongly agree to strongly disagree –not at all to very much

© 2001 Dr. Laura Snodgrass, Ph.D.17 Responses continued Numeric scales –give range and explain end points Semantic differentials –bipolar opposites good to bad happy to sad –usually three basic dimensions evaluation activity potency

© 2001 Dr. Laura Snodgrass, Ph.D.18 Responses continued Non-verbal scales –e.g. faces for kids labeling scales –usually good to label alternatives –may have different kinds of choices e.g. recommendation scales all on upper end

© 2001 Dr. Laura Snodgrass, Ph.D.19 Pretest Show friends or small sample –talk out loud –ask questions –alternative interpretations Refine questions –clarify –expand options as necessary