Research Designs

Agenda Experimental Designs Natural Experiments Time Series and Panel Designs Cross Sectional Designs Surveys Surveys

Announcements Next week- half of you meet in library Tuesday, half come here, flip for Thursday A through McGrath meet in Library Tuesday, Here Thursday Mickelson-Z meet here Tuesday, Library Thursday Don’t read Johnson 6, 11

Recap HypothesesVariables Units of Analysis Positive, negative, spurious, antecedent/intervening/mediating, moderating

Intro to Iyengar and Kinder Media and Public Opinion: Hypodermic Needles to Minimal Effects to ?? How might you study media effects Why might experiments be useful?

Experiment Allows for the greatest level of researcher control Systematically vary one variable at a time Control Group Treatment Group Stimulus/Treatment

Experiment Assignment Matching Matching Random Random Within Subjects Design Between Subjects Design

Classic Experiment Randomly assign groups Test at time 1 Provide treatment to one group Provide no treatment to other group Test at time 2 Both between and within subjects design

Iyengar and Kinder Hypotheses- Multiple Exposure influences agenda Exposure influences agenda Exposure influences accountability Exposure influences accountability Qualities of story matter Qualities of story matter Recruit a random sample of citizens Participants watch the news Mild deception- watch with us to avoid distractions

Iyengar and Kinder Pretest What are key IVs/DVs? Show TV news Control group- Unaltered News Control group- Unaltered News Treatment group- Added Story Treatment group- Added Story Post Test measures Findings- Agenda Setting and Priming on Most issues

Iyengar and Kinder Interesting Features? Create natural environment Tight Control of Stimulus Number of Conditions

No pre test Is it desirable to pretest? Between Subjects Test Example- Druckman and Holmes 2005 Comparison between groups Random Assignment makes comparison valid

Quasi Experimental Approaches Increased control over non experimental techniques Less control than true experiment

Sidanius and Pratto Social Dominance Theory Secret Shopper Paradigm Send different races, genders to do various tasks Look at and bargain for apartment Bargain for car Apply for job Controls Results- White men do best, African-American Men do worst

Natural Experiments Two similarly matched entities Differ in “one” key way E.g. Mondak Study of Newspapers in Cleveland and Pittsburgh E.g. political coverage on different networks Social Programs Concern- Lack of Control

Experimental Methods-Advantages Allow the researcher control over the situation Allows for more certainty of measurement Allow for stronger causal inferences- High Internal Validity Control of extraneous factors Control of extraneous factors

Experimental Methods- Disadvantage Limits to External Validity Subject Population Subject Population Reality of Situation Reality of Situation Cannot create some circumstances Cannot create some circumstances Demand Effects Demand Effects Cannot manipulate all factors Personality Personality Ethical Considerations Ethical Considerations Somewhat weak relationship between attitudes and behaviors?

Experiments- When Appropriate When feasible When ethical When factors of interest are controllable Public Opinion Change Media Studies

Time Series Look at cases at multiple times Panel Study- Re-interview same cases at set points Presidential Approval Track from 50s to present Track from 50s to present Look at Approval rating of president as Dependent Variable Look at Approval rating of president as Dependent Variable Look at fluctuations in other factors Look at fluctuations in other factors

Time Series- Advantages Causality- follows temporal patterns Gives best insight into over time change Panels- good for understanding change

Time Series- Disadvantages Must have data over time Consistency of measures over time Causality- confusion? Challenge- empirical generalization Panels Drop outs Drop outs Systematic bias in sample Systematic bias in sample Learning Learning

Time Series- When Appropriate When looking at change over time When looking at historical events When looking at development processes When looking at factors that cannot be easily manipulated in lab or do not vary cross-sectionally

Cross Sectional One time Look at multiple cases Cases should have variation Example- National Election Study

Cross Sectional Studies- Survey Cross Sectional Studies- Survey Random sample of population of interest Ideal Sample greater than 500 Fixed questionnaire Ask all at roughly same time Ideal for public opinion/voting behavior

Cross Sectional Designs- Advantages Does not require as much advanced planning Good for comparing groups of things Snapshot

Cross Sectional Design- Disadvantages Does not work for certain types of phenomena Limited ability to make causal connections Temporal issues Temporal issues Limited number of cases

Case Study Focus intense study on one case Often qualitative Challenge- generality

Surveys More a tool than a design Randomly select large number of people Administer same questionnaire May be cross sectional or time series Allows rich data collection Often most generalizable Expensive Not social Inflexible

Focus Groups 4-10 people together Have them discuss Relatively unstructured Advantage- study more social phenomena Generate new ideas Concern- generalizability Interpretation