Introduction: Research design Spring

The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 2

The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 3

The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 4

The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 5

The problem How do we make sure that quality differences observed among machine types are due to machine types per se  This is an issue of causality  We attend to “internal validity” so that when we observe differences between groups, we can assure ourselves that this is because of the “treatments” of interest 6

Review of internal and external validity

Internal validity: the two problems The two primary threats to internal validity. 1. Nonrandom selection into the treatment group (confounding variables)  Comparing apples with apples or apples with oranges?  Random assignment ensures apple to apple comparisons  Regression, matching, difference-in- differences also attempt to compare apples with apples 2. Reverse causation  The chicken and egg problem, which came first?  Is your dependent variable influencing your treatment (your explanatory variable)? If you can address these problems, you almost always have an internally valid study Randomly assigned experiments address both

External validity Is your sample representative of the population?  Make sure your study population is relevant to the general population  Address by randomly sampling

Good research is about addressing  Internal validity  External validity

Clarification Randomly sampling cases gets you?  External validity Randomly assigning to treatment group?  Internal validity Controlling for variables with regression addresses?  Internal validity What study design addresses both internal and external validity?  Field experiments

What is gold standard research design? Field experiment, e.g., Connecticut voting turnout Why? Addresses  Internal validity Nonrandom selection into the treatment Reverse causation  External validity What aspects of our lives are governed by gold standard research? In this class, we mostly do observational studies,  But the key to a successful observational research is always keep in mind how one study differs from a field experiment 12

Next class: STATA Kohler & Kreuter, Data analysis (2 nd edition) Chapter 1  Skip section (linear regression) Chapter 3  Only read section 3.1 Chapter 5  Read section 5.1 but skip and  Read section 5.2 Handout: “How to use the STATA infile and infix commands” (course website) 13

If you want to play around with Stata Visit numerical for basic info about accessing Stata in Athena numerical Look on pp. xxi-xxii of Kohler & Kreuter for info about download example data 14