1. Introduction: Research design 17.871 Spring 2012 1

2. The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 2

3. The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 3

4. The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 4

5. The Biggest Problem in Research: Establishing Causality Return to the case of voting machine problems in Florida After the election, we wanted to know: are some machines “better” than others? For the policy choice, we want to know if 5

6. The problem How do we make sure that quality differences observed among machine types are due to machine types per se  This is an issue of causality  We attend to “internal validity” so that when we observe differences between groups, we can assure ourselves that this is because of the “treatments” of interest 6

7. Review of internal and external validity

8. Internal validity: the two problems The two primary threats to internal validity. 1. Nonrandom selection into the treatment group (confounding variables)  Comparing apples with apples or apples with oranges?  Random assignment ensures apple to apple comparisons  Regression, matching, difference-in- differences also attempt to compare apples with apples 2. Reverse causation  The chicken and egg problem, which came first?  Is your dependent variable influencing your treatment (your explanatory variable)? If you can address these problems, you almost always have an internally valid study Randomly assigned experiments address both

9. External validity Is your sample representative of the population?  Make sure your study population is relevant to the general population  Address by randomly sampling

10. Good research is about addressing  Internal validity  External validity

11. Clarification Randomly sampling cases gets you?  External validity Randomly assigning to treatment group?  Internal validity Controlling for variables with regression addresses?  Internal validity What study design addresses both internal and external validity?  Field experiments

12. What is gold standard research design? Field experiment, e.g., Connecticut voting turnout Why? Addresses  Internal validity Nonrandom selection into the treatment Reverse causation  External validity What aspects of our lives are governed by gold standard research? In this class, we mostly do observational studies,  But the key to a successful observational research is always keep in mind how one study differs from a field experiment 12

13. Next class: STATA Kohler & Kreuter, Data analysis (2 nd edition) Chapter 1  Skip section 1.3.19 (linear regression) Chapter 3  Only read section 3.1 Chapter 5  Read section 5.1 but skip 5.1.3 and 5.1.4  Read section 5.2 Handout: “How to use the STATA infile and infix commands” (course website) 13

14. If you want to play around with Stata Visit http://ist.mit.edu/services/software/athena/ numerical for basic info about accessing Stata in Athena http://ist.mit.edu/services/software/athena/ numerical Look on pp. xxi-xxii of Kohler & Kreuter for info about download example data 14