More About Factorial Design Suppose experiment tests whether taking a game theory class causes a person to get more papers published We cannot simply examine whether people who took a game theory class had more papers published due to possible effect of self-selection into game theory class. Therefore, we should randomly assign people to game theory class.

Does Game Theory Lead to More Articles? DV: Number of papers published 4 years later IV: Took game theory class or not Two groups: Take game theory class Did not take game theory class

Does Class on Experiments Lead to More Articles? We also may want to determine whether taking a class on experiments helps a person get more articles published. Now we randomize whether a person takes a class on experiments

2 x 2 Factorial Design Class Taken Game Theory No Game Theory Experiments No Experiments Now we Have 4 groups in the experiment, randomly assigned. Each cell should have equal numbers of subjects

2 x 2 x 3 Factorial Now we randomly assign whether a person studies international relations, public policy, or economics. This is now a 2 x 2 x 3 factorial design with 12 groups

Economics Game Theory No Game Theory Experiments No Experiments Public Policy Game Theory No Game Theory Experiments No Experiments International Relations Game Theory No Game Theory Experiments No Experiments

Power Analysis: How Many Subjects Needed? Example: G*Power (free!) Hypothesize the magnitude of the effect (such as one additional paper published) Calculate sample size needed for two independent samples difference in means (average number of articles published for each of two groups) Answer will be number of subjects needed in each group

Exploiting Randomization When It Occurs Natural Experiments Exploiting Randomization When It Occurs

Natural Experiments Experimenter does not pre-plan experiment Experimenter finds a naturally-occurring random assignment mechanism that serves as a treatment

Why Randomization? Randomization allows comparison across groups by controlling for differences in other variables. The expected values of other variables are the same between across randomly-assigned treatment groups. If other variables have equal expected values across treatments, then differences in outcomes must be due to treatment variable

Dartmouth Roommate “Experiment” First-year roommates at Dartmouth College assigned randomly within “like-groups” based on gender and whether student Smokes Listen to music while studying Stay up late Are neat or messy Assignment random within 25=32 different groups

Treatment=Roommate In this case, the treatment variable is a student’s roommate The experiment does not have a true “control group” but many different treatments in the form of roommates Also in this case, the roommates are “treatments” for each other

First Step: Is Assignment Truly Random? Are there any variables that determine one’s assigned roommate other than Gender Smoking Late night habits Cleanliness Listening to music

DV’s and IV’s The DV in this case is a student’s GPA The most important IV is the roommate’s GPA Other IV’s included as control variables are a student’s own academic index (entrance exam scores and high school grades) and roommate’s academic index

Statistical Model

Conclusions Roommates influence each other’s performance in school Top performers reinforce each other Bottom performers reinforce each other Roommates also influence choice of social group such as fraternity or sorority Roommates do not influence each other’s choice of major

Other Examples of Natural Experiments