Quasi-Experiments: Good Enough for Social Science Political Science 30 Political Inquiry Quasi-Experiments: Good Enough for Social Science

Quasi-Experiments: Good Enough for Social Science The philosophy behind “quasi-” and “natural experiments” A classic example: Cracking down on Connecticut speeders Strengths and Weaknesses

The Philosophy of Quasi-Experimentation In a randomized experiment, scientists assign cases to control and treatment groups, and apply the treatment. In a“natural experiment,” Mother Nature assigns cases to control and treatment groups in some nearly random manner. In a“quasi-experiment,” scientists merely observe two or more groups of cases that have been treated differently.

The Philosophy of Quasi-Experimentation “Perhaps its fundamental credo is that lack of control and lack of randomization are damaging to inferences of cause and effect only to the extent that a systematic consideration of alternative explanations reveals some that are plausible.” – Campbell and Ross, 1968, p. 34. Translation: If no one can think of a confound, don’t worry, be happy.

The Philosophy of Quasi-Experimentation Step #1. Think hard about whether or not your groups of cases differ in their values of some confounding variable. Step #2. If there is a difference, try to make another comparison of two or more groups that are similar in every important variable other than the key IV. Step #3. Measure your DV and make time-series or cross-section comparisons.

A Classic Example In 1955, Connecticut had 324 traffic fatalities on its highways. Gov. Ribicoff cracked down on speeders by suspending their licenses in 1956. This change in policy is the “treatment.” In 1956, only 284 Connecticut motorists died, and Ribicoff declared victory.

A Classic Example: Time-Series Campbell and Ross compare CT before crackdown vs. CT after crackdown. Comparison plagued by differences between these groups (pp. 38-39): History Maturation Testing Instrumentation, Instability and Regression also at work (pp. 39-40)

A Classic Example: Possible Confounds History: Besides the treatment, other events take place over time. Maturation: Steady, long-term trends are also at work. Testing: The very act of measuring cases in a pre-test changes the cases.

A Classic Example: Cross-Section Compare trends in Connecticut with trends in adjacent, similar states. This eliminates threats to inference brought by many confounds. Gives us a way to judge instrumentation, instability and regression effects.

Strengths and Weaknesses Internal validity judges how well a research design has tested a causal relationship, in the cases examined. Quasi- and natural experiments fare worse on this criterion than lab experiments, because assignment is not truly random. Still, these are better than observational studies that compare very different groups.

Strengths and Weaknesses External validity judges how confident we can be that a causal relationship identified in our cases can be generalized to the outside world. Quasi- and natural experiments beat lab experiments on this count, because they take place in the real world. The limitation is that not every situation presents an opportunity for an experiment.