Qualitative Research Design Qualitative vs. Quantitative Research Two types of observational study Nonrandom case selection Selecting Cases on the Independent Variable Most similar systems Most different systems Income Inequality and Civil War

Qualitative vs. Quantitative Research Qualitative and quantitative studies are both types of observational studies. Quantitative research measures differences in number for variables, and usually studies a large number of cases (Large “N”). Qualitative research measures differences in kind for variables, and usually studies a small number of cases (Small “N”).

Qualitative vs. Quantitative Research Because it covers a broad range of cases, quantitative research yields conclusions that can be generalized (it has the strongest external validity). Because it looks closely at a few cases and traces causal pathways, qualitative research often outperforms quantitative research in its measurement validity and internal validity.

Qualitative vs. Quantitative Research When selecting cases for your quantitative research sample, it is imperative that you use random selection. In qualitative research, “selection must be done in an intentional fashion, consistent with research objectives and strategy.” (King, Keohane, and Verba, 1994, p.139)

Selecting Cases on the Independent Variable “Selecting on the independent variable” means “selecting your cases according to the values of the independent variable that they take on.” In order to do this, you have to know a little bit about all of your potential cases. In order to do this right, you cannot act as if you also know the values that the dependent variable takes on.

Selecting Cases on the Independent Variable The Most Similar Systems method selects cases that take on similar values of confounding variables, but different values of a key independent variable. This “holds constant” the confounds because they take on the same values in all of the cases. This is the design recommended by King, Keohane, and Verba.

Selecting Cases on the Independent Variable Most Similar Systems is: A NEGD with a treatment and comparison group. N O X O N O O

Selecting Cases on the Independent Variable The Most Different Systems method selects cases that take on very different values for multiple independent variables. If it turns out that these cases all take on the same value of a dependent variable, then we can rule out the independent variables as causes of the dependent variable. Less useful since it can only disprove a hypothesis.

Income Inequality and Civil War Income Inequality Poverty Civil War Colonial Past External Threat

Income Inequality and Civil War CaseIncome Inequality PovertyColonial Past External Threat Costa Rica ModerateYesYupNope El Salvador HighYesYupNope CubaHighYesYupNope

Income Inequality and Civil War We can hold the confounds constant by selecting these similar cases from Latin America. It appears that income inequality does lead to civil war. CaseCivil War? Costa RicaNo El Salvador Yes CubaYes

Can be a powerful research design when it is difficult or costly to study a large number of cases. When carried out correctly, can be internally valid. Do not need a large number of cases for a proper test. Implicit foundation for “area studies.” Belief that regions share many similarities, and that these similarities are related to similar outcomes (weak test) and not related to dissimilar outcomes (stronger test).

How Similar is Similar? In most similar designs, which covariates should you try to match? Similar but irrelevant covariates do not add anything to the test. Likewise, dissimilar but irrelevant covariates do not detract from the test. Both reduce degrees of freedom. Covariates that are related to both the treatment and outcome variables must be included whether similar or not – otherwise, omitted variables bias.

Qualitative Research Design II Selecting on the Dependent Variable Mills’ Method of Agreement Mills’ Method of Difference Example Dreze and Sen

Selecting on the Dependent Variable Selecting cases according to the value of the dependent variable that they take on is more controversial than selecting on the independent variable. It allows you to look at extreme values or divergent cases. “However, if this design is to lead to meaningful … causal inferences, it is crucial to select observations without regard to values of the explanatory variables. K.K.V.”

Selecting on the Dependent Variable: Method of Agreement When you use Mills’ Method of Agreement, you select cases that take on the same values of the dependent variable. This helps you to rule out possible causes, because independent variables that vary over these cases can’t cause the dependent var. This method can only disprove a hypothesis, because it can’t find a correlation.

Selecting on the Dependent Variable: Method of Agreement This design could help us rule out “early industrialization” as a cause of whether a country has a viable socialist party. CaseEarly Industrialization? Viable Socialist Party? FranceNoYes BritainYes

Selecting on the Dependent Variable: Method of Difference When you use Mills’ Method of Difference, you select cases that take on different values of the dependent variable. After you have selected your cases, you determine what values they take on for some independent variables. Perhaps one independent variable will vary across your cases, and explain the D.V.

Selecting on the Dependent Variable: Method of Difference Adding a country that has no viable socialist party can add causal leverage to our early investigation. CaseEarly Indust.?Feudalism?Viable Social Party? FranceNoYes BritainYes USAYesNo

Example: Dreze and Sen Both countries began a new political regime at mid-century with large populations and little wealth. They have diverged since then: “There is little doubt that as far a morbidity, mortality, and longevity are concerned, China has a large and decisive lead over India. (p. 205)” “What has brought about that lead is a matter of very considerable interest. (p. 206)”

Eckstein’s Crucial Case Studies Most likely case studies: if theory holds anywhere, it should hold in this case. Failure to support theory counts disproportionately against theory. Least likely case studies: if theory works in this case, it should work in all cases. Support for theory counts disproportionately in favor of theory. If a case is more or less likely, theory contains unstated premises.

Potentially Valid Single Case Designs That are nonetheless seldom used in political science.

Non-equivalent Dependent Variables Design Theory predicts treatment effect on one outcome variable but not on another similar outcome variable. O 1 X O 1 O 2 Other outcome serves as “comparison group.” Strength comes from “pattern matching” across different outcomes.

Interrupted Time Series Single case observed over time, pre- and post- treatment (aka: regression point displacement). OOOOOXOOOOO Analogous to a most similar design, with the case as its own “comparison group.” The narrower the window around the treatment effect, the more powerful the test. When to start and stop the series can be problematic. Need good estimate of functional form.

Problems Frequently Encountered in Case Study Research

Threats to Internal Validity History: Many case studies are conducted over time. Need to consider other variables/events that may affect outcome. Maturation: Again, change over time within the cases selected is likely to confound results. Need to model functional form. Testing: Since case studies are typically given by nature, not likely to be a threat; but if pre- and post-tests are administered, testing threats may exist.

Threats to Validity, continued Instrumentation: Often a problem. Cases selected because data is costly or difficult to obtain. Typically present verbal descriptions of the variables. Operationalization is even more important than in large-n designs. Regression: Possible, especially if case selected is a prominent “outlier.” Mortality: unlikely, since cases intentionally selected.

Selection Bias In case studies, we intentionally select some (small) number of cases for our sample. If we select cases from limited ranges of the outcome variable, we “truncate” that variable and introduce selection bias. Truncating the outcome variable produces (on average) an underestimate of the treatment effect.

Example of Selection Bias KKV give example of business school student who wants a high paid job and selects for his study sample only those graduates earning high salaries. He then relates salary to number of accounting courses. By excluding graduates with low salaries, he paradoxically underestimates the effect of additional accounting courses on income.