Basic issues in qualitative methods Qualitative methods reading group Carl Henrik Knutsen 30/6-2008

The literature King, Keohane and Verba, ch 1 George and Bennet, ch 1 Geddes, parts of ch 2

KKV Qual and quant: Different styles and techniques, but: one logic of inference Paying explicit and close attention to principles underlying logic of inference will generate more valid qualitative studies. Important assumption: Partial and imperfect knowledge about causation in the social world Descriptive and causal inference

KKV “Brilliant insights can yield interesting hypotheses, but brilliance is not a method”  method and social science The separation of theory, hypothesis deduction, and empirical validation is crucial to yield solid inference The role of variation and control Reporting the data collection process, openness and clarity, replicability Generating observable implications..the more the better: A case study is not a study with n=1. (temporal, spatial variation, and implications in different fields) “Maximizing leverage”: Explaining as much as possible with as little as possible. Not ==parsimony. Reporting uncertainty Skepticism and rival hypotheses.

G&B One logic? Epistemological: yes. Methodological: no. Disagreements with KKV: – Pays little concern to causal complexity – The DF-problem does not necessarily apply to case studies..”Deviant cases” and tests of necessity and sufficiency etc – Maximizing leverage and the dangers of concept stretching – Process tracing is not the same as increasing number of observable implications. The importance of sequence – The pedagogical and presentational aspect of KKV

G&B Strengths of different methods complement each other. The comparative advantages of case-study methods: – Conceptual validity – Deriving new hypotheses – Causal mechanisms Potential pit-falls – Case selection bias – Graded effects. How much? – Representativeness

Geddes Why has comparative political science generated so many sand-castles? Lack of explicit appreciation of fundamental scientific principles. The potential for cumulative research One practice that contributes to sand-castle generation: Trying to explain large and compound outcomes. “Grand claims” Rather a focus on hypotheses and TESTING. Single processes. Test argument on several cases  robustness. Derive different implications. Self-fulfilling prophecies: Testing arguments from a theory based on those very same cases. Throwing the baby out with the bathing water. What a theory is and how it is interpreted. Modernization theory. The problems of “Schools” and “School-identification”. Best kept to text- books? Debating arguments and not world-views.

Some virtues of qualitative studies Mechanisms and process tracing Intensive knowledge of subject Possibility of studying subjects that are hard to quantify Conjunctural causality and equifinality (necessity and sufficiency?)

Ragin’s QCA Boolean Algebra and truth tables Incorporate several cases Assess complex causal conditions that lead to outcome Shedding linearity

Ragin and inferences “Explaining all the variation”: Type I-errors What about uncertainty? How much do we really know about complex macro- phenomena? “If we had strong intuitions about how several causes can interact to produce an effect, there would be no need to rely on the mechanical procedure of “adding an interaction term” when an additive model fails. Yet because our intuitions are weak, we do not know what to really look for, and the tinkering with models seems the only alternative – at least if we retain the ambitious goal of providing law-like explanations. Given the dangers of tinkering, perhaps we should lower our ambitions instead” (Elster, 2007: 50)

A simple world Country XYABCZ

A simple world X or Z produces Y in a deterministic fashion. X also produces A, and B and C are factors that occur randomly in a country with a probability of 0,5. Neither A, B or C is connected to Y causally, but the social science researcher presuppositions leads him to identify A, B and C as causally relevant variables. The Boolean method would here lead the analyst to conclude that A + bC are the relevant causal pathways to generate Z. None are true When the number of cases is low, and the number of variables increases  The prob of type I error increases

Some problems facing inference The illusion of determinism How to deal with and describe uncertainty Selection bias The control problem Omitted variable bias Type I and Type II errors What is theory and what is empirical support. The dangers of grounded theory and tautologous testing “It depends on context”, “It’s the culture” etc: The temptation of ad hoc hypotheses. – The need for specification and stringency