Designing Social Inquiry Determining What to Observe: Things That We Should Be Careful about When We Design Researches Jaechun Kim

PROBLEM OF INDETERMINATE RESEARCH DESIGN What is research design? Design that shows how we are going to use our evidence to evaluate hypotheses… What is indeterminate research design? Nothing can be learned about the causal hypotheses…

Two cases of indeterminate research design More inferences than observations – Problem of too many variables and too few cases! Why does this lead to indeterminate research design? P. 119 – What explains successful joint collaboration on defense projects? 3 cases 7 IVs … why is it a problem? Too many variables, but too few cases…!

Mathematical model, see p. 121

Multicollinearity… What is it?? p. 122 Punch Kick Ability to Fight a Good Fight What if two IV s are perfectly correlated? Mathematical model, p. 123

THE LIMITS OF RANDOME SELECITON Random selection is best, but usually not feasible in social science… Why? Universe of cases is not known…? P. 125 You may fail to examine important cases… if you rely on random selection… Problem of “small n”  inherent problem of selection bias… so be careful! Careful non-random selection!! P. 126 good example!

TO AVIOD SELECTION BIAS Allow variation on your DV of possible! If possible… The dogs that didn’t bark… !!

EXAMPLES OF SELECTION BIAS P. 128 American investment  Internal violence Cases chosen: Countries with major American investments and good deal of violence Countries with no American investment and no violence What’s wrong with this case selection?

What determines presidential involvement in foreign policy decisions in the US? P. 132 Cases chosen? Policy decisions on which information about the president’s participation in meeting is available What’s wrong with this? Selection rule is correlated with low levels of presidential involvement!

Michael Porter’s case: What are the sources of “competitive advantage” for industries and firms? p Cases selected: Ten states which compete successfully in a range of such industries What’s wrong with this? Selection on DV…

Literature on deterrence theory: Deterrence: The use of threats to induce the opponents to behave in desirable ways… Students of deterrence have often examined “acute crises” in which deterrence have failed… What’s wrong with this research design and case selection? Dogs that didn’t bark…!

‘The dog that didn’t bark’ is an expression from a Sherlock Holmes mystery. It was an important clue that led to identifying the criminal. It seems that the killer entered and left the estate grounds one night but without the guard dog barking an alarm at the intruder’s presence as expected. From this non-event Holmes reasoned that the dog must have known the killer and that clue led to solving the case. Arthur Conan Doyle’s story, The Hound of the Baskervilles

P. 140 Snyder’s work… evaluate!! Myth of Empire (1991) P. 144 Kohli’s work… evaluate!!

Snyder’s puzzle: Why do states tend to over- expand? Before presenting his own explanation of this puzzle, Snyder summarizes the answers given by other schools of thought (from handout): Snyder's explanation of this puzzle: a different domestic political explanation:

Realist Explanation Cognitive Explanation Domestic Political Explanations

Snyder’s explanation of this puzzle - a different domestic political explanation  Coalition Logrolling and Coalition Ideology: Snyder claims that the myths of empire arise out of coalitions of interests that logroll (trade) various policy choices and use a nationalist/expansionist ideology to justify expansionist policies.

e.g. authoritarian rules induce high economic growth Cases selected: authoritarian rules – high econ growth and non-authoritarian rules – low econ growth What’s wrong with this?

e.g. Inkeles and Rossi p. 146 Industrialization  a particular prestige (attached to occupations) hierarchy Cases: a set of countries that was quite varied except that they all were industrialized!! What’s wrong with this case selection??

Some research designs are good for generating hypothesis, but not evaluating it…! Admit your limits!! Sometimes researchers select observations so the DV is constant e.g. Center for Disease Control (CDC) examining ‘cancer cluster’ p. 148 cf. political cluster What is use of it?

*Methodological Critiques of Others’ Works… *Helen Milner’s work…pp. 179~182