1. DISMANTLING THE QUANTITATIVE – QUALITATIVE DIVIDE Comments On Hypothesis Testing, Induction, Statistics, Fiction And Epistemological Anarchy Presentation at the 3rd International Conference on Interdisciplinary Social Sciences, Prato, Italy, July 2008 Michael Wood (michael.wood@port.ac.uk) and Christine Welchmichael.wood@port.ac.uk Portsmouth University Business School, UK There is a revised draft paper at http://userweb.port.ac.uk/~woodm/QualQuant.pdf http://userweb.port.ac.uk/~woodm/QualQuant.pdf and this presentation is at http://userweb.port.ac.uk/~woodm/QualQuant.ppt

2. My background …

3. We suspect this presentation may be a bit of a mess because it’s largely about things which don’t make too much sense. So we’ll try and impose a clear framework …

4. What conceptual framework is best for understanding research? Perspective 1 or Perspective 2 or Perspective 3 or Perspective 4 Which do you think will be the winner?

5. Perspective 1: Two kinds of research Positivism –Generalisation through statistics –Research progresses through hypotheses and deductions –Observer independent –Large random samples –Etc, etc Social Constructionism –Generalisation through theoretical abstraction –Research progresses through rich data and induction –Observer part of study –Small purposive samples –Etc, etc Based loosely on Easterby-Smith et al (2002: 30). They list 8 dimensions. Other authors may use different labels for the two types: commonest probably Quantitative vs Qualitative.

6. The two kinds of research? Quantitative / positivist / deductive –Hard and spiky Qualitative / phenomenological / social constructivist / interpretivist / etc –Soft and cuddly Often suggested that researchers need to choose one or the other.

7. Which side are you on? To hard and spiky people, soft and cuddly research is lacking in rigour To soft and cuddly people, hard and spiky research is superficial and lacking in richness and relevance … but is this a genuine dichotomy?

8. No …

9. Hypothesis testingInductive StatisticalE.g. 1: Glebbeek & Bax (2004) E.g. 4: Moutafi et al (2007) http://tinyurl.com/2qb6p9 http://tinyurl.com/2qb6p9 “Qualitative” / Non-statistical E.g. 3: Meyer & Altenborg (2007) E.g. 2: Britten et al (2000) Perspective 2: More than two types of research: e.g. …. Easterby-Smith et al (2002) list 8 bipolar dimensions: this leads to the possibility of 2 8 or 64 types of research.

10. Glebbeek and Bax (2004) To test the hypothesis “that employee turnover and firm performance have an inverted U-shaped relationship: overly high or low turnover is harmful.” They analysed the performance of “110 offices of a temporary employment agency” by the statistical technique of regression, and did in fact find the hypothesized curvilinear relationship. Britten et al (2000) To identify categories of “misunderstandings in prescribing decisions in general practice”. They used a series of interviews with patients and doctors to identify fourteen categories of “misunderstandings”. These categories emerged from the data and were viewed as illustrating important possibilities with implications for the training of doctors. However, no attempt was made to estimate how common each category was – just the fact that they had occurred, and so it could reasonably be assumed that they might occur again, was sufficient Meyer and Altenborg (2007) To investigate the impact that a “spirit of equality or balance” has on the merger process. The authors used a case study of a failed international merger. They found that “the principle of equality had the reverse effect on social integration to that predicted in the literature … negatively influencing social integration.” Moutafi et al (2007) To investigate the relationship between personality and managerial level. They got 900 participants to complete two personality tests and looked at the relationship between the results and their managerial level. There were four hypotheses – the first of which was that “conscientiousness should be positively correlated with level of management” – but these hypotheses were not mentioned in the abstract, which simply listed personality traits which were found to correlate – positively or negatively – with managerial level.

11. What does qualitative mean? General term from right hand side of Perspective 1 ………? Opposite of statistical ……… ? Focussed on qualities not quantities … ? Detailed information / in depth analysis (eg not superficial questionnaire) …… OK

12. But … Why not use different approaches in one project? –E.g. Glebbeek and Bax (2004) found statistical evidence for the hypothesised inverted U shaped relationship between staff turnover and firm performance, but why not back this up with case studies to look at possible reasons for this effect? –Britten et al (2000) identified categories of misunderstanding between doctors and patients, but why not do a statistical survey to see how common each category is? So …

13. Perspective 3: Multimethodology or Mixed methods Use different approaches in one project …

14. But … There are problems with many of the bipolar dimensions used to pigeonhole research –Concepts used may be vague, ambiguous or prone to misinterpretation –May not be bipolar: reasonable approaches may be omitted

15. Two bipolar dimensions? Statistical “Qualitative” Hypothetico-deductive“Inductive” “ …” indicates vague terms we don’t like

16. Or … do these fit on the same dimension? Statistical “Qualitative” X Empirical possibilities Deterministic laws Fictional possibilities ? ? Is this one dimension?

17. Extended statistical – non- statistical dimension Deterministic laws – what always happens Statistics – what sometimes happens Illustrative inference – what has happened at least once (some “qualitative research”) Fiction – what is possible / imaginable

18. And what about this … ? Hypothetico-deductiveInductive Deductive (e.g. applying a model) Using a framework or paradigm to define questions Much research is neither hypothetico-deductive nor inductive. There is no obvious linear dimension here, which is why we’ve made the layout of this slide a bit of a mess. So … X

19. Fourth perspective Avoid unhelpful concepts. If in doubt, shut up, or use more straightforward language! –Induction ? –Qualitative ? –etc No useful general categorisation schemes for research (=only useful grand narrative) –If you stick to such a scheme you risk ignoring useful possibilities –Epistemological anarchy. (= Postmodernism?) Feyerabend: “anything goes” But some concepts are worth careful thought …

20. Statistical methods Important approach –Over-estimated by proponents, underestimated by opponents Focus on null hypothesis tests is usually stupid –Better to measure size of effect –E.g. Glebbeek and Bax (2002) –Statistics = formal methods for doing induction Sampling / context needs care –E.g. Glebbeek and Bax (2002) “Qualitative” data often analysed statistically –Should be done properly

21. Following a framework or paradigm Neither hypothesis testing nor (pure) induction Kuhn’s normal science Obviously a good idea but not in the standard menu of approaches More helpful concept than induction because focuses attention on the framework and presuppositions

22. Hypotheses Guiding vs formal Formal hypotheses are tested. They may be –Null Sadly, statistical null Hs tend to dominate idea of hypotheses. –Non-null. Popper’s bold conjectures. Require imagination. Not boring! –Rigour is in the testing process Guiding hypotheses are explored Not restricted to statistical approaches. In fact hypotheses are usually best avoided with statistics

23. Fictional “data” Made up data may be more convenient –E.g. confidentiality problems –Thought experiments Fictions, fables, utopias, dystopias to explore … Widely used in mathematical modelling –Interesting possibilities fabricated to play out what-ifs Sometimes what is possible may be more interesting than what has actually happened –E.g. if we are interested in improving things –Dogmatic empiricism may be unreasonable? If we want to change things why focus exclusively on facts?

24. Does all this matter? We think so because … Avoids impoverishing research by adhering to very restricted perspectives Suggests new possibilities Avoids wasting time talking rubbish Fitting methods to the enquiry is important (e.g. check the two CRITIC acronyms in http://userweb.port.ac.uk/~woodm/rm/rm.ppt ) http://userweb.port.ac.uk/~woodm/rm/rm.ppt Fitting them to your favourite paradigm is not! All comments and suggestions welcome. These slides and a revised draft paper are at –http://userweb.port.ac.uk/~woodm/QualQuant.ppthttp://userweb.port.ac.uk/~woodm/QualQuant.ppt –http://userweb.port.ac.uk/~woodm/QualQuant.pdfhttp://userweb.port.ac.uk/~woodm/QualQuant.pdf