1. Issues and challenges around integrating qualitative and quantitative data Angela Harden, Professor of Community and Family Health InCQuuiRES, 8 th September, 2011

2. A centre for research, innovation and community engagement to promote health and wellbeing. 15 members of staff from a range of health and social science disciplines. Includes collaborative programme of research with Newham University Hospital NHS Trust (NUHT) focused on women, children and young people INSTITUTE FOR HEALTH AND HUMAN DEVELOPMENT

3. Overview of current SR projects TitleFunderMain collaborating institutions The effects of schools and school- environment interventions on health NIHRLSHTM, UEL, IOE Can specific approaches to community engagement help to reduce inequalities in health? NIHRIOE, UEL, LSE A meta-narrative review of conceptualisations and meanings of ‘community’ within and across research traditions AHRCUEL The acceptability, feasibility and transferability of interventions to improve early uptake of antenatal care NIHRUEL, IOE

4. Acknowledgements Colleagues at the EPPI-Centre, Social Science Research Unit, Institute of Education Co-convenors of the Cochrane Qualitative Research Methods Group

5. Outline Drivers and challenges of integration Review of approaches and methods Comparing methods Future challenges and issues

6. Drivers for integration Greater recognition of the value of qualitative research in evidence- based policy ‘Empty’ reviews Public perspectives and experiences Systematic reviews of complex interventions Issues of process and implementation Extension of evidence-based health care to other areas of public policy Growing tradition for integration in primary research (mixed methods research) Dedicated research funding for methodological research Establishment of dedicated methods groups (e.g. Cochrane Qualitative Methods Research Group)

8. Travelling east from Westminster, every two tube stops represent over one year of life expectancy lost –Data revised to 2004-08 Westminster Waterloo Southwark London Bridge Bermondsey Canada Water Canary Wharf North Greenwich Canning Town London UndergroundJubilee Line Differences in Male Life Expectancy within a small area in London Electoral wards just a few miles apart geographically have life expectancy spans varying by years. For instance, there are eight stops between Westminster and Canning Town on the Jubilee Line – so as one travels east, every two stops, on average, mark over a year of shortened lifespan. 1 River Thames 1 Source: Analysis by London Health Observatory of ONS and GLA data for 2004-08. Diagram produced by Department of Health Male Life Expectancy 73.6 (CI 71.9-75.2) Male Life Expectancy 78.5 (CI 75.5-81.6)

9. Mega-events and urban regeneration Examples of complex interventions with multiple anticipated outcomes Regeneration - a set of activities to reverse economic, social and physical decline Housing stock, service improvements, physical environment, creation of socially mixed population

10. McCartney et al. BMJ 2010; 340:c2369

11. McCarthy et al. (2010) conclude……. “There is little evidence that major multi-sport events held between 1978 and 2008 delivered health or socio- economic benefits for the population of the host country…..” “Future events such as the 2012 Olympic Games and Paralympic Games, or the 2014 Commonwealth Games, cannot be expected to automatically provide benefits. Until decision makers include robust, long term evaluations as part of their design and implementation of events, it is unclear how the costs of major multi-sport events can be justified in terms of benefits to the host population.”

12. Unanswered questions What happens during implementation? How do the various actors involved experience mega- sporting events and the associated regeneration? What are the pathways to impact? Do variations in the above relate to outcomes?

13. Challenges of integration New and evolving field Scale of task Paradigm wars Lack of training and expertise Few worked examples Infrastructure in development

14. Approaches and methods (1) Narrative summary Thematic analysis Grounded theory Meta-ethnography Meta-study Miles and Huberman's data analysis techniques Content analysis Case survey Qualitative comparative analysis Bayesian meta-analysis “Qualitising” “Quantitising” 2004

15. Approaches and methods ( 2 ) Meta- narrative mapping Critical Interpretive synthesis Realist synthesis Mixed methods synthesis Narrative synthesis Integration of qualitative and quantitative research Bayesian meta- analysis

16. Origins of approaches and methods MethodDeveloped by/ExemplarsContext and purpose Narrative synthesisPopay et al. (2005)Cochrane remit – to examine issues of process, implementation and experience Mixed methods synthesis Thomas et al. (2004) Harden and Thomas (2005) Informing policy to promote children’s health Bayesian synthesisRoberts et al. (2002)Factors that affect the uptake of childhood immunisation Critical interpretive synthesis Dixon-Woods et al. (2006)Access to healthcare for vulnerable groups Meta-narrativeGreenhalgh et al. (2005)To review research on diffusion of innovation to inform healthcare policy Realist synthesisPawson (2006)To develop and test theories of change underpinning complex policy interventions

17. Comparing approaches and methods MethodIdealist – realist continuum Deconstruct body of literature? Mixed methods lensOther characteristics Narrative synthesis RealistNoComplementary strengths stance Little iteration in methods Synthetic product aims to directly address policy Mixed methods synthesis RealistNoComplementary strengths stance and Dialectical stance Bayesian synthesis RealistNoAlternative or single paradigm stance Critical interpretive synthesis IdealistYesAlternative or single paradigm stanceIterative approach key Synthetic product requires interpretation Meta-narrativeIdealistYesDialectical stance Realist synthesisIdealistNoAlternative or single paradigm stance

18. In focus: Mixed methods synthesis Aim is to generate and test theory from diverse body of literature Exhaustive search, review questions, inclusion criteria and quality assessment largely specified a priori Each review typically has three syntheses: 1.Statistical meta-analysis 2.Thematic synthesis 3.Cross-study synthesis Recent examples Teenage pregnancy and social disadvantage (Harden et al., 2009) Knowledge and information needs of young people with epilepsy (Lewis et al., 2010) Behavioural interventions for weight management in pregnancy (Campbell et al., 2011) The effects of schools and school environment interventions (Bonell et al., 2011)

19. Mixed methods systematic reviews Working definition Combining the findings of ‘qualitative’ and ‘quantitative’ studies within a single systematic review, in order to address the same, overlapping or complementary review questions (Harden and Thomas, 2010)

20. Three ways in which reviews are mixed…. 1.The types of studies included and hence the type of findings to be synthesised (i.e. ‘qualitative/ textual and quantitative/numerical) 2.The types of synthesis method used (e.g. statistical meta-analysis and qualitative synthesis) 3.The mode of analysis: theory testing AND theory building

22. Review questions What is known about the barriers to, and facilitators of, healthy eating amongst children? Do interventions promote healthy eating amongst children? What are children’s perspectives on healthy eating? What are the implications of the above for intervention development?

23. REVIEW PROCESS Searching, screening and mapping Synthesis 1: Trials (n=33) 1. Quality assessment 2. Data extraction 3. Statistical meta-analysis Synthesis 2: Qualitative studies (n=8) 1. Quality assessment 2. Data extraction 3. Thematic synthesis Synthesis 3: Trials and qualitative studies Focus narrowed to ‘fruit &veg’

24. Children’s viewsTrials Recommendation for interventions Good quality Other Do not promote fruit and vegetables in the same way 00 Brand fruit and vegetables as an ‘exciting’ or child-relevant product, as well as a ‘tasty’ one 55 Reduce health emphasis in messages to promote fruit and vegetables particularly those which concern future health 56 Synthesis 3: Across studies

25. Increase (standardised portions per day) in vegetable intake across trials Little or no emphasis on health messages Synthesis 3: Across studies

26. In focus: Bayesian synthesis Aim is to test theory Findings from qualitative and quantitative research are ‘fused’ Only three worked examples to date Variation in weight given to the qualitative evidence Synthesis product is a set of weighted factors associated with/predicting the phenomenon under review Recent examples Factors affecting the uptake of childhood immunisation (Roberts et al., 2002) Factors that influence adherence to HIV medication regimes (Vollis et al., 2009) (Crandell et al., 2011)

27. Factors that influence adherence to HIV medication (Crandell et al., 2011)

28. In focus: Critical Interpretive Synthesis Aim is to generate theory from large and diverse body of literature Literature itself is an object of scrutiny (critical) Comprehensive search to identify sampling frame Purposive and theoretical sampling Analysis leads to generation of synthetic constructs and a synthesising argument Recent examples Access to health care for vulnerable groups (Dixon- Woods et al. 2006) Use of morphine to treat cancer related pain (Flemming, 2009) Nurses response to suicide and suicidal patients (Talseth and Gilje, 2011

29. Access to health care for vulnerable groups (Dixon- Woods et al. 2006) Based on 119 papers Interpretive qualitative analysis of diverse types of studies Conceptual and methodological problems with measures of health service utilisation Synthesising argument organised around a set of central concepts (e.g. navigation, adjudications) with the synthetic construct of ‘candidacy’ at the core

30. Access to health care for vulnerable groups (Dixon-Woods et al. 2006) Core construct – candidacy “Candidacy describes the ways in which people's eligibility for medical attention and intervention is jointly negotiated between individuals and health services…….candidacy is a dynamic and contingent process, constantly being defined and redefined through interactions between individuals and professionals, including how "cases" are constructed.” Central concepts Identification of candidacy Navigation Permeability of services Appearances at health services Adjudications Offers and resistance Operating conditions and local production of candidacy

31. In focus: Meta-narrative review Aim is to make sense of and understand diverse bodies of literature and their findings Literature itself is an object of scrutiny (critical) Searching is iterative, ‘snowballing’ a key technique Analysis leads to production of a set of meta-narratives (‘storylines of research’) Recent examples Spread and sustainability of innovations in health service delivery and organisation (Greenhalgh et al., 2005) Understanding the use of electronic patient records in health care organisations (Greenhalgh et al., 2009) See also the: Realist and meta-narrative evidence synthesis evolving standards project (RAMASES) (Greenhalgh et al., 2011)

32. A meta-narrative approach The influence of Kuhn’s ‘paradigms’ (1962) and the makings of the first meta-narrative The essential technique is interpretive synthesis exploring distinct research traditions, each with its own meta-narrative Methods of ‘unpacking’ the meta-narrative: exploratory methods; expert consultations; snowballing; database searching

33. Stages of a meta-narrative review (from Greenhalgh et al., 2009)

34. Meta-narratives identified in the electronic patient records review (from Greenhalgh et al., 2009)

35. Comparing approaches and methods MethodIdealist – realist continuum Deconstruct body of literature? Mixed methods lensOther characteristics Mixed methods synthesis RealistNoComplementary strengths stance and Dialectical stanceLittle iteration in methods Synthetic product aims to directly address policy Bayesian synthesis RealistNoAlternative or single paradigm stance Critical interpretive synthesis IdealistYesAlternative or single paradigm stanceIterative approach key Synthetic product requires interpretation Meta-narrativeIdealistYesDialectical stance

36. Future challenges More worked examples are key Focus on methods and tools for the actual integration – Enhancing transparency – Establishing rigour Further conceptual work to illuminate points of difference, strengths and weaknesses, fit for purpose Learning from, and contributing to, the mixed methods literature for primary research.

37. Thank you! a.harden@uel.ac.uk a.harden@uel.ac.uk Forthcoming book Gough D, Oliver S, Thomas J (Eds) (forthcoming, Feb 2012) An Introduction to Systematic Reviews. London: Sage