Mixed methods synthesis ESRC Methods Festival 2006 James Thomas Institute of Education, University of London

18 July 2006(2) Background to mixed methods approach Policy and practice concerns often precede, or go beyond, questions of effectiveness. Different types of questions require different combinations of study types to be included. Different combinations of study types demand different methods of synthesis. However, key principles of systematic reviews are not compromised.

18 July 2006(3) Case example of a mixed methods synthesis What is known about the barriers to, and facilitators of, healthy eating amongst children?* *The full report of this review is available at the EPPI-Centre website: Thomas J, Sutcliffe K, Harden A, Oakley A, Oliver S, Rees R, Brunton G, Kavanagh J (2003a) Children and Healthy Eating: A systematic review of barriers and facilitators. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

18 July 2006(4) Review question e.g. What is known about the barriers to, and facilitators of, fruit and vegetable intake amongst children aged 4 to 10 years? Trials (N=33) 1. Application of inclusion criteria 2. Quality assessment 3. Data extraction 4. Synthesis using statistical meta-analysis Views studies (N=8) 1. Application of inclusion criteria 2. Quality assessment 3. Data extraction 4. Thematic synthesis Trials and views Mixed methods synthesis SCOPING AND MAPPING (of 272 reports of 193 studies) Review process

18 July 2006(5) One review with three syntheses 1.Quantitative methods used to conduct a meta- analysis of data from trials. 2.Qualitative methods used to synthesise textual data from views studies (aided by NVivo). 3.Both qualitative and quantitative methods used to combine experimental studies of effectiveness (trials) with studies of peoples views (descriptive).

18 July 2006(6) Quality-assessment methods SYNTHESIS 1 Quantitative (Trials) Provision of pre- and post- data on outcomes Provision of data on all outcomes measured Employment of equivalent control/comparison group Resulted in high, medium and not sound/ low trials SYNTHESIS 2 Qualitative (Views) Quality of reporting (5 items) Sufficiency of strategies for reliability/validity (4 items) Extent to which study findings were rooted in childrens own perspectives (3 items)

18 July 2006(7) Methods for synthesis 1: Quantitative (Trials) Effect sizes from trials pooled using: –Statistical meta-analysis –Six different outcomes Heterogeneity across studies explored via: –Sub-group analysis –Qualitative analysis of textual data from trials

18 July 2006(8) Findings for synthesis 1: Quantitative (Trials)

18 July 2006(9) Methods for synthesis 2: Qualitative (Views studies) Stage 1: Thematic analysis of textual data (study authors descriptions of findings) = Descriptive themes Stage 2: Descriptive themes examined in light of review question = Barriers, facilitators and recommendations for interventions Stage 3: Thematic analysis revised = Analytical themes

18 July 2006(10) Findings of synthesis 2: Descriptive themes Influences on foods eaten Chosen foods Provided foods Foods in the home Foods in the school Parental influence and food rules Breaking rules Limited choice Social occasion Contradictions Food preferences Perceptions of health benefits Knowledge-behaviour gap Roles and responsibilities Non-influencing factors Concepts of healthy eating Healthy eating concepts Good and bad foods Health consequences

18 July 2006(11) Findings of synthesis 2: Analytical themes, barriers, facilitators, and interventions implications Brand fruit and vegetables as tasty rather than healthy. Reduce health emphasis of messages Do not promote fruit and vegetables in the same way within the same intervention. Create situations for children to have ownership over their food choices. Ensure messages promoting fruit and vegetables are supported by appropriate access to fruit and vegetables 1) Children dont see it as their role to be interested in health. 2) Children do not see future health consequences as personally relevant or credible. 3) Fruit, vegetables and confectionary have very different meanings for children. 4) Children actively seek ways to exercise their own choices with regard to foods. 5) Children value eating as a social occasion. 6) Children recognise contradiction between what is promoted and what is provided.

18 July 2006(12) METHODS FOR SYNTHESIS 3: ACROSS STUDY TYPES Product of thematic synthesis of views studies was the mechanism to combine trials and views studies. Conceptual and methodological matrix juxtaposed findings from views synthesis against findings from trial synthesis. Comparative analysis guided by 3 questions: –Which interventions match recommendations derived from childrens views and experiences? –Which recommendations have yet to be addressed by soundly evaluated interventions? –Do those interventions which match recommendations show bigger effect sizes and/or explain heterogeneity?

18 July 2006(13) Synthesis 3 (Across Study Types) Childrens ViewsTrials Recommendation for interventions Good qualityOther Do not promote fruit and vegetables in the same way None Brand fruit and vegetables as an exciting or child-relevant product, as well as a tasty one 55 Reduce health emphasis in messages to promote fruit and vegetables particularly those which concern future health 56

18 July 2006(14) Synthesis 3: Sub-group Analysis Increase (standardised portions per day) in vegetable intake across trials Little or no emphasis on health messages

18 July 2006(15) CONCLUSIONS Conceptually, our method allows us to integrate quantitative estimates of benefit and harm with qualitative understanding from peoples lives. Technically, the insights gained from the qualitative synthesis of views studies allows the exploration of heterogeneity in ways in which it would be difficult to imagine in advance. Our approach has raised a number of methodological and conceptual challenges…

18 July 2006(16) CHALLENGES To notions of defining sub-group analyses a priori. To public health interventions developed and trialled solely by experts. To traditional polarisation of qualitative and quantitative research (bridging paradigms).