1. Introducing framework synthesis and thematic synthesis Series of doctoral lectures on behalf of the Cochrane Qualitative and Implementation Methods Group http://methods.cochrane.org/qi/ Systematic Reviews in social sciences. Mixed methods and qualitative research reviewed. Doctoral course, Ughent, March 21-22, 2016 Prof. Dr. Karin Hannes Centre for Sociological Research Social Research Methodology Group, KU Leuven Karin.hannes@kuleuven.be Dr. Chris Carroll and Prof. Dr. Andrew Booth Scharr, University of Sheffield, UK

2.  (Thematic synthesis); Critical Interpretive Synthesis; Meta-ethnography 1.Only include “good” qualitative studies (?) 2.Constant comparison; iterative; interpretations generated from the data by reviewers 3.Create a theory Inductive (theory-generating)  Examples:  Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology 2008; 8.  Campbell R et al. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Social Science & Medicine 2003; 65:671-684. Methods of qualitative evidence synthesis

3.  Framework synthesis: 1.Only include “good” qualitative studies (?) 2.Map data from included studies onto an existing framework to test the framework/theory (a role for theory) 3.Test the “fit” of the theory to the question Deductive (theory-testing)  Examples:  Oliver S et al: A multidimensional conceptual framework for analysing public involvement in health services research. Health Expectations 2008, 11:72-84.  Brunton G, Oliver S, Oliver K, Lorenc T. A Synthesis of Research Addressing Children’s, Young People’s and Parents’ Views of Walking and Cycling for Transport London. London, EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2006. Methods of qualitative evidence synthesis

4.  “Best-fit” framework synthesis 1.Identify relevant pre-existing conceptual models or frameworks 2.Identify and extract all relevant qualitative studies satisfying review’s inclusion criteria (no exclusion based on quality) 3.Code data from included studies against framework 4.Use secondary thematic analysis/synthesis to generate completely new themes to supplement the framework’s themes 5.Create new framework and conceptual model or theory  Deductive and Inductive  Framework and Thematic/interpretive synthesis  Carroll C, Booth A, Cooper K. A worked example of “best-fit” framework synthesis: A systematic review of views concerning the taking of potential chemopreventive agents, BMC Medical Research Methodology 2011; 11: 29 Methods of qualitative evidence synthesis

5.  “the output of some methods of synthesis (Thematic Synthesis, textual Narrative Synthesis, Meta-aggregation, Framework Synthesis, and ecological triangulation) is more directly relevant to policymakers and designers of interventions”  Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009; 11;9:59.  Framework and thematic synthesis:  Methods for qualitative evidence synthesis recognised to be of particular value to policy-makers; for solving or addressing service problems (rather than simply generating concepts and theories)  Thorne S. Toward Methodological Emancipation in Applied Health Research, QHR 2010; 21(4):443-44. Methods of qualitative evidence synthesis

6. Framework synthesis Thematic synthesis

7. The case study The question: What are the barriers and facilitators of patient autonomy post stroke ? The studies: 1.Proot I et al, Patient autonomy during rehabilitation: the experiences of stroke patients in nursing homes, International Journal of Nursing Studies, 2000; 37: 267-76. 2.Mangset M et al, ‘We’re just sick people, nothing else’:... Factors contributing to elderly stroke patients satisfaction with rehabilitation, Clinical Rehabilitation, 2008; 22: 825-35.

8. Data extraction  What is it?  An attempt to reduce a mass of material (your included papers) to a much smaller body of text and numbers, amenable to analysis and the interpretation of findings

9. Data extraction form  Location  Setting  Sample (n)  Age  Gender  Ethnicity and other relevant variables (e.g. Marital status)  Socio-economic status  Study design  Intervention (if any)  Quality assessment criteria  Results?  Further citations  Format?????

10. Example form

11. What results do you extract?  What is your question?  Question: What are the barriers and facilitators of patient autonomy post-stroke? Keep the question in mind as you read: Are the data relevant to this question? Is the question answered by the data?

12. Extraction of results  Results/findings sections of included studies:  Verbatim quotations and/or  Authors’ statements clearly-supported by data  “... It was often difficult to distinguish [Schutz’s] first- from second-order constructs” Atkins et al. Conducting a meta-ethnography of qualitative literature: Lessons learnt, BMC Med Res Methodol 2008; 8: 21 Schutz A. Collected Papers 1971; vol.1: 361.

13. From: Mangset p.829

16. Extracting data for thematic analysis

18. Extracting or “coding” data using framework analysis  Identify or create a thematic framework or conceptual model  Develop own framework or themes or concepts for coding  Kirkevold M et al. Promoting psychosocial well- being following a stroke: developing a theoretically and empirically sound complex intervention, International Journal of Nursing Studies, 2012; 49: 386-97.

19. Extracting or “coding” data using framework analysis Code numberThemeDefinition 1DignityA self-concept characterised by self-valuing, self- acceptance, usefulness and belief in oneself 2Well-beingA basic mood of joy, pleasure and wellbeing; the absence of feelings of emptiness or sadness 3CopingAbility to adapt and cope within home and hospital settings 4Family SupportRelations with immediate family 5Support for patient’s coping efforts Patients take responsibility and are supported by others in this 6Bodily changes and impairments Difficulties generated by residual or post-stroke conditions Handout 1

20. Extracting or “coding” data using framework analysis

24. What now?  You have extracted your data  You may have coded your data against your framework (framework analysis)  You may have only extracted relevant data (without using a framework) (the first stage of secondary thematic analysis)  You may have coded some of your data but not others (“best fit” framework synthesis)  You now need to assign themes to those data without themes, i.e. Using secondary thematic analysis (data reduction, display, conclusion)  How do you do it?

25. Generating new themes  “Evidence does not speak for itself. It requires interpretation.... on the basis of our personal experience and a range of ideas that we use to make sense of our observations”  Kelly M et al. Evidence-based public health: A review of the experience of NICE developing public health guidance in England, Social Science and Medicine, 2010; 71: 1056-62.

26. Developing new themes C. “…“I will take that paper when P. is finished with it. I want to study it. Then I'll take my shoes off, and practise as I see it in the picture” (p04b, p. 11), Proot, p.271 Other / New theme?  Interpret and reflect  Create  Revisit and revise  Discuss  Agree... = Learning self-help skills / guided self-determination

27. Synthesis

29. A new framework Code numberThemeDefinition 1Dignity A self-concept characterised by self-valuing, self- acceptance, usefulness and belief in oneself 2Well-being A basic mood of joy, pleasure and wellbeing; the absence of feelings of emptiness or sadness 3Coping Ability to adapt and cope within home and hospital settings 4Family Support Relations with immediate family 5Support for patient’s coping efforts Patients take responsibility and are supported by others in this 6Bodily changes and impairments Difficulties generated by residual or post-stroke conditions 7Self-determination Define theme as a concept, based on data 8Need for information Define theme as a concept, based on data 9Lack of shared decision- making Define theme as a concept, based on data 10Acceptance of therapy Define theme as a concept, based on data

30. A new framework and synthesis ThemeCategories Dignity Psychological factors Well-being Self-determination Coping Physical factors Bodily changes and impairments Need for information Interaction with professionals and family Lack of shared decision-making Family Support Support for patient’s coping efforts Acceptance of therapy

31. Synthesis  Alternatively, go beyond the framework to create a new conceptual model or theory...  Revisit data to explore the relationships between the themes or findings of your framework  The “richer” or “thicker” the data, the deeper the resulting model or theory  Narrative and diagrammatic representations

32. Do you stop there?  Test your synthesis?  Disconfirming cases  Booth A, Carroll C, Illott I. Desperately Seeking Dissonance: Identifying the “Disconfirming Case”, Qualitative Evidence Synthesis, Qualitative Health Research 2012 (in press).  Morse, J. M. The significance of saturation. Qualitative Health Research 1995; 5, 147- 149.  Sensitivity analysis  By quality, population, location, setting etc.  By frequency and “thickness”  Carroll C, Booth A, Lloyd-Jones M. Should we Exclude Inadequately-reported Studies from Qualitative Systematic Reviews? An Evaluation of Sensitivity Analyses in Two Case Study Reviews, Qualitative Health Research 2012; 22(10)  Boeije et al (2011) Making a difference: towards a method for weighing the evidence in a qualitative synthesis, Journal of Evaluation in Clinical Practice 2011 17(4):657-63

33. What is the purpose of your review and synthesis?  To illuminate experiences of clients, professionals, policy makers,… and to develop theory  To be useful  To answer questions and to solve problems that cannot be addressed by quantitative research

34. Conclusions  Different methods for qualitative evidence synthesis  Choose an appropriate method  Extract and interpret data  Synthesis  Test your synthesis  Integrating and writing-up...