2. Question-led mixed methods research synthesis Centre launch 21 June 2005 David Gough and Sandy Oliver Institute of Education, University of London

3. Methods for Research Synthesis Programme David Gough Sandy Oliver Angela Harden James Thomas Rebecca Rees Adam Fletcher Lisa Underwood Ann Oakley Building on more than ten years’ experience of synthesising policy relevant research in health, education and social welfare

4. Professional Experience & Expertise Political Judgement Resources Values Habits & Tradition Lobbyists & Pressure Groups Pragmatics & Contingencies Research Evidence Adapted from Philip Davies, 2005 DIFFERENT TYPES OF EVIDENCE

5. Research production and evidence Evidence influenced by the context of its production All the activities of a research commissioning or granting agency (including training) All the activities of a research production facility (e.g. institute, department, university) The activities of a potential research user organization and its staff The entire ‘research regime’ in a country Adapted from Jonathan Lomas, 2005

6. Research methodology In question-led synthesis, methodology is A system of principles, practices, and procedures applied to a specific branch of knowledge. A set of procedures or methods used to conduct research. For asking the questions as well as findings the answers

7. Questions and methods Choosing questions Gathering questions –Policy makers –Practitioners –Service users –Members of society –Academics Setting priorities –Consensus development methods Framing questions –(un)answerable research questions Finding answers Choosing appropriate research designs Choosing methods for finding, appraising and synthesising the findings of research reports

8. How questions vary Policy makers Practitioners Service users Members of society Academics Collective questions Population wide? Costs? Practical, delivery issues? Personally relevant? Ethical? ConceptualPhilosophical? Answerable? ???

9. My question is, are we having an impact?

10. User question led synthesis What do we want to know (and who wants to know and why)? What do we know already (and how do we know it)? What more do we need to know (and how can we know it)? Can not be value free Involves intellectual work

11. Older people, accidents and injuries Population People over 55 Older patients Older people at less risk Interventions to prevent injury to reduce falls & frailty to prevent falls & injury to prevent accidents Exercise/ balance training Medical interventions Outcomes Knowledge Falls or fall related injury Strength, balance, gait, sway Design Controlled trials RCTs Quality assessed RCTs

12. Questions do make a difference 6 reviews of older people and accident prevention Total studies included 137 Common to at least two reviews 33 Common to all six reviews 2 Treated consistently in all reviews 1 Oliver et al. 1999

13. INTERPRETATION AND APPLICATION WHAT DO YOU WANT TO KNOW? WHAT HAS BEEN DONE? COMMUNICATION OF KNOWLEDGE PERSPECTIVES AND PARTICIPATION RESEARCH STUDIES AND METHODS WHAT IS KNOWN? (WITHIN THOSE PERSPECTIVES AND THAT FORM OF KNOWLEDGE) TYPES OF RESEARCH EVIDENCE What more needs to be known? RESEARCH QUESTIONS TYPES OF KNOWLEDGE

14. Interpretation and application What do you want to know? What has been done? Communication Perspectives and participation How/processes? Nature,/extent/frequency? RESEARCH QUESTIONS Perspectives/insights/concepts? Effects ? Multi method TYPES OF KNOWLEDG E Action research RESEARCH STUDIES AND METHODS Case study Survey Experimental (iv) Policy community (i) Research (iii) Practitioner (v) Organisational (ii) User of service/ public What do we know? How do we know it? RESEARCH EVIDENCE Nature,/extent/frequency Perspectives/insights/concepts? How/processes? Effects PERSPECTIVES AND PARTICIPATION Interpretation and application Communication What do we know? How do we know it? What don’t we know? How could we know it? What has been done? What do we want to know? TYPES OF KNOWLEDGE RESEARCH QUESTIONS RESEARCH STUDIES AND METHODS RESEARCH EVIDENCE (iv) Policy community (i) Research (iii) Practitioner (v) Organisational (ii) User of service/ public How/processes? Nature/extent/frequency? Effects? Perspectives/ concepts? Multi method Action research Case study Survey Experimental How/processes? Nature/extent/frequency? Effects? Perspectives/ concepts?

15. Need for synthesis Brings together what we know – whatever we are studying Contextualizes information from new studies Involves explicit systematic methods and thus transparency These may be lacking in non systematic reviews and expert opinion however excellent

16. Systematic review: key features Synthesises the results of primary research Uses explicit and transparent method A piece of research, following standard set of stages Accountable, replicable, updateable Need for user involvement

17. Question led synthesis Questions looking for answers Make implicit assumptions explicit All types of question so all types of research design Statistical, narrative empirical and conceptual synthesis Mixed methods synthesis

18. Dimensions of difference in synthesis models Review/research questions (impact, process, need, explanatory concepts) Research designs considered relevant Types of data - numerical or textual Quality assessment of different designs Breadth of question and research design Quantitative/narrative empirical/conceptual synthesis Variation in contribution of each study to the systematic synthesis

19. Review question e.g. What is known about the barriers to, and facilitators of, fruit and vegetable intake amongst children aged 4 to 10 years? Trials 1. Application of inclusion criteria 2. Quality assessment 3. Data extraction 4. Statistical meta-analysis ‘Qualitative’ studies 1. Application of inclusion criteria 2. Quality assessment 3. Data extraction 4. Qualitative synthesis Trials and ‘views’ Mixed methods synthesis

20. Findings for synthesis 1: ‘Quantitative’ (Trials)

21. Findings of synthesis 2: Descriptive themes Influences on foods eaten Chosen foods Provided foods Foods in the home Foods in the school Parental influence and food rules Breaking rules Limited choice Social occasion Contradictions Food preferences Perceptions of health benefits Knowledge-behaviour gap Roles and responsibilities Non-influencing factors Concepts of healthy eating Healthy eating concepts Good and bad foods Health consequences

22. Developing a framework Typology of research questions in the social science literature Methods of synthesis for answering such questions Matrix of synthesis methods and methodological development and challenges Creating solutions with others An emerging framework

23. Contacts and information NRCM MRS web http://www.ncrm.ac.uk/nodes/mrs/people.php SSRU web: http://www.ioe.ac.uk/ssru/http://www.ioe.ac.uk/ssru/ SSRU's EPPI web: http://eppi.ioe.ac.ukhttp://eppi.ioe.ac.uk Email: d.gough@ioe.ac.ukd.gough@ioe.ac.uk s.oliver@ioe.ac.uk