Using evidence to inform decision making on health systems in LMICs: what evidence do policymakers need? Harriet Nabudere, MD, MPH Knowledge Translation for Health Policy and Systems College of Health Sciences, Makerere University Kampala, Uganda

Supporting Use of Research Evidence (SURE) for Policy in African Health Systems SURE is a collaborative project that builds upon two existing initiatives – The Regional East African Community Health (REACH) Policy Initiative and the Evidence-to-Policy Network (EVIPNet) Africa. Both networks aim to promote the use of evidence in health policy decisions. REACH comprises the countries; Uganda, Tanzania, Kenya, Rwanda and Burundi. EVIPNet Africa comprises the countries of Burkina Faso, Cameroon, Centrafrique, Ethiopia, Mozambique, and Zambia.

Overall Strategy: SURE comprises 8 work packages. There are 6 work packages with scientific/technological objectives:  WP1: Production of research syntheses (evidence briefs for policy) to address priority policy questions.  WP2: Development and evaluation of strategies for improving access to research evidence to inform policy decisions, i.e.; user-friendly formats for evidence briefs and clearing house.  WP3: Developing and evaluating mechanisms for a rapid response service to meet policymaker needs for research evidence.

Overall Strategy (contd)  WP4: Developing and evaluating methods for conducting deliberative/policy dialogues that are informed by research syntheses/evidence briefs. Methods for involving civil society, the general public, the media in policy development.  WP5: Capacity-building for researchers, policymakers and civil society in developing and implementing evidence-informed health policies.  WP6: Comprehensive evaluation of the African partners initiatives to improve the use of research to inform health policy decisions.

Setting Priorities for Guidance: Consultations: Criteria: Important problem, viable options, opportunity for change, available evidence, etc Stakeholder Survey: Parliamentarians, policymakers, health managers, researchers, civil society Advisory Group: 3 policymakers, 1 researcher, 1 CS practitioner Framed and ranked issues that were considered high priority for the Ugandan health system.

SURE Year 2: Evidence Brief for Policy Title Authorship Problem Description Policy Options Implementation Considerations Completion of the evidence brief Dialogues and Evaluation

Evidence Brief for Policy: Title: ‘Improving Access to Skilled Attendance at Delivery’

Evidence Brief for Policy: Authorship: Contributing Authors: 2 members, REACH Uganda team 1 Senior Policymaker, MOH Skilled Birth Attendance Working Group: 2 Senior Policymakers (MOH), 1 Obstetrician Consultant/Researcher/ Academician

Evidence Brief for Policy: The Problem (1): Identification of evidence i.e national statistics & local data from MoH, UBOS, WHO, UN data etc. Advanced draft describing the problem Internal reviews by the working group & external reviews with revision of the problem section.

Evidence Brief for Policy: The Problem (2): The burden: high maternal mortality ratio 435 per 100,000 live births (DHS,2006) from MMR 670 (UN,1990) Coverage of Skilled attendance stood at 42% (DHS, 2006) and under 50% (AHSPR, 2009/2010) Maternal and perinatal causes constitute 13.2% of the total disease burden (WHO,2004)

Evidence Brief for Policy: Policy Options: Identification and appraisal of evidence (preferably systematic reviews of effects) for impacts (benefits and harms) of alternative organisational arrangements to address the problem Identification and appraisal of other evidence: local single studies.

Appraisal of the evidence: SURE checklist for systematic review methods 1. Check on methods used to search, select and appraise studies in systematic reviews PICOS criteria, data sources, timeframe, explicit selection criteria,.. 2. Check on methods used to analyse findings in systematic reviews Reliable reporting for study characteristics & results, analysis methods, appropriate combination of findings..

Appraisal of the evidence: GRADE framework for rating quality of evidence 1. Identification of important outcomes from a systematic review 2. Assessment of quality of evidence per outcome using GRADE framework 3. Tabulated summary of findings per outcome 4. Key messages in plain language

Evidence Brief for Policy: Policy Options (contd): Agreement and development of draft on 3 policy options (organisational arrangements):  Providing Intrapartum Care at first level Health Centre  Working with the Private-for-Profit sector  Maternity Waiting Homes

Evidence Brief for Policy: Implementation Strategies (1): Identification and appraisal of evidence (preferably SRs of effects) on barriers to implement the policy options & strategies to address these. Identification and appraisal of other evidence and local information e.g local single studies, national reports,…

Evidence Brief for Policy: Implementation Strategies (2): Agreement on the implementation barriers & strategies:  Implementation Barriers: poor care seeking behavior from mothers, social & economic constraints for mothers, inadequate HRH, inadequate health facilities and financing,  Strategies: VHTs, community mobilisation, community referral and transport schemes, strengthening health infrastructure and public-private collaboration

Evidence Brief for Policy: Implementation Strategies (3): Advanced draft describing the policy options, implementation barriers & strategies Internal review and revision of the policy options, implementation barriers & strategies

Evidence Brief for Policy: other considerations - Systematic reviews and single studies used were assessed not only for methodological quality but also, where possible, for: ◦ applicability to the local context ◦ equity across socio-economic groups ◦ scaling up considerations (costs and cost- effectiveness) ◦ gaps in the research evidence hence need for monitoring and evaluation

Full Evidence Brief for Policy Draft title and cover page Key messages (1 page) Executive summary (8 pages) Full Report (45 pages) Bibliography Description of methods Acknowledgements Conflicts of interest External review of the draft policy brief Revision of the full policy brief

Evidence Brief for Policy: Evaluation The evidence brief was the main input for discussion at two national policy dialogues for stakeholders including legislators, policymakers, researchers, civil society and the media A survey was conducted with the stakeholders about the usefulness of the design format of the brief

Overall assessment of the evidence brief for policy ‘The purpose of the policy brief was to present the available research evidence on a high-priority issue in order to inform a policy dialogue where research evidence would be just one input to the discussion. How well did the policy brief achieve its purpose?’ (N=22) Rating on a scale of 1 (failed) to 7 (achieved) Mean: 6.6Median: 7Range: 2

The need for guidance among decision- makers ‘ I want to use research evidence of the type that was discussed at the policy dialogue to help work through what I will say in a briefing, advocate for, or decide. (N=19)’ Scale of 1 (strongly disagree) to 7 (strongly agree) Mean: 6.3Median: 6Range: 2

The demand for health systems guidance Forwarded Message From: xxxxxxxxx To: Sent: Sunday, October 9, :34 AM Subject: POLICY ISSUES Greetings my sister. Wish you happy independence. There many policies we have to work on this year includiing legislative matters. You may have to give us more time. The following are pending - Palliative Care Policy - Interns Policy - Human Tissue Bill - Policy on Emergency Services. - The Tertiary Health Institutions Bill. We have to discuss one of these days.

Dr Harriet Nabudere SURE Project Coordinator College of Health Sciences, Makerere University Kampala, Uganda