1. STANDARDS OF EVIDENCE FOR INFORMING DECISIONS ON CHOOSING AMONG ALTERNATIVE APPROACHES TO PROVIDING RH/FP SERVICES Ian Askew, Population Council July 30, 2015

2. Research designs and types of evidence Summarizing bodies of evidence from multiple research studies Using bodies of evidence for decisions Three Issues 1 2 3

3.  Theoretical control for internal validity; rigorous design; quantifiable evidence Why are RCTs Considered the ‘Gold Standard’ for Generating Evidence of Impact?  Understandability by decision-makers with medical training  Publishability of findings for academic researchers  Usability as evidence base for systematic reviews

4. Evidence from Non-Randomized “Quasi- Experimental” Designs is Frequently Used to Answer Questions of Impact and Alternatives CONTROLLED BEFORE- AFTER  Controlled through matching rather than randomization INTERRUPTED TIME SERIES  An extended Before-After, with or without a control PROSPECTIVE COHORT, WITH OR WITHOUT CONTROL  Equivalent rigor for comparisons - Matching - Multi-stage sampling - Quantifiable evidence  Allows program managers some influence  Learn about implementation in “real-life” setting

5. A global health evidence framework [is] one which uses multiple domains to arrive at a summary judgment of the evidence for community or population health interventions or programs. Source: Luoto et al, 2013

6. Making Evidence-Based Decisions: Summarizing a Body of Evidence through Rigorous Reviews Using an Evidence Framework  Guided by theory of change  Methodical and transparent procedures for inclusion / exclusion of evidence  Rate evidence across multiple domains - Design; quantity; relevance; consistency; context….  Summary judgment of evidence informs recommendations

7. “Systematic Reviews” have traditionally focused on evidence of effectiveness, rather than implementation

8. EFFICACY of a service intervention in meeting health needs of the individual/couple  e.g. service delivery guidelines Which Review Process and Evidence Framework is Appropriate for which Type of Recommendation? EFFECTIVENESS of delivering interventions through programmatic approaches  e.g. sustainability programming guidance SUSTAINABILITY at system/national level  e.g. systems strengthening and scale- up/mainstreaming interventions INTERNAL VALIDITY CONTEXT, COST FEASIBILITY, IMPLEMENTATION

9. Fuller discussion of issues available in meeting report www.popcouncil.org/uploads/pdfs/2014STEPUP_ConsultationStandards.pdf

10. Three Suggestions for Improving Use of Research-Based Evidence for Decision-Making

11.  RFPs require and fund matched designs, multivariate analyses, rigorous documentation of implementation processes, clearer descriptions of intervention and study design (PICOT, TREND) Accept and support generation of ‘plausible’ evidence from non- randomized yet rigorous research 1

12.  Agreed evidence framework and rating system  Agreed process for determining strength of recommendation  Standards appropriate for type of decision Decide on a uniform standard of evidence 2

13.  Evidence-informed choice from options  Describe standard of evidence used Communicate findings from evidence reviews effectively and clearly 3

14. THANK YOU! Ian Askew Population Council iaskew@popcouncil.org