1. Bringing the evidence revolution to Ghana
Evidence to Action:
Bringing the evidence revolution to Ghana
Howard White
CEO, Campbell Collaboration
Evidence to Action 2019
ISSER, University of Ghana
11th July 2019
@campbellreviews @HowardNWhite

2. The rise and rise of evidence

3. The four waves of the evidence revolution
Rise of RCTs
Results agenda
(Outcome monitoring)
Rise of systematic reviews
But data are not evidence
Knowledge brokering

4. Knowledge brokering platform pyramid
This approach supports the institutionalization of the use of evidence
Checklists
Guidance & guidelines
Evidence portals
Evidence maps
Databases
Systematic reviews
Studies
Data
Knowledge brokering platform pyramid
More heavily brokered knowledge

5. Impact evaluations in 3ie database by year of publication
200 studies funded by 3ie
Source: Shayda Mae Sabet & Annette N. Brown (2018) Is impact evaluation still on the rise?
The new trends in 2010–2015, Journal of Development Effectiveness, 10:3, 

6. Increasing secondary education
RCT of removing secondary fees: 2000 students admitted SHS but not enrolled; 680 given four year scholarship.
Removing fees for secondary education:
Large effect on enrolment
Significant but small effect on test scores females only
Reduced sexual activity and more likely to practice safe sex
Fewer unwanted pregnancies
Source: Estimating the impact and cost-effectiveness of expanding access to secondary education in Ghana

7. Policy lessons from the study
Scholarship raised enrolment from 50% (in control) to 75%
25% still did not attend so fees not the only constraint
50% went without scholarship (but many delayed a year), so means testing might be considered
Social benefit of reduced fertility justifies subsidy
But impact on learning outcomes low – quality issues to be addressed

8. Impact evaluations in Ghana: no. of studies in selected countries
Source: 3ie database

9. No. of studies normalized by population
1 study per 2 million people
1 study per 80 million people

10. What should we do with these impact evaluations?
Impact evaluations are just one part of the evidence puzzle

11. The evidence-driven project cycle
Proposed programme
Prevalence data identify priorities & target group
Formative research
Consult evidence base to inform design
Formative evaluation in local context
Pilot programme: Efficacy studies
Go to scale with promising components: effectiveness studies
Keep testing as roll out to new populations / contexts / design features
Synthesize evidence across all studies
The evidence-driven project cycle

12. The Evidence Peter Principle
Monitoring
Factual data of what happened. Especially useful at lower reaches of causal chain. Outcome data identify problems. But used to (1) measure impact, and (2) identify programmes
Process evaluation
Factual analysis of programme management / implementation. Useful for improving programme performance. But use to measure outcomes.
Impact evaluation e.g. RCTs
Counterfactual analysis of effectiveness. Useful for determining which programmes work to inform finding and design decisions for the programme being evaluated. But used to make global recommendations.
Systematic reviews
Summarizes all available high-quality evidence on a particular issue. Useful deciding on for new programmes or programmes redesign to try and test.

15. Politicians’ syllogism
Something must be done
This is something
Therefore we must do this
That’s the fallacy of the undistributed middle

16. The evidence-driven project cycle
Proposed programme
Prevalence data identify priorities & target group
Formative research
Consult evidence base to inform design
Formative evaluation in local context
Pilot programme: Efficacy studies
Go to scale with promising components: effectiveness studies
Keep testing as roll out to new populations / contexts / design features
Synthesize evidence across all studies
The evidence-driven project cycle

17. The evidence-driven project cycle
Proposed programme
Prevalence data identify priorities & target group
Formative research
The evidence-driven project cycle

18. The evidence-driven project cycle
Proposed programme
Prevalence data identify priorities & target group
Formative research
Consult evidence base to inform design
Formative evaluation in local context
Pilot programme: Efficacy studies
Go to scale with promising components: effectiveness studies
Keep testing as roll out to new populations / contexts / design features
Synthesize evidence across all studies
The evidence-driven project cycle

19. Unwanted and teenage pregnancy

20. An example of various data sources: teenage pregnancy in Ghana
Is it an issue?
Over 30% teenagers pregnant or have a child (Preference data from DHS)
Teenage pregnancy linked to dropping out of school, early first birth (bad for baby and mother), and higher fertility (Correlational and prevalence data from DHS)
Identifying target groups:
Highest girls primary or less, Savannah belt, second quintile (lowest in highest) (prevalence data from DHS)
Programme design:
Systematic review evidence suggests few programmes effective, notably abstinence for teenagers. Multi component programmes most promising. (Systematic review evidence). Though see also
See also Stepup (Research resource)
Implication: we need to do something (probably multicomponent). But we have to test whatever we do.

21. Rise of systematic reviews: international development (no
Rise of systematic reviews: international development (no. of reviews published per year)
Source: 3ie database

22. Knowledge brokering platform pyramid
This approach supports the institutionalization of the use of evidence
Checklists
Guidance & guidelines
Evidence portals
Evidence maps
Databases
Systematic reviews
Studies
Data
Knowledge brokering platform pyramid
More heavily brokered knowledge

23. Evidence-based medicine is based on systematic reviews
The World Health Organization (WHO) follows a guideline development process, described in detail in the WHO Handbook for Guideline Development (2nd edition), overseen by the Guidelines Review Committee (GRC) established by the Director-General in The WHO Guidelines Review Committee ensures that WHO guidelines are of a high methodological quality, developed using a transparent and explicit process, and are informed on high quality systematic reviews of the evidence using state-of–the art systematic search strategies, synthesis, quality assessments and methods.

24. But FAILURE of global community to invest in evidence architecture (apart from health)
Impact evaluations
Review of education RCTs found 1,017
3ie database has 4,735 impact evaluations with 1,980 RCTs
Cochrane trials registry has 1.5 million
Systematic reviews
ERIC has around 1,000 education systematic reviews
3ie database has 692 systematic reviews
Epistemonikos has over 35,000 health reviews

25. In absence of that Ghanaian makers can use existing evidence resources
Databases (e.g. 3ie database, Campbell Library)
Evidence and Gap Maps
Local evaluations

26. The evidence-driven project cycle
Proposed programme
Prevalence data identify priorities & target group
Formative research
Consult evidence base to inform design
Formative evaluation in local context
Pilot programme: Efficacy studies
Go to scale with promising components: effectiveness studies
Keep testing as roll out to new populations / contexts / design features
Synthesize evidence across all studies
The evidence-driven project cycle

27. Findings from education review of over 200 studies

28. The Uganda country evidence and map
269 process evaluations, 207 impact evaluations and 7 formative evaluations

29. Uganda country evidence and gap

30. The Ghana Evaluation Map
EVIDENCE
The Ghana Evaluation Map

31. Join the evidence revolution

32. Knowledge brokering platform pyramid
Building the evidence architecture
Demand generation
Checklists
Guidance & guidelines
Evidence portals
Evidence maps
Databases
Systematic reviews
Studies
Data
Where do you fit in?
Use of evidence workshops
Evidence needs assessment
Evidence-based budgeting
Awards
Knowledge brokering platform pyramid
Supply of evidence and evidence products

33. Become an evidence revolutionary today
Thank you
Become an evidence revolutionary today
Visit