Coordinator, Ghana Monitoring and Evaluation Forum (GMEF) “Evidence-based Policy-making and Implementation” (EBPM&I Course) University Of Cape Town, Graduate School Of Development Policy And Practice Cape Town, South Africa, October 24 to 26, 2016 Presented by, Dede Bedu-Addo  Coordinator, Ghana Monitoring and Evaluation Forum (GMEF)

Background Executive course for strategic leaders and top managers in the public service. Designed to assist participants to use evidence to make well informed decisions about policies, programmes, projects and services and to improve government’s impact on society. GMEF and NDPC were sponsored by The Centre for Learning on Evaluation and Results (CLEAR) Anglophone Africa As part of the Twende Mbele project, a collaborative effort between the governments of Uganda, Benin, and South Africa to strengthen national evaluation systems through peer learning and collaboration.

Rationale Increasing pressure on policy makers to: develop more effective policies and direct and manage resources in more focused and efficient ways resulting in improved implementation and outcomes Recently, evidence-based policy-making is an increasingly prevalent approach. Better policies and better decision-making results are based on sound empirical evidence and solid rational analysis. Also critical to use evidence to improve implementation. Evidence-Based Policy-Making and Implementation (EBPM&I) focuses on establishing rigorously, objective evidence as a key informant of policy, but also for improving implementation of public services.

Course Objectives Participants gain: An understanding of the factors that influence policy and implementation processes and the difference between evidence and opinion An understanding of the reason for, and the method of constructing, a theory of change An understanding of how the use of evidence can improve the performance of leaders and departments as well as the implementation and impact of policies An opportunity to reflect on existing policy-making and management processes, their strengths and limitations An understanding of tools for evidence-based policy-making and implementation An understanding of the role of evidence in managing political and other influences on the policy and implementation process.

Definition of Evidence-based policy-making Evidence-based policy-making (EBPM) helps policy makers and providers of services make better decisions, and achieve better outcomes, by drawing upon the best available evidence from research, evaluations and other sources. “This approach stands in contrast to opinion-based policy, which relies heavily on either the selective use of evidence (e.g. on single studies irrespective of quality) or on the untested views of individuals or groups, often inspired by ideological standpoints, prejudices, or speculative conjecture.” (Segone 2004:27)

Evidence is Used to Understand A Problem (Its nature, size, dynamics) and to determine what works to address this - what interventions are effective? - what interventions work best to achieve specific outcomes? (and what interventions do not work?) - for which groups of people? - under what conditions? - with which modes of delivery/implementation? - over what time frame? - with what costs?

Opinion and evidence in policymaking and implementation

Evidence vrs Opinion Evidence: •Data that meet the standards of internal validity and external validity •Key qualities: independence, objective, verifiable Opinion: •Statements and claims that do not meet the standards of evidence •Key qualities: subjective, partial (selective), hard to verify

Policy is derived from …

How Do We Guard Against Inadequate Evidence? Evidence may not be complete or perfect—need a broad range of good quality evidence Evidence is often contested, so its important to understand the evidence and what its significance is Evidence may be limited e.g. it is context specific Its important to be able to distinguish between good and poor quality and to be able to weigh the balance of evidence

Different Types Of Evidence Expert knowledge Beneficiary consultations Published research Costings of policy options Other research Performance expenditure reviews Stakeholder consultations Economic and statistical modelling Policy and programme evaluations Research synthesis including systematic reviews Administrative data analysis

Quality of Evidence Internal Validity: Has the evidence scientific credibility? i.e. the extent to which the design and conduct of the study/report eliminates the possibility of bias Is the sample size adequate to detect the ‘true’ situation? If comparisons are being made, are the two groups truly comparable? Is there evidence of bias, e.g.: Selection bias? Attrition bias? Performance bias? Reporting bias?

Quality of Evidence External Validity: Does the study have relevance to the ‘real world in which you are working? Is the sample in the study/report similar to the population you are dealing with? Are the outcomes in the study/report the same outcomes that are of interest to your work? Are there any contextual factors mentioned in the study/report that would limit its relevance to your work?

The value of good evidence There is more and more good data and evidence production locally and internationally There is less and less justification for policy/decision making by intuition or whim Good evidence can depoliticise important and difficult decisions

CONTINUUM OF TYPES & SOURCES OF EVIDENCE CONTINUUM OF TYPES & SOURCES OF EVIDENCE USED None Opinion Informal Substantive Formal Research Scientific Research Review & assessment of available evidence No explicit basis for policy deci-sions Of the policy makers; or broadly accepted public opinion; lobby groups; others Anecdotal, stories, uncritical use of information to hand, partial & limited Careful truth seeking, developmental evaluation, data mining of survey & admin. data Empirical, appropriate design & methodology Statistical, comparative, causal Testing, analysing & synthesising available evidence. Rapid reviews; systematic enquiry; meta-analysis

Theory of Change seeks to Answer the Following Questions: 1) How is a policy/programme supposed to work? 2) What activities, mechanisms, and people have to be in place? 3) And in what sequence – what is the causal chain? 4) What resources are required – and are available? 5) What data is required – and is available? 6) Is the policy/programme achievable?

key highlights & Lessons It was very interesting to see how similar some of our M&E issues are across Africa, ie. availability and quality/reliability of data, low status of M&E, low M&E capacity, etc. The M&E system in South Africa is very comprehensive, well coordinated and impressive. A shift in focus from increased compliance to more cooperation among stakeholders in the M&E system is more effective. Most effective M&E systems prioritise and promote the active involvement of CSOs and grassroots organisations.

Nye tsu mi. - Krobo Akpe na mi. -Ewe Mi da mo ase. - Twi Nangode Nye tsu mi! - Krobo Akpe na mi! -Ewe Mi da mo ase! - Twi Nangode! - Hausa