1. 1 Developing country capacity for impact evaluation PREM Week 2007 Arianna Legovini, Africa Impact Evaluation Initiative (AFTRL) Africa Impact Evaluation Initiative

2. 2 Country impact evaluation capacity: why?  Country:  Develop culture of evidence-based policy making  Most rigorous tool for decision making  Bank:  Paradigm shift:  From-- tell government what works  To-- empower government find what works for them  Institutional requirements (new ICR guidelines, 2006):   “Evaluation of the borrower’s own performance”   “Evaluation of the performance of the Bank”  “D  “Development of the borrower’s independent evaluation capacity, including capacity for impact evaluation”

3. 3 Evidence-based policy making  How to change culture?  How to get higher ups to listen?  What are the incentives for implementing agencies to do it?  How can it be done right?

4. 4 PM/Presidency: Communicate to constituencies Treasury: Allocate budget Line ministries: Deliver programs and negotiate budget Relative cost- effectiveness of different programs Effect of government program Cost-effectiveness of alternatives and effect of sector programs Motivation and incentives

5. 5 How: No unique answer  Start with who would listen  Tackle cultural change from different sides  Awareness and knowledge to build critical mass  Demonstrations  Institutional change  Take advantage of opportunities –IDA pressure, EU requirements, ICR guidelines

6. 6 Institutional issues Central institute of evaluation with government mandate Pluralistic academic centers + competitive bidding External evaluation, non-experimental Internal evaluation, experimental Low development of government capacity High development of government capacity

7. 7 Other issues to think about  Exploit existing collaborations between academia and government  Support implementing agencies  Quality  Demonstration  Ensure quality  Discourage contracting early on  Separate international from local  Train implementing agencies  Get comparable results  Harmonization of survey instruments and indicators

8. 8 Support to implementing agencies and quality assurance  Training for agency and local academics  Policy discussion  Question policies, developmental hypotheses and causal linkages  Investigate alternatives  Highly qualified IE TA during early stages and throughout implementation

9. 9 Harmonization of data instruments  Harmonized basic questionnaires across evaluations  Evaluation-specific additional modules  Set of common outcome indicators to compare results across different evaluations

10. 10 As project TTL… IE takes time and resources  Get help from regional and network IE contacts  Separate product (IE-code), get supervision budget  Bank researcher TTL for IE?  Look for in country and sector IE initiatives to participate in (e.g. www.worldbank.org/afr/impact ) (e.g. www.worldbank.org/afr/impact )www.worldbank.org/afr/impact