1. Operational Aspects of Impact Evaluation
Arianna Legovini
Impact Evaluation Cluster, AFTRL

2. Operational guidance: Key steps
1. Identify policy questions
2. Design evaluation
3. Prepare data collection instruments
4. Collect baseline
5. Implement program in treatment areas
6. Collect follow up
8. Improve program and implement new
7. Analyze and feedback

3. Operational guidance: Keep it useful!
“Interesting” is what the government wants
Questioning program helps strengthen design
Provide answers to questions early
Change program as a result
Get better results for client and Bank!

4. Keep it relevant! 1. Right questions 2. Right team
3. Develop client capacity

5. Manage trade-offs Method &Data
Experimental
Least costly
Best estimates
Not always politically feasible
Non-experimental
More expensive
Good second best estimates
Always feasible
Administrative data (expanded to control)
Most cost effective
Piggy back on ongoing data collection
Cost effective
Coordination intensive
Collect special surveys
Tailored and timely
Expensive

6. Budget Financing Data collection Consultants Staff time
Include baseline costs in PPF
Include follow up costs in project costs
Use Trust Funds to finance consultants
Initiate product IE-code and obtain BB supervision funds
Financing

7. M&E Operational messages
Plan M&E while planning project
Build an explicit team
Develop budget and financing plan
Influence roll-out to obtain control groups
Use quantitative & public allocation criteria
Randomization is ethical
Strengthen monitoring systems