1. Impact Evaluation and Administrative Data Systems
N. Krishnan, C. Rockmore
Africa Impact Evaluation Initiative
Dakar, 18 December 2008

2. Opportunities and challenges
Most impact evaluations use survey data
Unique opportunity in education to use administrative and test data
Need to improve the quality and usability of administrative data
Integrate with test score data
Case studies

3. Data sources Administrative Test data School surveys Beneficiary only
Output and process data
Facility level of disaggregation
Systems in place, low additional cost
Test data
Individual level
Main outcome
School surveys
Independent verification
Complementary information
Output and outcome data
Costly

4. Administrative data quality
Relevance: feedback loop from IE and others
Accuracy: identify systematic misreporting
Timeliness: September vs. January
Accessibility: can it be integrated and used?
Interpretability: methods and documentation
Coherence: does it match other sources?
Single ID,

5. Build within existing frameworks and plans
Integrated and modular data systems
Use common identifiers for the same concepts (schools, children, and teachers)
Identifiers (codes) should have meaning
Harmonize data collection efforts
Look at sector plans and work within them

6. From evaluation to data systems
Eritrea: supporting data for malaria
2004 evaluation entered district level data in a database
Ministry of Health malaria program continued maintaining the database with facility data on a monthly basis
intervention output data (nets distributed, medicines, insecticide by facility and date of delivery) and
in and out-patient outcome data on fever cases.

7. Use of administrative data for assigning grants
Niger: mitigating misreporting
School grants on the basis of classroom use to avoid misreporting on number of pupils
Uganda
District level statistics to allocate rewards

8. Score cards (Tableaux de bord)
Link input and output data to results
Compare across administrative levels
Feedback loop possible for lower levels
Can identify the low-input, high-result and high-input, low-result cases

9. Use of data in impact evaluations

10. South Africa
IE question: Did the Dinaledi schools program improve student enrolment and performance in Mathematics and Physical Science relative to similar schools?
Outcome of interest: Number of students entering, writing and passing Math and Physical Science in the senior certificate/school-leaving exam
Data used:
School characteristics from school level EMIS data
Outcome data from exam centre level testing data
Issues:
School codes and exam centre codes are not the same, and often follow no relation to each other
A school could be associated with more than one exam centre and vice versa

11. Dinaledi results (Gauteng province)
Pass rate out of students who entered
Dinaledi Schools (Average)
Matched Schools (Average)
Difference: Treatment-Control
HG Math
0.72
0.62
0.10***
SG Math
0.64
0.60
0.04**
HG Physics
0.83
0.74
0.09***
SG Physics
0.68
0.08***
***significant at 1%, **significant at 5%