Improving Learning Outcomes: Textbooks and Expert Teachers Impact Evaluation Team Albert (Chuku) Duprigny Reginald King John Swaray Sahr Gbamanja Eunice Dapaah David Evans Keiko Inoue AFRICA IMPACT EVALUATION INITIATIVE, AFTRL Africa Program for Education Impact Evaluation

Questions Does the provision of textbooks and textbook capacitation improve learning outcomes at the primary level? Does the provision of VCDs of expert teachers improve learning outcomes and teacher quality? What is the combined effect?

Indicators OUTPUTS Improved student-textbook ratio Access to textbook capacitation program Access to the supplementary teaching materials (VCD+TV+solar power) OUTCOMES 1.Improved teacher test scores 2.Improved student achievement test scores Simple literacy and numeracy tests NPSE (national test in class six) 3.Improved student attendance

Evaluation Design Random experimental design Randomly select 5 out of 19 councils, within which 30 schools are randomly selected. TEXTBOOKNO TEXTBOOK EXPERT TEACHER 30 schools NO EXPERT TEACHER 30 schools

Sample Minimum Effect size: 0.20 (assumed) Power: 80%; Level of significance: 0.05 Students per school (grades 4-6): 150 No of schools: 30 each treatment, 30 control

Data Existing Data EMIS (gender, enrollment, textbook ratio, etc.) NPSE (class six exit exam) Attendance registers Newly Collected Data Validated attendance rates (twice per term) Student numeracy and literacy test results Teacher competency on NPSE Equipment delivery and retention monitoring

Staffing Plan GOVERNMENT Teacher training, monitoring, planning, textbook procurement and distribution, statistics, decentralization, local councils UNIVERSITIES Education researchers and graduate students from Univ of SL and Njala Univ DEVELOPMENT PARTNERS Plan SL, Concern, DfID, ADB, UNICEF, UIS, World Bank

Timeline: 3-year program Year 1 ( school year) August: Local council consultation and randomization September: Teacher textbook capacitation intervention to grades 4, 5, 6 September: Textbook intervention to grades 4, 5, 6 April: Expert teacher VCD intervention to grades 4, 5, 6 May: NPSE June: Student literacy & numeracy test; teacher competency test Twice per term: drop-in attendance checks

Timeline: 3-year program Year 2 ( school year) September: Check input retention September: Textbook distribution to grades 1, 2, 3 (control group) – pending budget availability May: NPSE (5 th graders from year 1) June: Student literacy & numeracy test; teacher competency test (grades 4, 5, 6) Twice per term: drop-in attendance checks

Timeline: 3-year program Year 3 ( school year) September: Check input retention May: NPSE (4 th graders from year 1) June: Student literacy & numeracy test; teacher competency test (grades 4, 5, 6) Twice per term: drop-in attendance checks

Budget Items Program design Textbook IE Teacher textbook capacitation IE Literacy and numeracy test design Teacher-version of NPSE Questionnaire design Data collection/entry Attendance checks Travel (vehicles) Test & questionnaire administration Data input Data analysis Consultants/students