1. Information Campaigns to Strengthen Participation & Improve Public Schools: Experimental Evidence from Two Studies in Uttar Pradesh and Madhya Pradesh Stuti Khemani and Priyanka Pandey The World Bank

2. Motivation  Widening belief that encouraging “participation” by citizens, or clients of services, can improve the quality of publicly provided education services  Education policies across states in India have created specific institutions for community participation to improve public schools  Emerging survey evidence that citizens are not informed about or aware of these institutions  We test (using the ‘gold standard’ of randomization) what impact providing information has on participation in and performance of public schools

3. Study 1 (with Pratham and J-PAL):  Jaunpur district in Uttar Pradesh (UP)

4. Institutions of participation in UP  Village Education Committee (VEC)  5 members: elected head of village government (Gram Pradhan), senior head teacher from village public schools, 3 parents

5. Roles of the VEC  Monitoring: Visit schools and inspect records  Planning and Implementation -- Receive and decide how to spend government grants -- Participate in selection of Shiksha Mitras (contract teachers recruited from within the community) for government schools if there are not enough teachers. -- Build or maintain school rooms -- Manage mid-day meals, distribute scholarships -- Raise more money for the school -- Work with school teachers to improve education quality -- Encourage parents to improve child attendance

6. Evidence on Participation: Baseline Survey  Survey took place over March-June 2005  In each of 280 Villages (randomly selected), we collected data on: Learning Outcomes  30 households randomly selected  all children between the ages of 7 and 14 are tested on basic reading, writing, and math skills Community Participation and Local Governance  10 households (of the above 30) randomly selected and surveyed  All Government primary school head-teachers surveyed  All VEC members surveyed School Resources and School-Functioning  All government primary schools surveyed

7. Parents don’t know that a VEC exists

8. VEC members don’t know their roles Percent of VEC Members Who: Don’t know that they are members of the VEC Have not heard of SSA * Don’t know that funds are provided to VECs to improve schools * Head Teachers 4.2%0.5%4.2% Other VEC Members 22.7%67.6%73.6% * Of those who know they are members of the VEC

10. Parents of children at low levels of learning are particularly unlikely to know this…

12. Interventions to Strengthen Community Participation: (1) Information about VECs  Small, informal meetings in each hamlet during 2 days in a village  Village-wide meeting on 2 nd or 3 rd day, with participation of key VEC members—Gram Pradhan and School Teacher  Distribution of pamphlets to VEC members listing and explaining their roles

13. Interventions to Strengthen Community Participation: (2)  (1) + Testing Tool  In hamlet meetings, facilitators begin testing children; community invited to test children themselves and prepare hamlet-level “report cards”  In village-wide meeting, hamlet volunteers invited to present testing tools and “report cards”  Testing tool provides additional information (about learning), mobilizes community, and builds capacity in monitoring

14. Interventions to Strengthen Community Participation: (3)  (2) + Teaching Tool of “Read India”  In village-wide meeting, Pratham facilitators present “Read India” teaching tool  Offer to train anyone who would like to hold reading classes  Teaching tool provides additional information (about how to improve learning), mobilizes community, (“come forward to make your village a reading village”), and builds capacity in teaching

15. Experience with Interventions  65 villages each received Interventions 1, 2, and 3 between September and December 2005  Repeat visits in February to hand-out and explain pamphlets to VEC members  85 villages served as controls  215 village-wide meetings recorded in 195 “treatment” villages; on average, attendance of 108 villagers/meeting (Village size: average 360 hshlds, min 146, max 1289)  Village Pradhan, and School Head-teacher attended 68.2% and 71.7% of meetings respectively

16. Experience with Interventions  ”Read India” intervention received large response—local youth volunteered to hold regular classes for 2-3 mths No. of Class-Sequences in “Read India” Villages 11 1 2 4 6 3 10 7 3 44 2 33 2 0 2 4 6 8 12 01234567891011121316 No. of Class-Sequences No. of Villages

17. Endline Survey  Survey took place over March-June 2006  Impact evaluation using: -- matched panel of 2500+ households -- matched panel of 260+ govt. schools -- matched panel of 16,400+ children -- VECs: matched 240+ villages, with responses of all VEC members collapsed by village

18. Impact Evaluation  Difference-in-Difference estimates  With and without controls  Standard errors clustered by village

19. Interventions did happen—impact on knowledge and training of VECs (clustered std. errors in parentheses) T1T2T3Any Parents know about VEC 0.030 (0.12) 0.019 (0.014) 0.022 (0.012) 0.023 (0.009) VEC member knows s/he’s a member 0.106 (0.05) 0.097 (0.051) 0.068 (0.048) 0.091 (0.04) VEC member has heard of SSA 0.062 (0.041) 0.069 (0.04) 0.074 (0.042) 0.068 (0.03) VEC member reports being “trained” 0.161 (0.054) 0.133 (0.05) 0.219 (0.054) 0.172 (0.038)

20. No impact on self-reported VEC activity (clustered std. errors in parentheses) T1T2T3Any VEC members report complaining -0.012 (0.029) 0.048 (0.034) 0.007 (0.028) 0.014 (0.024) VEC members report raising money -0.005 (0.011) -0.01 (0.011) -0.007 (0.01) -0.007 (0.009) VEC members reported school inspections -0.564 (1.195) -0.890 (1.32) -0.433 (1.502) -0.624 (1.088) Average over the three -0.193 (0.395) -0.284 (0.434) -0.144 (0.496) -0.206 (0.36)

21. No impact on self-reported VEC activity  No impact on -- self-reported hiring of additional teachers, -- distribution of scholarships, -- implementation of mid-day meals, -- various others

22. No impact on parent-reported participation (clustered std. errors in parentheses) T1T2T3Any Visited school to monitor or complain -0.016 (0.025) -0.040 (0.025) -0.014 (0.025) -0.023 (0.020) Donated to school -0.001 (0.012) -0.001 (0.011) -0.006 (0.010) -0.002 (0.008) Volunteered at school -0.008 (0.010) -0.020 (0.009) -0.010 (0.010) -0.013 (0.008) Complained or talked about problems in school 0.028 (0.019) 0.015 (0.017) 0.019 (0.019) 0.021 (0.014) Average over family of outcomes 0.001 (0.009) -0.011 (0.009) -0.003 (0.010) -0.005 (0.007)

23. No impact on head-teacher reported parent participation (clustered std. errors in parentheses) T1T2T3Any Have parents visited the school -0.037 (0.070) -0.034 (0.067) -0.105 (0.073) -0.056 (0.053) Have you organized a parents meeting 0.029 (0.070) 0.118 (0.060) 0.097 (0.070) 0.082 (0.054) Did parents volunteer in the school 0.069 (0.060) 0.045 (0.060) 0.072 (0.065) 0.061 (0.047) Did the school get an allocation from the panchayat -0.010 (0.029) 0.009 (0.029) -0.006 (0.031) -0.002 (0.023) Did the school receive parents' donations -0.042 (0.041) -0.009 (0.047) -0.051 (0.038) -0.033 (0.036) Average over the family of outcomes 0.002 (0.029) 0.026 (0.025) 0.001 (0.027) 0.011 (0.020)

24. No impact on head-teacher reported school resources (clustered std. errors in parentheses) T1T2T3Any Textbooks 0.026 (0.028) -0.013 (0.034) -0.013 (0.036) 0.000 (0.026) Scholarships 0.049 (0.048) -0.031 (0.055) 0.054 (0.049) 0.022 (0.042) Seating Furniture -0.063 (0.052) -0.057 (0.053) -0.040 (0.044) -0.054 (0.038) Maps and Charts 0.005 (0.039) 0.002 (0.039) 0.029 (0.033) 0.011 (0.031) Electricity -0.031 (0.027) -0.022 (0.030) -0.014 (0.022) -0.023 (0.021) Water -0.054 (0.033) -0.071 (0.038) 0.003 (0.020) -0.043 (0.021) Toilets -0.016 (0.061) -0.052 (0.062) -0.107 (0.055) -0.056 (0.048) Does the school serve midday meal 0.001 (0.051) 0.059 (0.048) 0.054 (0.047) 0.037 (0.043) Average over the family of outcomes -0.005 (0.013) -0.028 (0.014) -0.017 (0.013) -0.017 (0.011)

25. No impact on number & presence of teachers in public schools (clustered std. errors in parentheses) T1T2T3Any Number of teachers (head-teacher report) 0.159 (0.169) 0.116 (0.152) 0.152 (0.160) 0.141 (0.127) Teacher absence (head-teacher report) -0.022 (0.037) -0.031 (0.031) -0.028 (0.033) -0.027 (0.026) Teacher absence (random check) -0.007 (0.028) -0.011 (0.025) -0.046 (0.025) -0.021 (0.021) Teachers teaching/Teachers present 0.010 (0.059) -0.042 (0.053) -0.088 (0.065) -0.040 (0.047) Average effect over the family of outcomes 0.005 (0.038) -0.008 (0.031) -0.041 (0.033) -0.014 (0.027)

26. No impact on public school enrollment and attendance (clustered std. errors in parentheses) T1T2T3Any Log(Boys Enrolled) (head-teacher report) 0.041 (0.048) 0.027 (0.050) -0.020 (0.069) 0.017 (0.045) Log(Girls Enrolled) (head-teacher report) 0.001 (0.077) 0.020 (0.074) 0.013 (0.075) 0.012 (0.071) Boys present (direct observation)/ Boys Enrolled 0.029 (0.041) -0.004 (0.042) -0.053 (0.041) -0.008 (0.032) Girls present (direct observation)/ Girls Enrolled 0.053 (0.043) -0.006 (0.035) -0.027 (0.035) 0.006 (0.028) Average effect over the family of outcomes 0.031 (0.027) 0.009 (0.026) -0.022 (0.029) 0.007 (0.023)

27. Impact on Learning (clustered std. errors in parentheses) T1T2T3 Reading level at endline 0.009 (0.025) 0.014 (0.025) 0.064 (0.026) Math level at endline -0.001 (0.016) 0.014 (0.016) -0.005 (0.017) Writing level at endline 0.009 (0.009) -0.005 (0.008) 0.004 (0.009) N=15,500+; Controls include child age, sex, baseline learning levels, baseline school enrollment status; Reading levels: 0=nothing, 1=letters, 2=words, 3=paragraphs, 4=story; Math levels: 0=nothing, 1=numbers, 2=subtraction or division

28. Closer look at impact on reading Children who could not read at Baseline (N=2288): T1T2T3 First Stage: Attend Read Class IV Impact of Read Class read letters 0.042 (.031) 0.033 (.034) 0.077 (.035) 0.131 (.023) 0.592 (.303) read paragraph -0.006 (.015) -0.013 (.012) -0.007 (.014) -0.051 (.106) read story -0.006 (.01) -0.013 (.008) -0.008 (.009) -0.063 (.074)

29. Closer look at impact on reading Children who could read letters at Baseline (N=3539) T1T2T3 First Stage: Attend Read Class IV Impact of Read Class read letters -0.008 (.016) -0.015 (.014) 0.022 (.013) 0.132 (.02) 0.163 (.098) read paragraph -0.010 (.022) -0.025 (.021) 0.036 (.022) 0.277 (.171) read story -0.001 (.014) -0.010 (.014) 0.033 (.017) 0.258 (.135)

30. Closer look at impact on reading Children who could read words or paragraph at baseline (N=3673) T1T2T3 First Stage: Attend Read Class IV Impact of Read Class read letters -0.001 (.006) 0.006 (.004) 0.006 (.004) 0.074 (.012) 0.068 (.065) read paragraph 0.031 (.019) 0.009 (.019) 0.044 (.017) 0.606 (.271) read story 0.009 (.026) 0.009 (.025) 0.032 (.027) 0.447 (.388)

31. Improvement over time among illiterate children

32. Improvement over time among children who could recognize letters

33. Improvement over time among children who could read words or paragraphs

34. Summary of Impact  Impact on learning driven by local youth volunteering to hold reading classes  Children who enrolled in these classes made significant improvements in reading within a few months  No discernable impact on activity within public schools, or by VECs  No anecdotal evidence of VECs, or Pradhans, or school teachers, supporting these volunteer-led reading classes  Bottomline—evidence of participation to improve learning outcomes, but outside public schools