1 A Randomized Evaluation of a School Scholarship Program in Rural India Karthik Muralidharan (with Michael Kremer, Venkatesh Sundararaman) Conference on Public Private Partnerships in Education World Bank, Washington DC 7-8 June, 2007

2  Background  Research Questions & Experimental Design  Questions for Discussion/Feedback Outline

3 Private Schools in Rural India  Private fee-charging schools are widespread in rural India -Especially in areas with poorly performing public schools -28% of villages had private schools in 2003 (probably much higher now) -18% of school enrollment in rural India is in private schools (2006)  Measures of private school performance are superior to public schools – especially with village fixed effects -Teacher attendance, teaching activity, student attendance, test scores  Children attending private schools come from advantaged backgrounds relative to those attending public schools -Difficult to infer causal effect of private schools on performance -Omitted variables and also additional years of schooling -High levels of elite exit from public schooling  Several important questions need to be answered to assess the suitability of a voucher/scholarship/PPP model for improving school education outcomes in India

4 Andhra Pradesh Randomized Evaluation Study  3-way partnership between Government of Andhra Pradesh, Azim Premji Foundation, and the World Bank  5 year MoU with GoAP to systematically study the most promising policy options to improve developing country education via randomized allocation of programs  4 major interventions have been implemented and evaluated so far (Muralidharan & Sundararaman 2006) -Group and Individual-level performance pay for teachers -Use of contract teachers as opposed to civil-service teachers -Cash block grants to schools  Starting a long-term study on the impact of school choice/scholarships/vouchers for disadvantaged children to attend private schools under AP RESt

5 Existing Research on School Vouchers  Several studies consider the impact of randomly offering students a chance to attend private/charter schools -Rouse in Milwaukee -Peterson et. al in New York City, Washington DC, and Dayton, Ohio -Angrist et. al in Colombia -Cullen et. al in Chicago -Hoxby & Rockoff in Chicago  These report varying extent of benefits to voucher recipients -No one finds voucher recipients doing worse off  But more evidence is needed on the overall impact of school vouchers on all the students in a system -Hoxby (2003), Lavy (2006), Hsieh and Urquiola (forthcoming)

6 Decomposing Peer Effects  Randomly allocated voucher programs typically create 4 distinct categories of students -In public school, don’t apply for voucher -Apply for voucher, but don’t win (and typically go to public school, but sometimes go to private school anyway) -Apply for voucher, and win (and likely to go to private school) -In private school, independent of the voucher program  Best studies typically compare the 2 nd and 3 rd groups -ITT and ToT measures  But we don’t know if the first and fourth groups are worse off as a result of peer effects -The public school potentially loses its most motivated students -The private school gets students below its existing average  Ideally, we would want to randomize entire communities into voucher programs and compare the result for all students with the results of all students in similar “control” communities without vouchers

7 Proposed Research Design  An Indian village is pretty close to a “closed economy” in terms of school choice – especially for primary schooling  Identify ~200 villages that all have an existing private school  Two stage randomization -Randomly select half the villages to receive school vouchers -Randomly select children in “voucher villages” to receive them  Track learning outcomes of all children in all schools in both treatment and control villages -Child-level comparison gives the impact on participants -Village-level comparison gives the overall effect of the voucher program  Phase 1 (Pilot) with ~32 villages starting now, with a planned expansion to ~200 villages next year

8 Scholarship Program Design (1 of 2)  The universe of children eligible for the scholarship will be those currently in government schools (grades 1-3) -Provides a sample of disadvantaged children without having to do means testing  The number of scholarships to be offered in a village will be capped at ~30% of the enrollment of all the government schools -Don’t want to empty out the government school -Parents will be required to apply for the scholarship to be eligible (should provide higher first stage but we will find out)  Once a child receives a scholarship, he/she will continue to receive one till the end of primary school subject to meeting attendance requirements and taking the end of year tests -Differential exposure to program across cohorts of recipients

9 Scholarship Program Design (2 of 2)  No topping up – scholarship amount will be set at around the 80th percentile of the private school fee distribution across all villages -Private schools can determine the number of such places at this scholarship rate (but must accept all students who are allocated to these places by a lottery – limits cream skimming)  All expenses for books, uniforms, and school supplies are being covered by the scholarship. -A transport subsidy may be provided in some cases, but may not be required if the choice is being exercised within the village. -The total scholarship spending per child (all inclusive) is expected to be around Rs. 3,200/child per year (USD 80/year). -This is significantly less than the spending per child in the government schooling system  At least Rs. 4,000/year counting only pure variable costs  Over Rs. 5,000/year including various overhead costs

10 Resource Equalization  Even if the “scholarship villages” do better it could be a reflection of additional resources in these villages -Resource equalization is a problem for many other studies as well  Possible solutions: -Make public school lose money for every student who leaves  Not easy since the main expense is very lumpy (teacher salaries)  Also politically much more difficult -Provide matching resources to the public schools in control villages  Doubles “scholarship cost” of program  Operationally more difficult because it becomes another “treatment” -Use estimates from other studies to net out the effect of additional resources in public schools  Most likely course of action – especially since this in Andhra Pradesh  Existing studies in AP will provide estimates of the returns to the 2 main categories of inputs to public schools – additional teachers, and cash grants with school-level flexibility on spending (no ‘infrastructure’ though)

11 Questions We Are Getting At  Do private schools perform better even after accounting for the unobserved variables that might determine private school enrollment? -Answer by comparing scholarship winners to losers in voucher villages -Can think of the question as whether marginal spending on education is best routed via the private sector  What is the aggregate impact of the program (and is any group worse off)? -Answer by comparing voucher villages to non-voucher villages -Unit of analysis is the grade-level average score  What is the functional form of peer effects?  How do parents exercise choice?

12 Questions We Are Not (Yet) Getting At  Effects of competition on productivity of public schools? -Not at this point – negative incentives for public schools are quite difficult practically (lumpy inputs), politically, and potentially ethically -Positive incentives for retention could be considered (your thoughts?)  Benefits of better matching? -Unlikely to be the focus at the primary level  Adequacy and nature of supply response? -Not at this point -Scholarships will only be redeemable at schools existing prior to study  Combining choice with the effects of information to parents on school performance? -Not in Phase 1, but could be an orthogonal/additional treatment

13 Issues for Discussion (1 of 2)  Take up -Aspiration gap, uncertainty of funding/unanticipated expenses -Larger distances to travel -Have required an expression of interest in applying (want a good first stage), and the number appears high (~90%)  Medium of Instruction/SR Adjustments -Most private schools are “English Medium” -Need to allow enough time for SR adjustments to be made  What grades to target? -Had initially intended grades 1-4 (moving to 2-5) -Now only looking at KG, 1, 2 (moving to 1, 2, 3)  Scholarship amount? -Don’t want top ups, 80 th percentile makes it attractive for most schools and covers marginal costs for almost all schools

14 Questions for Discussion (2 of 2)  Transport - to provide or not? -More competition versus greater complexity -Very hard to provide a “standard” option for transport. -Best option may be to provide a subsidy to parents -Piloting both options  Thoughts on ways to provide incentives for the public school to retain children without explicit penalties  What expectations should the public school teachers be operating under?  How much and what kind of information should we provide to parents?  Thoughts on verification/payment/fraud prevention?