1. Mexico: School-Based Management Paul Gertler Harry Anthony Patrinos Marta Rubio-Codina

2. Motivation Increasing number of projects with School- Based Management (SBM) components Relatively little evidence being produced on outcomes Analysis of large-scale compensatory education program in Mexico with SBM component, thru a retrospective evaluation

3. Outline Background on compensatory program $ SBM component Data & identification strategy Results Conclusions

4. Mexico’s Compensatory Programs Supply-side intervention started in 1991 Managed by SEP (Secretariat of Public Education), implemented by CONAFE (Consejo Nacional de Fomento Educativo, or National Council of Education Promotion) Channel resources to the worst performing schools: –Reach the most disadvantaged –Reduce schooling inequalities Increase schooling availability and school quality Fine targeting of rural areas according to marginality Since 1991, program has constantly evolved in terms of targets & interventions

5. CONAFE Interventions Learning Materials Provision: –Provision of school and student supplies Teacher Training AGEs (Programa de Apoyo a la Gestión Escolar, or School Management Support Program): –Monetary incentives and other support to increase parental involvement in school management. Parental Associations exist by law but are rather dysfunctional Other interventions (not evaluated): –Improved technology and infrastructure –Teacher monetary incentives

7. Highlights Each year 4.5 million primary & lower secondary students receive school materials (notebooks, pencils, rulers, geometry kits); 16,000 primary schools & telesecundarias received educational materials & computers Construction or rehabilitation of educational facilities, including classrooms, toilets, playgrounds within schools: almost 10,000 works each year About 13,400 teachers receive monetary incentives intended to reduce mobility & improve teaching; 122,000 teachers in 40,000 schools receive technical & pedagogical feedback Parental organizations from more than 12,000 kindergartens & 47,000 schools receive funds to improve school Initial Education component serves almost 0.5 million parents through orientation in early stimulation & child-care techniques Multicultural and indigenous education activities undertaken

8. Compensatory Education in Mexico Budget$221 million No. of children5.6 million Avg cost per child$50

9. Program Relatively Low-cost $50/student – Compared to: $527 telesecundaria, $477 general middle school

10. AGEs Support & finance training for Parent Associations (APFs) Parents trained in school management of funds transferred to APFs, participatory skills, information on achievements of students & ways parents can help improve learning Financial support to AGEs consists of annual grants transferred quarterly to APFs’ school accounts, from $500-$700 a year, according to size of school

11. AGEs: Example of SBM with Limited Powers

12. More on AGEs Cannot spend money on wages & salaries for teachers Most of the money goes to infrastructure AGEs designed to promote school-parent cooperation AGEs increase school autonomy through participation  Gets parents into the school, thru formal channel: 1.AGEs allow parents to see teacher show up 2.Parents can witness if their children pay attention 3.Participation gives parents power

13. 46% of Primary Schools have AGEs

14. Identification I Treatment Schools: schools intervened (continuously) between 1998 - 2001 Control Schools: schools intervened starting in 2001 or not yet intervened CONAFE Treatment: –Dummy =1 if intervention year –Number of Periods school has received CONAFE OPORTUNIDADES Treatment : –Dummy =1 if intervention year –Ratio of OPORTUNIDADES beneficiaries in the school (intensity) Intermediate School Quality Measures: –Repetition Rates –Failure Rates –Intra-Year Drop Out Rates

15. Identification II Identification II School level dif-in-dif estimation: School FE (  s ): changes in outcomes as a function of changes in treatment status Identification assumption: change in controls measures what change in treatment would have been without AGEs (counterfactual); cannot test directly, but test pre-intervention trends are equal, likely that assumption is fine Test: pre-intervention trends equal between treatment & control schools State * Time Dummies:  st : controls time varying changes in state education policy and other state changes in economic trends Treatment-Specific Time Trends (eg, different evolutions b/w treatment & control schools over time) Time varying school characteristics: intensity of Carrera Magisterial Robust SE Clustered at the School (allow serial correlation within a school over time)

16. Data Sources & Sample CONAFE Administrative Data (1991 - 2002) OPORTUNIDADES Administrative Coverage Data (1997-2003) Outcomes: School Census (Censo Escolar 911), 1995- 2003 2000 Mexican Census & 1995 Conteo 2 sub-samples: –All non-indigenous (general) rural schools –Q34: Non-indigenous rural schools in the top 2 quartiles of the CONAFE 2000 Targeting Index Distribution Sample likely to better balanced Pre-intervention trend

17. Sample Sizes All Schools OP Treatment OP Control Total Number of Schools (%) CONAFE Treatment 1,912 217 2,129 (39.55%) CONAFE Control 2,571683 3,254 (60.45%) Total Number of Schools (%) 4,483 (83.28%) 900 (16.72%) 5,383 Q34 Schools CONAFE Treatment 1,365123 1,488 (36.01% ) CONAFE Control 2,101 543 2,644 (63.99%) Total Number of Schools (%) 3,466 ( 83.88%) 666 (16.12%) 4,132

18. Sources of Variation Number of schools attended increases over time Differences in CONAFE and OPORTUNIDADES phase-in over time & space Intensity of OP treatment (OP Ratio) varies across schools Variation in timing of type of treatment within CONAFE schools

24. Threats to Identification I Placement Bias: interventions are non-randomly allocated –Sign of bias depends on whether better/worse schools are intervened first –Time*State dummies capture shifts/changes in the allocation of resources in each state Treatment status correlated with school characteristics: –If time invariant characteristics, school FE solve the problem –If time variant characteristics… … re-do analysis on the sub-sample of CONAFE Schools that receive all 3 CONAFE interventions (AGEs & Supplies & Teacher Training) Trend * Treatment capture evolution of treated schools over time Results robust although larger estimated AGEs effects Better schools might be receiving AGEs first. AGEs effect might disappear… …CAPAGEs intervention starting in 2003: provide guidance in the administration of the AGEs monetary support

25. Threats to Identification II Treatment might affect the distribution of students’ skills in the school; if so, then treatment correlated with unobserved ability –Check whether changes in total school enrollment over evaluation period –No significant changes in enrollment observed Spillover effects within the school: –School average rate includes OP beneficiary & non-beneficiary students –If positive spillover effects from OP beneficiaries to non-beneficiaries, aggregate school effects can come from learning improvements from beneficiary and/or non-beneficiary students –OPORTUNIDADES effect potentially over-estimated Spillover effects across schools/geographical areas: –Better/worse performing students select themselves into CONAFE schools –Unlikely given: (i) school choice is unlikely to be a dependent variable in rural Mexico (ii) small non-significant variation in enrollment in CONAFE school

26. AGEs are Cheap and Effective While unit cost of compensatory program overall is $50 AGEs are much less costly component Parents at participating school receive $500-$700 a year 45,000 schools; 4.5 million students Total cost of AGE school grants is $26 million a year  Annual unit cost: $5.86 per student oOportunidades – also effective, with other important benefits – pays students at primary level $100-$200 depending on grade  Pathway: Not infrastructure – but Parental Participation oQualitative assessment confirms

27. Conclusions Demand-side intervention have effect on learning outcomes: –nutritional component? OPORTUNIDADES is very effective in keeping students in school Results by intervention show large effects from AGEs that remain even after controlling for other education policies: – decentralization at the local level is effective & low cost Large efficiency gains Difficult to econometrically identify the effect of school input provision No evidence of synergies