1. The impact of decentralization on schools and student performance Elizabeth M. King Development Research Group The World Bank February 23, 1999

2. Elizabeth M. King The World Bank, 2/3/99 Presentation Outline o Three cases of decentralization to the local level o Evaluation issues o Evaluation strategies o Results o Lessons

3. Reforms in school management Premise is that “if local participants regain a sense of agency … they can be a powerful force for school improvement.” Why?

4. Elizabeth M. King The World Bank, 2/3/99 Because actors with the best information about a subject could make better decisions more quickly about that subject...

5. Elizabeth M. King The World Bank, 2/3/99 … and because teachers often work autonomously inside the classroom since centralized school systems are seldom tightly monitored.

6. Elizabeth M. King The World Bank, 2/3/99 Reforming school management o A key policy issue is how to design incentives and contracts to ensure that teachers and school heads behave in accordance with their mandate. Public School Principals Teachers

7. Elizabeth M. King The World Bank, 2/3/99 Reforming school management o Reduce agency problems within schools by increasing monitoring, control, and influence by others. Public School Principals Teachers Parents Local Community

8. Elizabeth M. King The World Bank, 2/3/99 Cases of school management reform o Chicago School Reform, 1989/90 o El Salvador’s community-managed schools or EDUCO (Educacion con Participacion de la Communidad), 1991 o Nicaragua School Autonomy Reform, 1993

9. Elizabeth M. King The World Bank, 2/3/99 Cases of school management reform o Chicago School Reform, 1989/90 »Local School Councils (LSCs): representatives of parents, community members and teachers »LSCs hire, fire and evaluate school principal; advise on curriculum, textbook selection, and student discipline; approve budget and School Improvement Plan »Additional resources to support school improvements

10. Elizabeth M. King The World Bank, 2/3/99 Cases of school management reform o El Salvador’s community-managed schools or EDUCO (Educacion con Participacion de la Communidad), 1991 »Community education associations (ACEs): parents and community members »Legally responsible for operating EDUCO schools »ACEs hire, fire, supervise teachers, administer fund transfers from central government, raise additional resources

11. Elizabeth M. King The World Bank, 2/3/99 Cases of school management reform o Nicaragua School Autonomy Reform, 1993 »Consejo Directivo: principal or director, elected teachers, parents, and students »Makes decisions pertaining to personnel, school budget and plan, finance, and pedagogy »Revenues from students retained in the school

12. Elizabeth M. King The World Bank, 2/3/99

13. Elizabeth M. King The World Bank, 2/3/99 References o Chicago »Consortium on Chicago School Research o El Salvador »DEC Research Group in collaboration with Ministry of Education of El Salvador o Nicaragua »DEC Research Group in collaboration with Ministry of Education of Nicaragua and international consultants

14. Evaluating decentralization reform We know less about decentralization than we claim to know.

15. Elizabeth M. King The World Bank, 2/3/99 Questions for evaluation o How are the new structures functioning? o Is there really greater local decision- making? More local participation? o How are the schools using their greater autonomy and accountability? o Has the reform influenced the classroom and student outcomes?

16. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues  The logic of the reform is that the mandated change in school governance will expand local decision-making and participation and thereby lead to advances in student engagement and learning. Indicators »Local decision-making »Local participation »Student outcomes

17. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues o If participation in a reform is voluntary or if the reform content is subject to choice, it may be difficult to measure impact. »Need to explain reform participation separately from impact »Choose appropriate evaluation design

18. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues o Decentralization is often not the only reform taking place. How to isolate its effect from other changes, such as an increase in available school resources? »Identify other relevant policy changes »Need for appropriate evaluation design

19. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues o Considerable diversity among communities and schools -- in resources and circumstances -- will mean variability in response to the reform. »Examine diversity »Identify factors explaining diversity

20. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues o By its nature, decentralization itself can take a variety of different forms (e.g., school improvement plans). »Need to describe new management structures and define evaluation problem accordingly

21. Elizabeth M. King The World Bank, 2/3/99 Evaluation issues o The reform may be fairly new at the start of an evaluation. When is it too early to assess the reform? »Mostly a process evaluation, a mid-course assessment »May be difficult to assess long-term impact on student outcomes

22. Elizabeth M. King The World Bank, 2/3/99 Evaluating impact o Evaluation method »Chicago: Surveys with recall, and case study method »El Salvador: Matched and reflexive comparison using school and parents survey data »Nicaragua: Matched and reflexive comparison using school and parents survey data, plus qualitative methods

23. Elizabeth M. King The World Bank, 2/3/99 Evaluating impact o Instruments (Chicago) »Quantitative evaluation –Survey of teachers (1991) and principals (1992) »Case study through observation and interviews in selected schools (1993)

24. Elizabeth M. King The World Bank, 2/3/99 Evaluating impact o Instruments (El Salvador & Nicaragua) »Quantitative evaluation –School survey with questionnaires applied to principals, teachers, and students –Parents survey linked to school survey –Sample of reform and non-reform schools o Nicaragua: Focus group meetings with principals, teachers, council members

25. Elizabeth M. King The World Bank, 2/3/99 Evaluating impact o Performance indicators »Local decision-making: by whom and which decisions »Local participation: by whom and for what »Perceptions and attitudes toward the reform by principals, teachers, parents, council members »Student outcomes -- enrollment, repetition, continuation, achievement test scores, and attendance

26. Elizabeth M. King The World Bank, 2/3/99 Evaluation findings

27. Elizabeth M. King The World Bank, 2/3/99 Chicago, IL o Local decision-making and participation »One-half of teachers agree they have more influence on policy »Teachers more involved with reform also more positive about school operation »In small schools (<350), teachers significantly more positive about the LSCs and the reform òSource: Consortium on Chicago School Research

28. Elizabeth M. King The World Bank, 2/3/99 Chicago, IL o Local decision-making and participation »Composite index of principals’ attitudes: –36% very positive –31% moderately positive –22% somewhat negative –11% very negative »More positive when teacher collegiality high and community relations positive; small schools òSource: Consortium on Chicago School Research

29. Elizabeth M. King The World Bank, 2/3/99 Chicago, IL o Student engagement »No significant change in attendance rates after reform; already high prior to the reform »Substantial declines in student mobility »A few schools showed gains in enrollment òSource: Consortium on Chicago School Research

30. Elizabeth M. King The World Bank, 2/3/99 Chicago, IL o Learning »Drop in grade repetition in all case schools »Principals report some improvement in test scores, but principals cautious about interpreting short-term gains òSource: Consortium on Chicago School Research

31. Elizabeth M. King The World Bank, 2/3/99 El Salvador o Local participation »Higher participation by parents and school councils in EDUCO schools than in traditional schools

32. Elizabeth M. King The World Bank, 2/3/99 El Salvador o Local participation Hours/month teachers meet with parents 5 3.5 EDUCOTraditional 4.5 1.5 Number of ACE visits to the classroom EDUCOTraditional

33. Elizabeth M. King The World Bank, 2/3/99 El Salvador o Student enrollment and learning »EDUCO students have 3 fewer days of absence per year than non-EDUCO students; stronger result for newer EDUCO »Despite being poorer, EDUCO students do as well as non-EDUCO students in achievement tests

34. Elizabeth M. King The World Bank, 2/3/99 El Salvador Assets of students’ families Own HouseWith ElectricityWith Sanit.With Water

35. Elizabeth M. King The World Bank, 2/3/99 El Salvador Student engagement and learning Days missed Math score Language score

36. Elizabeth M. King The World Bank, 2/3/99 El Salvador o Student engagement and learning »EDUCO students have 3 fewer days of absence per year than non-EDUCO students; stronger results for newer EDUCO »Despite being poorer, EDUCO students do as well as non-EDUCO students in achievement tests »ACE visits have positive effect on language scores and number of days present

37. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Questions for evaluation »Did the reform promote school autonomy? Why would it not? »School may choose not to exercise autonomy or lack of resources to do so »Reform may not be clear »Local actors may diverge in interpretation

38. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Questions for evaluation »Did the reform promote school autonomy? »Is the level of autonomy related to the degree of influence perceived by principals, teachers, council members? While participants and power-sharing are part of the reform, previous governance style may be obstructive.

39. Elizabeth M. King The World Bank, 2/3/99 Nicaragua Dec. 1995  Dec. 1996  Dec. 1997 School-householdSchool- household survey Isurvey II Gr 3 & Yr 2 Grs 4,5 & Yrs 3,4Achievement tests I test II Gr 4 & Yr 3 Qualitative evaluation Visit I Directors, teachers, parents Evaluation design

40. Areas of school decision-making o Salaries and incentives »Setting salaries »Establishing incentives for teachers & administrators o Personnel »Hiring & firing teachers, directors, administrative personnel o Classroom & pedagogy »Determining class size »Designing curriculum »Selecting textbooks o Maintenance & infrastructure »Developing infrastructure projects »Repair buildings o Administration »Planning school budgets »Distributing textbooks »Informing community »Relations with teachers’ union o Teacher supervision, evaluation & training Elizabeth M. King The World Bank, 2/3/99

41. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Local decision-making and participation »Data source: Questionnaire for principals, teachers, council members on 25 decision areas »De jure autonomy not equal to de facto autonomy »Considerable diversity in de facto autonomy among program schools »Evidence of spillover to non-program schools

42. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Local decision-making and participation

43. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Local decision-making and participation »Additional findings: –High consistency in reports by principals, teachers and council members –De jure autonomy is the only statistically significant determinant of de facto autonomy, especially with respect to administrative functions

44. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Local decision-making and participation »Perceptions of local actors –Less than one-half of respondents were positive about parent participation –Positive view correlated with duration of de jure autonomy

45. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Local decision-making and participation Findings from focus group meetings »Disparate understanding of role of council and parents in new structure »Directors and teachers see reform as mainly administrative »Parent participation on the rise -- but a few instances of hostility between parents and teachers

46. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Student engagement and learning Findings from focus group meetings »Reform interpreted as putting pressure on teachers and parents to improve student performance »Focus on achievement led teachers to be more innovative, to assess student learning more regularly, to have more contact with parents

47. Elizabeth M. King The World Bank, 2/3/99 Nicaragua o Student engagement and learning Perceptions of local actors: »About half of respondents (40-60%) were positive about student achievement. Small percentage (5-14%) were negative. »Over half (55-65%) were positive about teachers’ attendance; parents least positive. »Positive view correlated with duration of de jure autonomy.

48. Elizabeth M. King The World Bank, 2/3/99 Lessons from the field

49. Elizabeth M. King The World Bank, 2/3/99 About the process of reform o Time essential to clarify new roles in new governance structures o Time essential to effect real systemic change o There is no one reform across schools o Conflict and anxiety may increase as change occurs o Past governance style and school- community relationship influence adaptation to reform

50. Elizabeth M. King The World Bank, 2/3/99 About impact on learning o Short-term perceptions and test gains suggest improvement o But cautious optimism about long-term gains recommended o Leading indicators may be important -- such as decline in student absenteeism o Important to consider diversity among schools and implications about equity

51. Elizabeth M. King The World Bank, 2/3/99 About evaluating decentralization o Three-part data collection important »School and household characteristics »Student achievement tests »Focus group meetings on specific issues: Perceptions by school personnel and parents o Different stakeholders contribute different perspectives about reform