School autonomy and student achievement School autonomy and student achievement. An international study with a focus on Italy Angelo Paletta Maria Magdalena Isac Daniele Vidoni

The international evidence: ICCS: across countries, school autonomy does not have a significant effect on student civic knowledge; however, the effect depends to some extent on the country context - although in most countries, the effects are nearly nil, in some contexts the parameters are significantly higher than the average effect . PISA: mixed results indicating both positive and negative effects of school autonomy on educational performance. Differential effects might depend on the area of decision-making under investigation (what is taught, how student are assessed, publicity of results) and also on other characteristics of the educational system such as levels of development and accountability. “autonomy affects student achievement negatively in developing and low-performing countries, but positively in developed and high-performing countries” (Hanushek E., Link S., Woessmann L. 2011)

The international evidence on the effect of school autonomy on civic knowledge

School Autonomy in Italy: some remarks Bassanini Law 1997: schools acquire legal personality Constitutional Reform of 2001: “The autonomous schools are the fundamental brick of the school system” (art. 117.m) Autonomy is: Curricular, Organizational, Financial (ability of diversifying sources of funding) All teachers (~800K) are civil servants employed by the Ministry of Education To the point, school principalship is not a career per se. Rather, it is the “last step of the teaching career” The Law n.59, 1997, concerning the reform of the public administration, introduces the idea of ‘autonomy’ of schools, conferring them the status of ‘juridical subjects’, with powers and prerogatives that were previously exclusive of the central authorities. The Law states that autonomy includes three basic aspects. 􏰀  Curricular and didactic autonomy, as freedom of taking independent decisions on enriching curricular provisions and adopting more effective teaching methods and approaches (design, implementation, evaluation) 􏰀  Organizational autonomy, as freedom of taking independent decisions about the ‘school time’, authority delegation and internal allocation of human resources. However, these decisions are taken within personnel constraints as yearly fixed by school authorities and without opportunity to select or dismiss permanent staff. 􏰀  Financial autonomy, intended as the schools capability to diversify funding sources (public or private), as well as to allocate the State funds without constraints.

School autonomy in ICCS and PISA: the dimensions and the space for action in Italy taking decisions about curriculum planning, curriculum delivery, choice and use of textbooks, appointing teachers, dismissing teachers, establishing student assessment policies, determining the content of in-service professional development programs for teachers, teacher appraisal, budget allocations within the school, extracurricular activities, student admittance policies and establishing teachers’ salaries

Research questions What is the relationship between school autonomy and multiple measures of student achievement (civic knowledge, math and reading) in the Italian context? To what extent is the size and direction of this relationship generalizable across diverse outcome variables?

Method Data sources: The International Civic and Citizenship Education Study (ICCS) – 2009. The ICCS data for Italy matched with national data on student achievement - INVALSI. Samples: International: 32 ICCS countries; 4423 schools; 116118 students. Italian: 153 schools; 2599 students.

Italian data - variables: Data and Methods Italian data - variables: Dependent variables – Civic knowledge, mathematics and reading performance (cognitive tests - α > .80). Explanatory variable - School autonomy Control variables Student characteristics: age, gender, SES, language spoken at home School context: location, size, teacher student ratio, average SES. taking decisions about curriculum planning, curriculum delivery, choice and use of textbooks, appointing teachers, dismissing teachers, establishing student assessment policies, determining the content of in-service professional development programs for teachers, teacher appraisal, budget allocations within the school, extracurricular activities, student admittance policies and establishing teachers’ salaries. Method of analysis for Italian data: multivariate multilevel analysis (students within schools; regression model estimated for the three dependent variables simultaneously)

Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy

Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy M1 student control variables are related to student achievement in all domains. Together, the control variables explain: 17% of the total variance in civic knowledge, 8% of the total variance in reading (Italian language) and 2% of the total variance in the mathematics achievement

Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy M2 school autonomy shows: a statistically significant negative relationship (β = - 0.105, se = 0.033) with all three outcomes the effect is common for all achievement measures and explains nearly 1% of the total variance in civics, reading and mathematics

Results of multivariate multilevel analysis to explain variation in students Civic Knowledge, Italian language and Mathematics for Italy M3 the negative effect cannot be attributed to higher levels of autonomy In fact: only the linear term shows a statistical significant effect (β = - 0.109, se = 0.035) the effect of the quadratic term is almost nil (β = 0.006, se = 0.013)

Discussion and conclusions For Italy: higher levels of school autonomy are associated with lower student achievement irrespective of the subject in which achievement was measured. Possible leads for interpreting results: Especially in elementary and middle school, the autonomy is only partial: Organizational and didactic NOT really financial: limited funding diversification, constraints to use of (human) resources Contradictions of a low school autonomy, but an even lower accountability: to the point mostly bureaucratic accountability, little emphasis on learning outcomes  scarcity of feedback processes to school improvement Limitations and steps forward: Findings should be further explored by looking at different effects of different areas of decision-making as well as measures of school accountability.

Conclusions and future research Especially in the first phase of education, we would be confronted with a mere partial view of all the available decision-making levers Contradictions of a school autonomy "in midstream” Contradictions of a low school autonomy, but an even lower accountability! Especially in the first phase of education, which is the focus of our research, we would be confronted with a mere partial and unsatisfactory view of all the available decision-making levers of the school autonomy. For example, the ability to diversify funding, but having to pay for the internal organizational rigidity regarding the use of time and functions of teachers, or the contract between the high teaching autonomy and rigidities in the use of teachers, but also use of space and equipment that are part of the budgets of municipalities A second factor is no less important to try to explain the contradictions of school autonomy in Italy. The lack of accountability as a mechanism to balance school autonomy. Today in Italy, as will be evidenced by many papers on this topic, it is a process of great change that will lead to systematic evaluation of schools and school leaders. In perspective, the self-evaluation in schools and external evaluation should be used to identify areas of weaknesses and develop plans for improvement. However, if we look back, we noted that the autonomist process has been accompanied by formal mechanism of bureaucratic accountability, with little emphasis on learning outcomes, cognitive and non cognitive outcomes.

Thank you for your attention