Self-Evaluation versus External Evaluation Serban IOSIFESCU Chairman Romanian Agency for Quality Assurance in Pre-University Education

Policy context Policy options : A specific Law on Quality of education (since 2005). Accreditation is part of the quality management systems and procedures National standards used for internal and external evaluation (approved in 2007 and 2008). National specialized institutions for quality evaluation (for pre-university and higher education). Involving school administration (i.e. School Inspectorates and Ministry) in quality control. The results of internal and external evaluation are public. OECD Review of Evaluation and Assessment in Education - http://www.oecd.org/publications/romania-2017- 9789264274051-en.htm Romanian Agency for Quality Assurance in Pre-University Education accomplishes: External evaluation of quality of education offered by educational institutions (pre-primary, primary, secondary, tertiary-non-university levels, including initial VET offered by schools). Provisional authorization, full accreditation and recurrent evaluation of educational institutions.

Self-evaluation versus external evaluation Since 2007-2008, we are able to document the difference between self-evaluation (SE) and external evaluation (EE): schools, usually, overestimate their level of performance (fearing possible reactions from parents and students and sanctions from local authorities and school inspectorate). Interventions in order to develop a common “quality culture” and to diminish the difference between SE and EE: Public events – conferences, workshops etc. Printed materials disseminated – guidelines, manuals, posters, leaflets etc. Digital materials – including video tutorials, promotional videoclips etc. In 2012 – in depth analysis of differences between SE and EE for a sample of about 800 schools (about 15% from the total number) - a chapter in the Report on the level of quality at system level): Persistence of differences, but in both directions: overestimation at SE level (8.6%-25.3% of schools, for different indicators), but also underestimation (13.8% -29,96% of schools). There are are few differences in the overall judgement for institutional capacity then in judging educational effectiveness and the quality management processes themselves. Kindergartens tends to underestimate, schools tends to overestimate.

Self-evaluation versus external evaluation In 2015, two new reports on quality of education were published: the first one, for rural education, the other, general, at system level. Both reports (each one based on representative samples - 10% from the total number of schools) approached this issue, by comparing average scores in SE and EE for each indicator. In rural schools, the difference between SE and EE varied from -5.1% to +16.1% (the average difference being 6.8%). The largest differences, in rural schools, were in overestimating the delivery of counselling and orientation services for students and the quality of the learning facilities (classrooms and equipment). At system level, the difference between SE and EE varied from -7.1% and +11.6% (the average difference being 3.7%). The largest differences, at system level, were in overestimating the quality of the school development plan and the delivery of counselling and orientation services for students, and in underestimating the use of the learning facilities (classrooms and equipment). The rural schools (and, generally, the school functioning in disadvantaged communities) tend to overestimate their performance and are less prepared for SE. The level of over/under estimation decreased in 2015 (compared with 2012).

Operational context, today Self evaluation with improvement (and accountability) purposes at unit level: generating and making public, online, the Yearly Internal Evaluation Report on quality of education (using the quality cycle - ”plan-do-check-act”). External evaluation with accountability (and improvement) purposes at school and system levels: generating and making public, online, the external evaluation reports; using aggregated data (from external evaluation reports) for reporting on quality of education, at system level. Internal and external evaluation processes are available online. The SE and EE reports are public on the page of each school unit. EE improved the overall performance of schools: we compared, in 2015, the performance of 800 schools (evaluated externally in 2011) with the other (not evaluated) schools, using data from three reference years (2009, 2011 and 2014). Implementing the recommendations of the OECD Review of Evaluation and Assessment in Education. New standards are on their way for approval (by Government decision). General use of “Efficiency index”. We will continue to aim SE improvement: our long term purpose is to become unnecessary.

Thank you for your attention!