THE „ABC” OF KNOWLEDGE AND POLICY RELATIONS The Hungarian National Assessment of Basic Competences ESZTER BERÉNYI „KNOWANDPOL” PROJECT ELTE UNIVERSITY BUDAEST

STRUCTURE OF THE PRESENTATION How do knowledge and policy interact through the construction of an educational assessment system Fabrication process of the ABC: mutual borrowings and blurring of the boundaries Configuration is in the details Sampling issues The staff of the ABC Double facedness as legitimating force

The Assessment of Basic Competences –Launch: 2001 –2001, 5th and 9th graders; –from 2003 on, 6th and 10th –from 2004, 8th graders –Key competencies (reading and mathematical skills complying with the PISA definition).

KNOWLEDGE POLICY EDUCATIONAL ACHIEVEMENTS NUMBERS, INDICATORS, BENCHMARKS EVIDENCE BASED POLICY MAKING; GOVERNANCE BY NUMBERS; KNOWLEDGE BASED LEGITIMATION OF POLICY MAKING ABC: a central issue in the Hungarian public education policy in the first decade of te 21st century Why? Because of the intensive knowledge-policy relations; knowledge and policy dimensions are not clearly separable form each other

ACCUMULATED KNOWLEDGE HOW TO GOVERN THE AUTONOMOUS SCHOOLS? SYSTEMIC STUDENT ASSESSMENT Evolutions in the field of „educational measurement” and the shifts of problem setting in the educational policy field (what is perceived as a problem to be managed) met and intersected each other. From the Knowledge point of view: the most sofisticated educational measurment – from the policy point of view: an excellent tool for system development and accountability IEA PISA Monitor surveys KNOWLEGDE POLICY decentralization autonomy

FABRICATION OF THE ABC: A PROCESS OF MUTUAL BORROWINGS (Sturdy) Policy goals Scientific- methodological considerations KRT and a process of political knowledge production simultaneously New common vocabulary (“statistical validity”; “reliability”;“summative” measurements; accountability)

Sampling The cohorts/grades to be tested –1st version Testing at the beginning of each school phase (5th and 9th grades) Development in the focus; helping the teachers –2nd version Testing at the end of each school phase (4th, 6th, 8th, 10th grades) Summative type of assessment; feedback; evaluation system „So here in Hungary the diagnostic slipped into the summative type of evaluation. Now they write the Assessment at the end of each school year, and it is used as the most cruel labelling of schools and teachers” (researcher-expert)

The size of the sample –Representative sample information on a special issue; informing policy –Whole sample numbers are not only information but benchmarks; accountability „This is very good, that we have these Competence Assessments in the 4th, 6th and 10th grades. But I will never understand why these are whole sampled and not based on a representative sample. The counter argument is that if it is full-scale, than each school will be faced with its own data. Well, I think, that the goal of this should be to set some standards concerning the most important competences, and than we can publish these standards in a book and the institution decides to use this book or not.” (expert, ex-decision maker)

The added value „Traditional” comparison of schools –„gross” results –Comparison by university entrance exams; national subject area competitions Equality concerns; focus on educational accountability: Filtering the results- two models of added value calculations –Based on Socio-Economical Status calculations –Based on longitudinal surveys – more „accountability friendly” model

Constructing an ABC-compatible staff – configurational positions in the Evaluation Centre Beginning of the 1990s: Evaluation Centre at a research-oriented background institute (OKI) of the ministry 2000: E.C moved to a state-financed subcontractor institute 2004: to another one… 2007: Educational Authority- an agency that carries out educational administrational tasks. From scientific activities to a new kind of role: executive role (processing the routine activities of the cycle of the yearly ABC), with a strong scientific and legal background

Conclusion ABC: Double faced. Its legitimizing force is thanks to the fact that if necessary, in the public discourse, it can be presented as „knowledge” without referring to its policy implications. However, in fact, its power is thanks to the fact that it is not „knowledge”+”policy”, but it is in fact both, at the same time. There is a constant awareness to the rules and expectations of both of the fields. The gradual emergence of a field of its own, the „ABC-field”

Eszter Berényi Thank you for your attention!