ON QUALITY IN EDUCATIONAL RESEARCH Experiences from evaluations and assessments of projects funded by UVK(CES) Prof. em. Berit Askling

UVK has been allocating research funds since There has been a great pressure on UVK to allocate its funds in the best way possible. Many studies have been carried out on projects funded by UVK.

My purpose is not to judge - the conclusions, relevance and truthfulness of each study but to reflect on the observations drawn from the studies and discuss possible improvements with regard to research quality

DATA 1 Researchers production (applications and progress reports). Surveys: Applications Kim and Olstedt Applications Aasen et al Applications Askling

Data 2 Evaluations: Progress reports Björkqvist et al Progress reports + InterviewsArnot et al

Data 3 UVKs peer review work: grade and priority Applications Aasen et al Applications Askling

SOME FIGURES 1 Surveys: Applications Applications Applications Evaluations Progress reports 28 Progress reports + interviews 14

SOME FIGURES 2 UVKs peer review work: Applications Applications

Some general criticism; many researchers… are imprecise and vague when they present what they want to do, are unclear in their approaches and arguments when selecting theory and method, seem to gather around more or less the same subjects and research areas and work within the same theoretical frames of reference. Research is domestic and relates to domestic needs and problem areas.

Surveys and evaluations have - focused on the scientific and academic dimensions, and - have been carried out without consideration of the context of the projects concerned. On the other hand UVK´s peer-review groups have taken into account a variety of special conditions listed in the Handbook for Reviewers.

The evaluations and surveys undertaken by UVK raise fundamental questions about; -quality in educational research -how quality is evaluated and assessed -how quality in educational research correspond to expectations of different stakeholders

The surveys and evaluations bring to the fore some issues that ought to be discussed What is quality in educational research? How to assess and evaluate quality in educational research? Who is responsible for quality in educational research? On what national and local arenas can quality be assessed and evaluated?

Some questions for discussion: Do we really have a problem with research quality or is it a question of research communication? Should UVK have higher demands on scientific criteria? Are the evaluators too critical (and too decontextualised) in their assessments? Is UVK´s instruction – to support basic research of high quality and at the same time support an infrastructure for research – unrealistic?

Some questions… Is there a lack of academic quality control within educational research (at institutional, faculty board, and department levels)? Are relevance criteria outplaying scientific criteria in educational research? Are departments too dependent on external funding (and on “markets”) for keeping a high level of standard in research?