1. ON QUALITY IN EDUCATIONAL RESEARCH Experiences from evaluations and assessments of projects funded by UVK(CES) Prof. em. Berit Askling 2008-05-06

2. UVK has been allocating research funds since 2001. There has been a great pressure on UVK to allocate its funds in the best way possible. Many studies have been carried out on projects funded by UVK.

3. My purpose is not to judge - the conclusions, relevance and truthfulness of each study but to reflect on the observations drawn from the studies and discuss possible improvements with regard to research quality

4. DATA 1 Researchers production (applications and progress reports). Surveys: Applications 2001-2003Kim and Olstedt Applications 2001-2005Aasen et al Applications 2006-2007Askling

5. Data 2 Evaluations: Progress reports Björkqvist et al Progress reports + InterviewsArnot et al

6. Data 3 UVKs peer review work: grade and priority Applications 2001-2005Aasen et al Applications 2005-2006Askling

7. SOME FIGURES 1 Surveys: Applications 2001-2003178 Applications 2001-2005182 Applications 2006-2007 58 Evaluations Progress reports 28 Progress reports + interviews 14

8. SOME FIGURES 2 UVKs peer review work: Applications 2001-2005182 Applications 2005-2006 58

9. Some general criticism; many researchers… are imprecise and vague when they present what they want to do, are unclear in their approaches and arguments when selecting theory and method, seem to gather around more or less the same subjects and research areas and work within the same theoretical frames of reference. Research is domestic and relates to domestic needs and problem areas.

10. Surveys and evaluations have - focused on the scientific and academic dimensions, and - have been carried out without consideration of the context of the projects concerned. On the other hand UVK´s peer-review groups have taken into account a variety of special conditions listed in the Handbook for Reviewers.

11. The evaluations and surveys undertaken by UVK raise fundamental questions about; -quality in educational research -how quality is evaluated and assessed -how quality in educational research correspond to expectations of different stakeholders

12. The surveys and evaluations bring to the fore some issues that ought to be discussed What is quality in educational research? How to assess and evaluate quality in educational research? Who is responsible for quality in educational research? On what national and local arenas can quality be assessed and evaluated?

13. Some questions for discussion: Do we really have a problem with research quality or is it a question of research communication? Should UVK have higher demands on scientific criteria? Are the evaluators too critical (and too decontextualised) in their assessments? Is UVK´s instruction – to support basic research of high quality and at the same time support an infrastructure for research – unrealistic?

14. Some questions… Is there a lack of academic quality control within educational research (at institutional, faculty board, and department levels)? Are relevance criteria outplaying scientific criteria in educational research? Are departments too dependent on external funding (and on “markets”) for keeping a high level of standard in research?