ECER September Contested Qualities of Educational Research Session 3a Reflection and Self Evaluation for Teachers and Trainers Gerald Heidegger and Irina Michel

How should Self Evaluation be approached? National and international discussions concentrate too much on quality assurance Development of quality is strongly connected to self evaluation and reflection of teachers and trainers Reflection gives a systematic possibility to dectect actions to improve quality

Aspects of the history of quality management and its tranfer to the educational sector - The idea of quality management stems from production industry (ISO 9000) -During the last decade the idea of quality management has been broadly transfered to non-profit organisations (EFQM – European Foundation for Quality Management) -The first non-profit sector that has taken over quality management was the care sector – soon stressing self-evaluation (because a lack of „hard“ outcomes) -Today a lot of public schools go through evaluation processes. -For disadvantaged young people: Our method QSED (Quality through Self-Evaluation and Development)

Contexts of Evaluation common understanding of the phenomenon that should be evaluated historical and cultural embeddedness of the phenomenon of evaluation national annd international evaluation discussions Output indicators or personal development From ISO 9000 via EFQM to Q2E, QSED different evaluation interests, fears and, targets

Carried out by OECD, CEDEFOP Evaluation trends Carried out by IDEA, IOCE Indicator centered evaluation innovative evaluation approaches targeting empowerment or participation strict, highly structured procedures Measuring soft outcomes

External Evaluation Advantages Seemingly „objective „ Reliable Outcomes Easily transferable results No self-deception No cheating (???) Should be combined with (internal) self- evaluation In order to…. Disadvantages To measure „soft“ outcomes is very difficult/impossible Processes are aiming at the targets of the external evaluation In this way processes are severely narrowed down Teachers/Trainers are objects of external powers They become disempowered They become passive avoid these

staff orientation (3) staff contentedness (7) processes (5) societal responsibility and image (8) resources (4) customer contentedness /satisfaction (6) company results (9) criteria of EFQM politic and strategy (2) leadership (1) Figure 6.1: Criteria of EFQM Existing European evaluation approaches: example EFQM

… basic instrument for focus evaluation …as basis for the development of a quality handbook … as basis for a quality comparison between schools...basic instrument for the development of a school specific evaluation instrument … help for the development of a school specific profile of strengths and weaknesses … basic instrument for the development of a quality model Q2E can be used as... Figure 6.4: Options for using Q2E

How should Self Evaluation be approached?

Quality Development through Self Evaluation with a Web Based Tool?  CVET actors from six European countries were involved in the construction process from the beginning of the project „REFLECTIVE EVALUATION“ The main product: a web based tool is centred on self evaluation of teachers/trainers & adaptable to specific cultural/ institutional needs FOR People at risk in the labour market (at threat of being fired; unemployed; women returners) AND Participants in conventional CVT courses !

Quality Development through Self Evaluation with a Web Based Tool? Interactive web based tool Six dimensions (reflection areas) Four Levels Adaption through the users themselves Pictures, audio, video Multiple methods for reflection Material pool

Six dimensions Reflective Evaluation Self-Evaluation & Quality Development C ollaboration & Networking A ssessment & Recognition Work Conditions Values & Values & Roles Roles Teaching & Teaching & Learning next

Four Levels System Institution Team/Interpersonal Individual

Challenges System Level - Criteria for QM dictated from outside - Stiff quality assurance instead of dynamic quality development Institutional Level - Adaption to market oriented results instead of own aims Team-/Individual Level - Resentment to participate actively - Feeling forced by QM

Solutions System Acknowledgement of „soft“ outcomes Institutions External evaluation combined with internal self evaluation Team/Individual For an easier and continuing adjustment to self evaluation for example our self reliant quality development tool for higher motivation through interactivity and multimedia