From programme accreditation to quality assessment of study programme groups Heli Mattisen 27.03.2013

Agenda Higher education and quality assurance systems in Estonia Principles of the new type of assessment - implementation and lessons learnt

ESTONIAN EDUCATION SYSTEM Master’s programmes 60-120 ECTS credits (1 to 2 years)   Integrated Bachelor’s and 300-360 ECTS or (5 to 6 years) Bachelor’s programmes 180-240 ECTS credits (3 to 4 years) Professional higher education programmes 180-270 ECTS credits (3 to 4.5 years) Doctoral programmes Secondary education

Higher edaucation in Estonia Higher education institutions (HEI) No of HEIs No of students No of students per HEI public universities 6 48 406 8 068 private universities 1 976 state institutions of prof HE 10 10 094 1 009 private institutions of prof HE 9 6075 675 state institutions of VET 2 1516 758 total 29 67 067 2 304 Tere 4

EKKA’s mission EKKA’s mission is, in cooperation with its partners, to promote quality in the field of education and thereby increase the competitiveness of the Estonian society.

EKKA’s core values Equity: Competency: Cooperation: Openness: applying the principles of equal treatment in all its activities Competency: commanding original and up-to-date know-how in its field; including experienced external assessors; engaging in continuous professional development and introducing the results of its activities internationally Cooperation: including partners in development activities, flexibly accommodating their needs and capabilities Openness: being open to new ideas, flexibly reacting to changes in the environment; maintaining the transparency of evaluation procedures and publicising the results

Estonian Higher Education Quality Agency -EKKA Main activities Development of principles and procedures of external evaluation Assessment of the quality of a study program group (SPG) in HE Institutional accreditation (IA) in HE Accreditation of study program groups in VET Training and counseling Publications International cooperation

Higher Education Quality Assessment System in Estonia 1997-2009 2009-2011 2011 ... Accreditation of study programmes STATE RECOGNITION Quality assessment of a study programme group at least once every 7 years Institutional accreditation (voluntary) Institutional accreditation at least once every 7 years TRANSITIONAL EVALUATION Registration of study programmes by the Ministry of Education and Research STATE RECOGNITION: The right to conduct studies in a study programme group

The need for the shift The existing system was resource and time consuming for both HEIs and the state Did not have much impact on the organizational learning and development of the university as an organization The focus of the accreditations was too broad – they were simultaneously focused on controlling and on giving recommendations for further development In the case of a negative decision, the result of the accreditation could be closure of the study programme, thus the honesty in analysing development needs was strongly challenged

Purpos and assessment areas of IA The purpose of IA is to support the development of strategic management and culture of quality in HEIs, inform stakeholders of the outcomes of the main activities thereof, and enhance the reliability and competitiveness of Estonian higher education. Areas of assessment: organisational management and performance teaching and learning research, development and/or other creative activity service to society.

Assessment areas of SPG QA Quality assessment of SPGs focuses on study programmes, the content and organisation of studies and on instruction-related development activities within a study programme group. These aspects are not evaluated in the course of institutional accreditation. Assessment areas: study programme and study programme development resources teaching and learning teaching staff students

1. Study programme and study programme development The content and structure of the study programme are consistent with its objectives and learning outcomes. Different parts of the study programme form a coherent whole. The launch or development of the study programme is based on legislation, development plans, analyses (including labour market and feasibility analyses), and professional standards; and the best quality is being strived for. The study programme development takes into account feedback from students, employers, alumni and other stakeholders.

2. Resources Resources (teaching and learning environments, teaching materials, teaching aids and equipment, premises, financial resources) support the achievement of objectives in the study programme. There is a sufficient supply of textbooks and other teaching aids and they are available. Adequacy of resources is ensured for changing circumstances (change in student numbers, etc.). Resource development is sustainable.

3. Teaching and learning Modern teaching methods are used in teaching. The process of teaching and learning is flexible, takes into account the specifics of the form of study and facilitates the achievement of learning outcomes. The process of teaching and learning supports learning mobility. Assessment of learning outcomes (including recognition of prior learning and work experiences) is relevant, transparent and objective.  

4. Teaching staff Teaching staff with adequate qualifications exist to achieve the objectives and learning outcomes of the study programme, and to ensure quality and sustainability of the teaching and learning. Overall student assessment on teaching skills of the teaching staff is positive. Qualified foreign and visiting teaching staff and practitioners participate in teaching within the study programme. The teaching staff is engaged in professional and teaching-skills development.

5. Students Student places are filled with motivated and capable students. Dropout rates are low, the proportion of students graduating within the standard period of study is large. Students study as foreign or visiting students at other Estonian and/or foreign higher education institutions as part of their studies. Employment rate of alumni is high. Alumni and their employers are satisfied with their professional preparation and social competencies.

Involvement of stakeholders All principal stakeholders of HE provided their inputs during the process of developing the evaluation policies and procedures; educational institutions in particular were included in the development process. Focus group interviews All HEI vice-rectors Steering group and working group

Piloting QA of SPG in 2012 and 2013 SPG Informatics and Information Technology December 2012 – February 2013 Tallinn University of Technology (14 programmes) University of Tartu (5 programmes) Reports and decisions available on EKKA website November 2013 Tallinn University Estonian Information Technology College

Challenges: finding balance The new system of QA is developed to give feedback on study programme groups, in this way supporting their development. This approach enables assessments of a more general nature than the evaluation of a single study programme. Major challenge: to find a balance between two conflicting realities: on the one hand, HEIs expect feedback on every single study programme and; on the other hand the experts` limited time resources, which do not enable them to deal with all study programmes in depth.

Challenges: tailor-made The external quality assessment system goes hand in hand with internal work on quality in universities. The quality agency proposes the areas of assessment and (minimum) standards as well as questions to ask within each standard. The questions in the standards are not compulsory The HEIs are asked to specify the areas where they expect to receive more thorough feedback from experts The study programmes to be assessed within an SPG are chosen in cooperation with the quality agency and the HEI.

Challenges: (self)assessment as a part of every-day life The HEI is expected to provide as little extra data for evaluation as possible, and experts are provided with direct links to the available information sources. The challenge is to find a balance between universities` workloads in preparing their self-evaluations and additional documentation for assessment, and assessment experts` workloads in analysing and contextualising that information.

Challenges: modern technology A tailor-made assessment - structure of self-analyses varies as does that of the experts assessment reports. The more diverse the structure and content of the reports are, the harder it is to fulfil (even describe) or foresee future requirements while developing an electronic environment. The electronic evaluation platform that was intended for the SPG assessment was not that well perceived by HEI nor by experts.

Summary The format and procedure of the SPG assessment was well perceived by piloting HEIs. The reports were considered to be helpful in planning the development of SPGs. Experts` workloads in processing the information as well as training of experts should carefully be planned. The information (and cooperation) needs of HEI, quality agency and experts should be thoroughly studied to continue with the development of electronic assessment platform.