Concepts and experiences in Quality Management In Higher education In Flanders, The Netherlands, EU, EHEA Prof dr A Govaert

Connection between internal, external quality management and accreditation internal quality management external quality management (SER,visitation) Accreditation

Meeting Gent Prof dr A Govaert Improvement Accountability Assurance Internal quality management Visitation Accreditation InternalExternal Dynamics Stability Connections between internal, external quality management and accreditation Uit: Tijdschrift voor Onderwijsrecht & Onderwijsbeleid nr. 1, 53-60: GARRE, P.

Prof dr A Govaert Connection between internal, external quality management and accreditation internal quality management external quality management (visitation) Accreditation

Prof dr A Govaert QUALITY MANAGEMENT The higher education institutes (university colleges) and the universities are responsible for the internal quality management of the learning activities. –They permanently monitor the quality of the learning activities on their own initiative –They involve students, alumni and external experts from the professional field in processes of internal and external quality control The study programmes, in cooperation with all concerned, are responsible for the realisation of the quality management system of the university in their education

t Prof dr A Govaert INTERNAL QUALITY MANAGEMENT Obliged by the government The organisation chooses the quality control system Preparation to and succession of external quality management  Indirect: helps determining accreditation  Direct: a subject of the framework of accreditation

Prof dr A Govaert

Quality framework on all levels, for all study programmes, for all services,.... General level –Mission –Lines of strength of the policy ( ) –Policy documents –Policy plans departments and services (2010) –Action plans Prof dr A Govaert

Departments –Policy documents –Policy plans Organisation units (study programmes, course units, expertise cells, knowledge and learning centres) –Mission –Key proceedings –General goals –Concrete goals (quality frameworks) Prof dr A Govaert

Check: follow-up Follow up –Measurements and inquirements with feed-back to those concerned –Self evaluations –Follow-up of the policy and actions plans –Reporting (among others, ZER-report) –Follow-up external reporting (i.e. Follow-up of the visitation report) 09/07/ Mieke Vanhoorne

Action: guarantees ECTS 9/07/ Mieke Vanhoorne

Action: input and consultation institutions Academic Council (AR) University negotiation committee (HOC) Overall committee for prevention and protection at work (OCPBW) General student council (ASR) Student forum Education and quality council (OKR) Departmental Council (DR) Departmental Negotiation Committee (DOC) Departmental Student Council (DSR) Regional committee for prevention and protection at work (RCPBW) Education Council (OR) Education Advice Council (OAR) Education Student Council (OSR) Quality Improvement Teams ( KVT’s) Workgroups Mieke Vanhoorne 12 09/07/2016

Prof dr A Govaert Connection between internal, external quality management and accreditation internal quality management external quality management (visitation) Accreditation

Prof dr A Govaert External quality magement Visitations Evaluation of study programmes by a commission of visitation in a published public report  Based upon the self-evaluation report (ZER) and the visitation  Using a collective visitation protocol Base and essential factor for accreditation

Phase 1 The writing of the self-evaluation report (framework) and the composition of the assessment panel. The parties of the study programme to be assessed conduct a critical self-analysis; The results of this self-evaluation are described in a self- evaluation report; VLIR/VLHORA composes an assessment panel in consultation with the study programmes to be visited Prof dr A Govaert

Phase 2 The assessment visit itself by the assessment panel The assessment panel studies the self-evaluation report; The assessment panel prepares for the assessment visit during the preparatory meeting; The assessment panel visits the study programmes. This includes: -discussions with parties from the various sections of the study programme; -study of the available material in situ; -assessment site visits to facilities; -oral reports on the provisional findings, conclusions and recommendations. Prof dr A Govaert

Phase 3 Publication of the public assessment visit report The opinions and recommendations of the assessment panel are recorded in an assessment visit report; The recommendations of the assessment panel are followed by the study programme. Prof dr A Govaert

Banja Luka Prof dr A Govaert EXTERNAL QUALITY management: features Per study programmes or group of study programmes) Simultaneously at all higher education institutes/universities with the same study programmes or group of study programmes (cohort) As a rule, an assessment panel is made up of five or six members: three or four experts, an educational expert and a student. Evaluation of the quality of the study programme Evaluation of the quality control process: PDCA-circle Collection of strong elements and elements that the management should pay attention to Counterstatement report: study programme managers and staff can react Published external report (within 24 months)

The assessment panel members are expected to at least: participate in the preparatory meetings; prepare the visits to the study programmes by thoroughly reading the self-evaluation report and other documents received; participate in the visits and contribute to the proper handling of the interviews with the various interlocutors; contribute to the committee’s final opinion; read, comment on and approve the various parts of the assessment visit report drafted by the secretary of the committee in consultation with the chairman of the committee; participate in the inauguration meetings and the discussion of the general part of the assessment visit report when sub-panels have been established. visit GTG Gent Prof dr A Govaert

The chairman of the assessment panel is also expected to: cooperate on the composition of the other members of the assessment panel in the starting up phase; lead the committee through the entire process (leading the interviews during the visits; preparing and chairing the meetings; taking decisions when there is a lack of unanimity in the committee; etc); monitor together with the secretary the uniformity of the assessment visit process when sub-panels have been established; read, comment on and complement the various parts of the assessment visit report drafted by the secretary of the committee before they are submitted to the committee members; intervene when a member of the assessment panel misbehaves in terms of content or deontology during the activities as part of the assessment visit. visit GTG Gent Prof dr A Govaert

Banja Luka Prof dr A Govaert Evaluation bodies Accreditation request contains the published public report of the independent external evaluation and NOT SER Authoritativeness, Independency, Expertise, of members of the visitation commission

Comparison between the study programmes The comparative chapter that forms part of the general part of the assessment visit report presents the committee’s most important conclusions and recommendations in a comparative framework. This chapter is written on the basis of the final study programme reports. visit GTG Gent Prof dr A Govaert

Banja Luka Prof dr A Govaert Visitations Visitation protocol Guidelines for study progromme Guidelines for visitation commissions Guidelines to draw up the published external evaluation

Banja Luka Prof dr A Govaert Connection between internal, external quality management and accreditation internal quality management external quality management (visitation) Accreditation

Banja Luka Prof dr A Govaert ACCREDITATION The formal acknowledgement of a course based upon a decision of an independent organisation in which it is determined that the study programme meets previously fixed quality and study programme level standards The management of the institute applies for the accreditation – Condition to finance the course by the government – Condition to grant recognized diploma The accreditation organisation NVAO reports annually to the parliament

Banja Luka Prof dr A Govaert Accreditation framework existing courses SIX SUBJECTS –Objectives of the study programma –Programme –Effort of staff –Means / Facilities –Internal quality management system –Results Accompanying ASPECTS (FACETS) and CRITERIA

Banja Luka Prof dr A Govaert Decision rules for accreditation ASPECTS are being evaluated by means of a scale »Excellent »Good »Sufficient »Insufficient The judgement per SUBJECT is established on the basis of the importance of the different aspects that are involved (a way of deliberation) To obtain a POSITIVE FINAL JUDGEMENT every subject has to be at least sufficient –For each main subject –For each location

Banja Luka Prof dr A Govaert Example of a final judgement about a subject SubjectAspect InsufficientSufficientGoodexcellent Criteria for evaluation The end qualifications are adequately translated to learning goals... ProgrammeDemands alignedx… Relation objectives/contentx Coherencex Level of learning possibilities xThe actual period of study equals to the estimated period of study …… Judgement about subject Programme: sufficient Reasons: …

design quality

quality assurance

Quality Results ExpectationsCurriculum

32 EU standards for evaluation and accreditation State of the art Tempus ESABIH 24-25/02/2010

33 1. Bologna process Sorbonne joint declaration (1998): harmonisation of the architecture of the European higher education system Bologna declaration (1999): European higher education area Objectives towards the EHEA: Readable and comparable degrees Two main cycles (undergraduate-graduate) ECTS Mobility Cooperation in quality assurance European dimension in higher education (joint degrees, international parntership) Tempus ESABIH 24-25/02/2010

34 1. Bologna process Monitoring the achievement of the goals: Bologna follow up group Stocktaking Report Two-yearly Ministerial conferences Key element of succes: overall participation and collaboration Important results uptil now -BAMA -EQF -ESG Tempus ESABIH 24-25/02/2010

35 2. ESG Encouraging European cooperaton in quality assurance with view to develop comparable criteria and methodologies Berlin, 2003: A set of standards, procedures and guidelines on QA Peer review system for QA/Accreditation agencies Bergen, 2005: adoption of the ESG Londen, 2007: EQAR Tempus ESABIH 24-25/02/2010

36 2. ESG 3 parts: -Internal quality assurance -External quality assurance -External quality assurance agencies Generic vs specific requirement What should be done vs how it should be achieved Tempus ESABIH 24-25/02/2010

37 2. ESG 3 parts: -Internal quality assurance -External quality assurance -External quality assurance agencies Generic vs specific requirement What should be done vs how it should be achieved Tempus ESABIH 24-25/02/2010

38 2. ESG Part 1: Internal quality assurance in higher education institutions Policy and procedures for quality assurance Approval, monitoring and periodic review of programmes and awards Assesment of students Quality assurance of teaching staff Learning resources and student support Information systems Public information Tempus ESABIH 24-25/02/2010

39 2. ESG Part 2: external quality assurance of higher education Use of internal quality assurance procedures Development of external assurance procedures Criteria for decisions Processes fit for purpose Reporting Follow-up procedures Periodic reviews System-wide analysis Tempus ESABIH 24-25/02/2010

40 2. ESG Part 3: external quality assurance agencies Use of external qualtiy assurance procedures Official status Activities Resources Mission statement Independence External QA criteria and processes Accountability procedures Tempus ESABIH 24-25/02/2010

41 2. ESG Peer review system for QA agencies Reviews every 5 years National review or ENQA organized Process: self-evaluation, independent panel of experts, public report Tempus ESABIH 24-25/02/2010

42 3. Reflections of VLHORA Peer review september 2008 Renewal membership ENQA 2009 Registered EQAR 2009 Interesting and useful but timeconsuming experience Tension: generic ESG vs national framework (f.e. independance of the QA agency, follow-up procedure, processes fit for purpose) Interpretation of the generic ESG (f.e. reporting: different readers have different expectation about the content of the assessment report) Tempus ESABIH 24-25/02/2010

43 VLHORA QA activities since : first generation assessments focus on quality improvement : second generarion assessment focus on quality improvement and accountability (accreditation) two –layerd system: assessments (VLHORA) and accreditation (NVAO) Tempus ESABIH 24-25/02/2010

44 VLHORA 2013: third generation assessment combination of institutional audits ans programme assessments and accreditation Tempus ESABIH 24-25/02/2010

Mieke Vanhoorne 1. Institutional audit The purpose of the institutional audit is to establish whether the institutional management, starting from its vision on the quality of education, is able to set up an effective system of quality management with which it can guarantee the quality of its study programmes. What is the institutional vision on the quality of its education ? How does the institution plan to realise this vision ? How will the institution measure the degree of realisation of this vision ? How does the institution work to improve the quality ? Who is responsible for what ? 45

Mieke Vanhoorne Institutional audit: evaluation process Management consultation (NVAO-institution) Accreditation portrait (NVAO) Critical reflection (Institution) (max. 50 pages) On-site visits –Exploration: discussions –In-depth discussions of key items and audit trails Evaluation procedure (Auditcommission) Advisory report 46

Mieke Vanhoorne 2.Limited evaluation of study programmes The evaluation is the result of a discussion with “peers” about the quality of the study programme and is focused on the following questions: 1.What is the aim of the study programme ? 2.How does the realisation happen ? 3.Have the objectives been achieved ? 47

Mieke Vanhoorne Limited evaluation of study programmes: evaluation process and accreditation Critical reflection (study programmes) (max. 25 pages) –Standards –Projects for improvement –Plans for the future On-site visit Evaluation procedure Evaluation report NVAO-decision: accreditatie 48

THANKS Thanks Questions Banja Luka Prof dr A Govaert