1 Preparation of the SER as done in Flanders by VLHORA -WELCOME –

2 Preparation of the SER as done in Flanders by VLHORA 1.VLHORA 2.Planning 3.Self-evaluation report 4.Site visit 5.From site visit to assessment report 6.Assessment report 7.Exercise 8.Questions

1. VLHORA VLaamse HOgescholenRAad Flemish Council of University Colleges  Quality Assurance Agency  Umbrella Organisation Official consultation and advisory body All policy aspects regarding  higher education  scientific project research  internationalisation  financing  social services  quality assurance  … 17 staff members, 10 concerned with quality assurance 3

1. VLHORA The VLHORA as a quality assurance agency was appointed by the government to organise the on-site visits of all programmes of the Flemish University Colleges. The objectives of these visits are: To enhance the quality and accountability of the study programmes, To monitor the quality of the study programmes by delivering reports ready for use to the accreditation body, To deliver information concerning the quality of the study programmes to all relevant stakeholders, especially students. 4

1. VLHORA The Flemish Council of University Colleges guarantees a qualitative and indepedent organisation of the on-site visits of the study programmes. The VLHORA relies on expert assessment panels with a double mandate: To judge the quality of the study programmes independently, transparently and based on internationally accepted criteria, To help improve the quality of these study programmes by formulating clear and concrete recommendations. 5

1. VLHORA QA activities since : first generation assessments focus on quality improvement : second generation assessments two-layered system: assessments (VLHORA) and accreditation (NVAO) focus on quality improvement and accountability 2013: third generation assessment combination of institutional audits, programme assessments and accreditation 6

QUALITY ASSURANCE SYSTEM OVERVIEW Improvement Cycle NVAO Accreditation Report Board FORMAL DECISION Accountability Recognition Commission DECISION INDEPENDANCE LAW VLHORA EQA Report Panel Decision Framework Decision Rules Improvement & Accountability SER HEI Programme HEI IQA Improvement & Accountability

8 2. Planning WHAT Formal announcement Information session Cv’s of potential commission members Consultation meeting Composition panel Recognition Commission Inauguration decision WHEN (example) ✐ Autumn 2008 ✐ January/February 2009 ✐ Spring 2009 ✐ Autumn 2009 ✐ Spring 2010 ✐ June 2010 (at the latest)

9 2. Planning WHAT Submission SER Inauguration meeting Site visits Assessment report Request for accreditation (by HEI) End of accreditation WHEN ✐ ✐ Spring/Summer 2010 ✐ Autumn/Winter 2010 ✐ inauguration meeting + 2 years (at the latest) ✐ 2 months after publication AND 6 months before end of accreditation ✐

10 3. Self evaluation report - goal- to stimulate IQM-internal reflection internal preparation study programme to inform the assessment panel !!!Those responsible for the management of the study programme conduct a critical self-analysis !!! The results of this self-evaluation are described in a self-evaluation report !!!SER = important starting point for external evaluators !!!Improvement and accountability

11 3. Self evaluation report - language, length, form & scope- Language, length, form and scope SER limited to 36,000 words Compulsory appendices:  Student numbers  Quantity and quality of staff  Success rates of students In case of:  Several study programmes in one SER  Specialisations in the study programme  Different locations ...  Clear differentiation in the description

12 3. Self evaluation report - level of analysis- Description of state-of-the-art Critical analysis and reflection Problems encountered and how they are dealt with Criteria should be the starting point of the SER Evaluation of process and product

13 3. Self evaluation report - contents- -Introduction (genesis SER, genesis of the study programme, organisation chart...) SIX SUBJECTS (Accompanying ASPECTS and CRITERIA) Objectives of the study programme Programme Effort of staff Means / Facilities Internal quality management system Results -Conclusion (strengths, weaknesses and corrective measures, future policies for the study programme...)

14 3. Self evaluation report - appendices- Obligatory appendices and tables Additional appendices if relevant  Relevant to the analysis of the SER  Pay attention to a transparent classification and make them practical to use

15 4. Site visit  During 2,5 days the assessment panel gathers more information about the quality of the study programme  Discussion with stakeholders of the study programme  Study of the available material in situ  Assessment site visits to facilities  Oral report on the provisional findings, conclusions and recommendations

16 5. From site visit to assessment report First draft assessment report  Response of study programme / HEI Decision of the assessment panel to (not) include the remarks in the final assessment report Final assessment report  Written statement To be included as an appendix to the assessment report Assessment report  Official publication

6. Assessment report INDICATORS are being evaluated by means of a scale: Excellent – Good – Sufficient - Insufficient CRITERIA = sufficient/insufficient is established on the basis of the importance of the different aspects that are involved (a way of deliberation) FINAL JUDGEMENT = sufficient/insufficient To obtain a positive final judgement every subject has to be at least sufficient for each main subject 17

7. Exercise: analysis of a SER Example of a SER in Flanders Consider yourself as a panel member and give a judgement about the subject ‘Quality assurance’, based on the assessment framework 18

7. Exercise: analysis of a SER How are the different aspects concerning the subject ‘Quality assurance’ operationalized? How would you judge the different indicators of quality assurance? Give a provisional score and think carefully about the reasons underpinning your judgement. How would you judge the criteria IQA? Which additional documents would you like to consult during the visit in order to verify your judgement? Which questions would you like to ask to which group in order to verify your judgement? 19

Criteria 6 Internal Quality control Indicator 6.1 evaluation results The course is being evaluated periodaclly through usage of testable targets. Systematic measures for following up on teaching process are introduced. Quality structures are established and quality teaching on the stydu program is permanently monitored Indicator 6.2 Measure for improvement Results of evaluation are the starting point for strategic and operational approach in introduction, imrpovement and development of demostrable measures necesarry for the realisation of the educational goals. Improvement measures are based on threat and weakness notived during evaluations process. Indicator 6.3 Involving co-workers, students, alumnus and professonal field Co-workers, students, alumnus and the professional field are being involved in the internal quality control. 20

Indicator 6.1 evaluation results The course is being evaluated periodaclly through usage of testable targets. Systematic measures for following up on teaching process are introduced. Quality structures are established and quality teaching on the stydu program is permanently monitored Description of the quality policy and the approach of the internal QA? Existence of quality structure? Depersonalised summary of the measured results of the study programme? Usage of results obtained during evaluation process? QA instruments: questionaires, evalutaiontools, etc? 21

Indicator 6.2 Measure for improvement Results of evaluation are the starting point for strategic and operational approach in introduction, imrpovement and development of demostrable measures necesarry for the realisation of the educational goals. Improvement measures are based on threat and weakness notived during evaluations process. Degree to wich past targets were achieved? Degree to which the targets for the future are well founded? Improvement actions in the study programme: allocation of resources, designation of responsibilities and power, planning and monitoring project management? Special attention for the response to findings and recommendations of the former assessment visit and results of student evalutation 22

Indicator 6.3 Involving stakeholders Co-workers, students, alumnus and the professional field are being involved in the internal quality control. Performance of the boards and panels involved in the internal QA, including student participation Involvement of the staff and students in decision making and evaluations as the part of the IQA involvement of alumnus and the workingfield in educational evalutaions and curriculum innovations Contacts between study programmes and the alumnus and workinf field 23

Global conclusions When you write or read an SER always keep in mind the decisionframework  to fulfill the requirements set in the accreditation framework the SER must in any case include information on all aspects of the different indicators To a large degree the SER is decisive for the proceeding of the assessment visit and its effectiveness.  the better the SER the beter the assessment panel will be able to do its work  keep in mind that the reader of your SER is not a person of your study programme, the SER is the first information that the assessment panel get and is the basis for the interviews  give a SWOT at the end of each indicator 24

Global conclusions It is important that all parties that are playing an active roll in a study programme are involved in drawing up the SER  the SER must reflect the vision of the entire study programme Less is more  only give relevant and necesarry information needed for the judgement  only use additional appendices when the are relevant and necesarry, make clear in the SER why they are relevant SER = critical analysis of the study programme, an opportunity to identify strenghts and points for improvement, a proposal of steps that conslidate those strenghts and improve those points for attention. 25

26 8. Questions? ???