NVAO’s external quality assurance procedures Ann Van Neygen La Rochelle 11 th of May 2012

2 ‘Higher education’ ‘Government’ ‘Society’ Enhancement Accountability Information Recognition Funding Support Appreciation Support Appreciation Value for € Proof of use Quality Access Quality Access (Consumer) protection Support

Stimulate quality culture  institutional audit Commit professionals / Increase academic ownership  programme assessment Reward earned trust  limited programme assessment Stimulate HE to aim above threshold  accredit as satisfactory, good, excellent 3

Institutional audits Initial accreditation (New) programmes that want to offer a recognised degree All bachelor and master’s programmes, including associate degrees Accreditation programmes that already offer a recognised degree Publication of decisions (and officially recognised degrees) 4

Publication Report Official register (Initial) accreditation Concentrate on content (& focus on performance)  learning outcomes (Initial) accreditation Concentrate on content (& focus on performance)  learning outcomes Audit Focus on policy & practice re. internal QA system  teaching and learning Audit Focus on policy & practice re. internal QA system  teaching and learning 5 Institution Programme Report

6 Effective implementation (by progr. assessment ) Standard 3: Results Standard 1 and 2 : Vision and policy Standard 4: Improvement policy actplan do check Standard 5: Organisation and decision-making structure 1.What is the vision of the institution regarding the quality of the education it provides & of the development of a quality culture? 2.Does the institution have an appropriate policy to realise this vision? 1.What is the vision of the institution regarding the quality of the education it provides & of the development of a quality culture? 2.Does the institution have an appropriate policy to realise this vision? 3.How does the institution measure the degree to which this vision is realised? 4.How can the institution demonstrate that it systematically improves the quality of its programmes? 5. Does the institution have an effective organisation and decision-making structure regarding the quality of its programmes? Positive / Negative / Conditional

Installed by NVAO At least 4 members, including 1 student The panel commands administrative (Board), educational and audit expertise, is acquainted with developments in the higher education sector at home and abroad, and is authoritative One of the members with Board expertise will act as chair The panel is independent (its members have had no ties with the institution over at least the past 5 years) Not part of panel: NVAO process co-ordinator and secretary (also independent of the institution) 7

Conversation on management level Institutional profile: results of previous accreditation Critical reflection by HEI (max. 50 pages + appendices) Site visit 1 st visit (1-2 days): exploration of issues, first interviews and feedback, and panel decision on topics for audit trails 2 nd visit (2-3 days; 2 to 4 weeks after 1 st visit): issues emerging from the meetings & documents studied during 1 st visit; audit trails; at the end feedback to HEI on overall judgement and considerations Audit trails: panel follows the trail from the institutional level to the implementation of policy and/or the management of problems or vice versa 8

Judgement on standards: Meets, does not meet or partially meets the standard Overall judgement: Based on its vision of the quality of the education provided, the board of the institution has implemented an effective quality assurance system, which enables it to guarantee the quality of the programmes offered. Positive, negative or conditionally positive 9

If positive judgement then NVAO approval valid for 6 years If conditionally positive then explicit statement of relevant conditions and positive judgement within 1 year: If conditions are met then NVAO approval for 6 years If conditions are not met then approval expires If negative judgement then approval is withheld for at least 3 years: comprehensive programme assessments additional assessments for those programmes which have already had limited assessment 10

For each standard: Findings Considerations Judgment Improvement suggestions Summary for wider audience Report and decision is published on the site of NVAO 11

12 Extensive programme assessment Limited programme assessment InstitutionInstitution Yes No Is this institution recognised? Recognition procedure Institutional audit Has this institution undergone an institutional audit? Yes No + + decision - - decision

13 Programme assessment Is this programme on the register/ recognised? (Lim./Compr.) initial accreditation procedure (Lim./Compr.) accreditation procedure - - decision + + decision - - decision Extensive programme assessment Limited programme assessment Satisfactory - Good - Excellent Deleted from register Included on register Improvement period

14 Limited programme assessment Extensive programme assessment Assessment of content of the programme focus on performance Detailed assessment of content, policy and procedures 3 standards16 standards Themes 1.Intended learning outcomes 2.Programme and staff quality 3.Achieved learning outcomes & Learning assessment 1.Intended learning outcomes 2.Programme 3.Staff 4.Services 5.Quality assurance system 6.Achieved learning outcomes & Learning assessment Conclusion: unsatisfactory, satisfactory, good, excellent quality

At least four members, one of whom is a student; at least two authoritative subject-matter experts At least one with teaching experience at relevant level; aware of latest international developments in the discipline expertise in the professional field (where applicable) educational expertise assessment or audit expertise; 15

The panel is independent no ties with the institution offering the programme for at least five years; an independent, external secretary trained and certified by NVAO; signed declarations of independence and codes of conduct; panel composition and declarations of independence will be published and made public; the parties involved in the assessment can report any matters that could affect the independence to NVAO. 16

17 ProgrammeModule xModule yModule z... Intended LOs... (C. &) LOs Assessment... (ECTS) Course catalogue Achieved? …

Who ? New programmes: NVAO Recognised programmes: evaluation organisation Information dossier (max. 50 pages + appendices) Site visit Conversations with management, teachers, students, work field Overall judgment and considerations Short feedback to HEI 18

Judgement on standards: New programme: Satisfactory, unsatisfactory Recognised programme: Satisfactory, good, excellent, unsatisfactory Overall judgement: New programme: Satisfactory, unsatisfactory Recognised programma: Satisfactory, good, excellent, unsatisfactory 19

New programmes: If positive judgement then NVAO approval valid for 6 years If negative judgement: no approval Recognised programmes: If positive judgement then NVAO approval valid for 6 years If negative judgement: Limited approval Conditions for improvement If conditions are not met approval expires 20

Generic (basic) quality The quality that in all reasonableness could be expected of a bachelor’s or master’s programme within higher education, and this from an international perspective. Unsatisfactory The programme does not provide generic quality. Satisfactory The programme provides generic quality. Good The programme is of notably higher quality than generic quality. Excellent The programme is of a quality very much above generic quality and fulfils an exemplary role for other relevant programmes. 21

For each standard: Findings Considerations Judgment Improvement suggestions Summary for wider audience Report and decision is published on the site of NVAO 22

23 Application Allocation Admissibility Analysis Proposed decision Final decision Publication  Letter  Institutional details  Programme details  Expert panel report  Letter  Institutional details  Programme details  Expert panel report  By managing director  First handler =policy advisor  Second handler  Portfolio handler = executive board member  By managing director  First handler =policy advisor  Second handler  Portfolio handler = executive board member  Screening application  Panel report  All standards judged  Proof of payment  Completeness  Screening application  Panel report  All standards judged  Proof of payment  Completeness  Template for Analysis  Substantiation judgements  (Objective) findings  (Subjective) considerations  Quality panel report ≠ quality programme  Template for Analysis  Substantiation judgements  (Objective) findings  (Subjective) considerations  Quality panel report ≠ quality programme  Decision NVAO  Procedure followed  Panel composition  Essential programme details  Substantiation decision  Status and period of validity  Decision NVAO  Procedure followed  Panel composition  Essential programme details  Substantiation decision  Status and period of validity  NVAO website  Netherlands: Centraal Register Opleidingen Hoger Onderwijs (CROHO)  Flanders: HE Register  NVAO website  Netherlands: Centraal Register Opleidingen Hoger Onderwijs (CROHO)  Flanders: HE Register

24 Assessment system Internal QA Self-eval report External QA Panel report

25

Open and inclusive approach One framework for all assessments in a given type of procedures E.g. one assessment framework for all types of initial accreditation Fit for purpose (~appropriate burden) Respect for institutional autonomy Internal quality assurance system Assessment in line with institutional choices on the basis of the relevant framework Programme level Institutional level 26 ? ?

Generic, descriptive standards ≠ Narrow, prescriptive and highly formulated  Broad acceptance Focus on what should be done (~fitness of purpose) ≠ How they should be achieved  Do not dictate practice Focus on teaching & learning (~education) Incl. academic standards ≠ Research Incl. professional orientation ≠ Service to society 27

Central documentation for External QA Self-contained document – relatively brief Basic information Institution / programme details Presents perspective shared by (internal) stakeholders Addresses each standard in relevant framework rationale, practice, ambition Provide insight into strengths & weaknesses Maximising strengths? Converting weaknesses into strengths? Mandatory annexes 28

Direct window on what programmes aims to do & what a programme actually does Less focus on policy, process & procedures But process & procedures are easier to assess Facilitates international transparency & comparability E.g. International benchmarking Open to all types of teaching & learning Technology Enhanced Learning / Distance learning Work-based learning / Assessment of prior learning Provides a common language for HE & stakeholders Teaching & Learning 29