1. Keynote 2: Combining programme and institutional aspects in QA: U.K. Nick Harris QAA - UK

2. + some thoughts on the BEST to TEST

3. dimension 1: the HE system - does size matter? a bit of UK geography Scotland19 Northern Ireland2 Wales13 England134 London Manchester and also - does diversity within the system matter ?

4. National variants of QA within UK England (c 134 HEIs) Institutional audit (6 year cycle with mid point check) Scotland (c 19 HEIs) Enhancement-led institutional review (ELIR) peer review panel includes students Wales (c 13 HEIs) Developmental review in Wales Northern Ireland (2 HEIs) as England review is used to indicate less emphasis on accountability but more on improvement

5. dimension 2: the legal context matters Higher Education in the UK is different no national degrees.. but use of word regulated no requirement for accreditation.. but.. formal external QA since 1988 degrees are awarded by autonomous HE institutions HE institutions (universities and univ colleges are responsible for the standard of their degrees and the quality of their study programmes Bachelors and Masters system VERY well established – Bachelors (3 or 4 years) are employable – Masters (1 or 2 years) for specific purposes

6. dimension 3i: maturity matters the agencies we are where we are because of the journey travelled pre-1988 University quality & standards unregulated 1990 CVCP (UUK) Academic audit 1992 Further and Higher Education Act HEQC (audit) + HEFCE (subject review) 1996Major review of HE (Dearing / Garrick reports) 1997i) QAA (combining HEQC, HEFCE unit, Wales, Northern Ireland and Scotland QA offices ii) development of explicit basis for quality and standards to be shared by HE and others 2000/1introduction of revised QA framework (external subject review ended; new QAA)

7. dimension 3ii: maturity and diversity matters institutions are not all the same (they are where they are because of the journey travelled) pre-1960s the old and the ancient universities 1960s expansion phase 1.. conversions and new universities still a binary system + - polytechnics; teacher training colleges; art colleges; the QA universities – External Examiners + some internal QA systems polys – External QA system requiring internal + Ex Ex 1970sexpansion phase 2 Polys converted to autonomous universities 1990sexpansion phase 3 colleges etc become university colleges 2000sexpansion phase 4 degree awarding powers for private sector lot of HE delivered by Further Education Colleges

8. Dimension 4: the purposes of QA In the UK the purposes of QA are to: secure academic standards of qualifications secure quality of learning opportunities accountability (including for public funding of HE) provision of information about quality and standards enhance students learning opportunities and experience QAAs mission: … is to safeguard the public interest in sound standards of higher education qualifications and to inform and encourage continuous improvement in the management of the quality of higher education.

9. Dimension 5: QA procedures a brief history of QA in UK HE internal programme scrutiny external examiners at programme/subject level – appointed by institution there since ? for ever.. and still a key part of QA external institutional audit through different bodies / different guises since 1980s external programme scrutiny through different bodies / different guises from 1980s to 2001 phase i) judgement as description phase ii) judgement as numerical score

10. Dimension 5: QA procedures and frameworks Current UK Quality Assurance Framework regular internal institutional arrangements including student surveys (at programme / module level) external examiners (programme / module level) occasional external audit / review.. by peers published information about quality and standards in individual institutions Academic Infrastructure the reference points for ALL academic standards and QA procedures some accreditation linked to professional practice published national level student surveys (NSS) national collation of statistical returns (HESA)

11. Academic Infrastructure (AI) used across the UK: applied irrespective of a specific QA procedure Framework for HE qualifications (national agreement) subject benchmark statement (subject communities) programme specification (institutional documents) diploma supplement (UK transcript) (formal record of each students achievement) setting out standards

12. using the Academic Infrastructure (AI) Framework for HE qualifications (national agreement) subject benchmark statement (subject community) programme specification (institutional document) diploma supplement (UK transcript) (student record) Code of Practice quality management

13. the Code of Practice the sections cover quality management for key stages: Programme design, approval, monitoring and review ( 7) Admissions to higher education (10) Collaborative provision and flexible and distributed learning (including e-learning) ( 2) Students with disabilities ( 3) Work-based and Placement learning ( 9) Assessment of students ( 6) External examining ( 4) Academic appeals and student complaints on academic matters ( 5) Career education, information and guidance ( 8) Postgraduate research programmes ( 1) Also Accreditation of Prior and Experiential Learning - guidelines [ Credit guidelines ]

14. Briefing visit Briefing paper, SWS, TQI, published reports Audit visit Audit trails Report and its Annex Between visits Last day of audit visit Current QA procedure in England: Institutional Audit

15. Audit trails Sampling trails – normally based on the institutions own way of managing and reviewing its academic standards and quality of provision Supplementary trails – of particular topics proposed by the institution and accepted by the audit team there is no expectation that an audit will necessarily include a supplementary trail is made by the audit team at the briefing visit, following discussion with the institution Selection of audit trails will bear in mind the value added to the audit process by undertaking the trail will also bear in mind that trails are a means of gathering evidence, not an end in themselves their findings are not reported separately but they contribute to the evidence on which the team draws for its findings, judgements and comments

16. On the audit trail... A B Documents about the audit trail Use made of AI Code of practice, FHEQ, Benchmark Statements Programme Specifications Student evaluation Student liaison Internal review – how well does it work for assurance and enhancement ? Handbooks - good information? Evaluation of learning support & resources Students understanding of expectations Assessment policies in practice? Match with level of award? Progression data – used well? View of institutions policies and processes in operation

17. The judgements 'the confidence that can reasonably be placed in the soundness of the institutions present and likely future management of the academic standards of its awards 'the confidence that can reasonably be placed in the soundness of the institutions present and likely future management of the quality of the learning opportunities available to students 3 levels Confidencemost common Limited confidence occasional – serious No confidencerare – VERY SERIOUS

18. Features of good practice & Recommendations Features of good practice - in the context of the institution that make a particularly positive contribution to the institutions approach to the management of the security of academic standards and the quality of provision Recommendations essential – currently putting quality and/or academic standards at risk – require urgent corrective action advisable – have the potential to put quality and/or academic standards at risk – require preventative or corrective action desirable – have the potential to enhance quality of learning opportunities and/or further secure the academic standards of awards

19. dimension 6: the ethos.. (avoiding the quality wars) in the UK ethos underpinning QA (now) encompasses: collegial / partnership respectful of mission respectful of institutions autonomy prepared to celebrate good practice honest and open judgements based on evidence encapsulated in the six questions… what are you trying to do? purposes why are you doing it? reasons how are you going to do it? methods why are you doing it that way? optimisation how do you know it works? effectiveness how can you improve it? enhancement

20. The reports … are not the end Summary - judgements, commentaries, recommendations, features of good practice Report – seven section headings Annex to the report -same section headings as in the report but comprising technical details sharing the information in the audit reports and QA Outcomes from.. thematic papers drawing on evidence in audit reports ELIR themes reports in Scotland

21. Different contexts.. different procedures but a similar direction of travel self regulation [ external quality control ] external quality assurance internal QA with externality risk based QA Austria Germany Norway Switzerland UK Netherlands / Flanders ?

22. Different contexts.. different procedures but a similar direction of travel self regulation [external] quality control external quality assurance internal QA with externality risk based QA Austria Germany Norway Switzerland UK Netherlands / Flanders ? TRUST / RESPONSIBILITY

23. Building trust.. gaining responsibility shared purposes and principles context dependent procedures Purposes – must be clear and shared Public accountability for standards and quality Responsibility for improvement Principles …. must be related to ESG Same principles apply to all types and sizes of institutions Transparency is essential Must underpin policies and procedures Procedures Depend upon context but MUST produce clear evidence – available to all

24. Some practical issues.. Building trust in self regulation –has to be progressive, to an agreed plan, –with clearly identified benefits – and checks a quality culture –changing the ethos and hard work –progressive but to a clear plan from quality assurance to enhancement –doesnt mean a lower level of assurance –does mean a different emphasis –should build to a virtuous circle Light touch –doesnt mean soft touch –is really about shift of touch.. less external / more internal !

25. Some questions? What are the real (shared) purpose (s)? How are the (necessary) standards agreed ? Can / should reports be public? Who selects / how are the experts selected ? How do procedures deal with heterogeneity? Can we avoid procedure become predominant? What about ranking? Why bother? Data, information (?) and knowledge

26. do I stop.. or.. do I go.. ?