Exploring chapter B7 of the UK Quality Code and what external examining means to Institutional Review (IRENI) The Quality Assurance Agency for Higher Education. Registered charity numbers and SC Maureen McLaughlin Assistant Director & IRENI Method Coordinator, Reviews Group, QAA

Overview Context for B7: -UUK/GuildHE review -UK Quality Code for Higher Education B7: External Examining -Key points Importance of external examiners to Institutional Review -Scrupulous use

B7: External Examining – context QAA’s review of the Academic Infrastructure UUK/GuildHE, Review of external examining arrangements in universities and colleges in the UK Recommendations: o 3 principles – institutional autonomy, institutional accountability, transparency o national set of minimum expectations of the role o national set of criteria for appointment o support for external examiners o core content of reports o response to reports o concerns procedures, internally and externally o student engagement

UK Quality Code for Higher Education Setting and maintaining threshold academic standards Part A Assuring and enhancing academic quality Part B Information about higher education provision Part C Expectations: Things higher education providers expect of each other and which the general public can expect of all higher education providers of ‘sound practice’ Replace precepts Indicators

Chapters of the Quality Code A1: The national level A2: The subject and qualification level A3: The programme level A4: Approval and review A5: Externality A6: Assessment of achievement of learning outcomes B1: Programme design and approval B2: Admissions B3: Learning and teaching B4: Student support B5: Student engagement B6: Assessment of students and accreditation of prior learning B7: External examining B8: Programme monitoring and review B9: Complaints and appeals B10: Management of collaborative arrangements B11: Research degrees Part A: Setting and maintaining threshold academic standards Part B: Assuring and enhancing academic quality Part C: Information about higher education provision General introduction

B7: External Examining 18 indicators of sound practice: defining the role (Indicators 1-3) nomination and appointment (Indicators 4-6) carrying out the role (Indicators 7-10) recognition of the work of external examiners (Indicator 11) external examiners' reports (Indicators 12-16) serious concerns (Indicators 17, 18) Expectation: Higher education providers make scrupulous use of external examiners

Defining the role (Indicators 1-3) assurance scrutinise whether threshold standards are being maintained assessment process measures student achievement rigorously and fairly standards and student achievements are comparable with other HEIs enhancement identification of good practice opportunities to enhance the student learning experience impartial, independent advice, informative comment, recommendations

Nomination and appointment (Indicators 4-6) institutional processes and regulations – appointment, support, termination national criteria (‘core+’ approach) for appointment, emphasis: o competence for the role - standards - enhancement - experience o minimise potential conflicts of interest preparation for the role – provider’s role and responsibilities

Carrying out the role (Indicators 7-10) clear ‘instructions’: o appointment to what modules/programmes/awards o roles, responsibilities, powers: o membership of exam/assessment boards o sampling of assessed work transparency about external examining/examiners endorsement of assessment outcomes evidence to carry out the role: opportunities to meet students, purposes availability of evidence communication with staff

Recognition of the work (Indicator 11) ensure ‘supply’ of external examiners o create space o develop forms of recognition o support aspirations benefit from external examiners’ experience o communicate appointments o disseminate experience

External examiners' reports (Indicators 12-16) annual report, designated recipient core content, aligned with role definition standards enhancement and also - confirmation of ‘sufficient evidence’ - confirmation that previous report issues addressed - professional body issues - end of term of office overview reports available to students consideration of reports and actions in response feedback to external examiners

Serious concerns (Indicators 17, 18) confidential report to the head of the institution QAA Concerns scheme (or the PSRB): concerns which indicate serious systemic or procedural shortcomings, not isolated mistakes or occurrences of bad practice QAA's concerns scheme: guidance for external examiners, /Documents/QAA_concerns_scheme_guidance_external_e xaminers.pdf /Documents/QAA_concerns_scheme_guidance_external_e xaminers.pdf

Issues arising to date... definition of role and how that is applied by the HEI Workload application of the appointment criteria sharing of reports with students as part of wider QA processes (nb B5 & Part C). the actions taken in response to the EE

External Examining: The student perspective The Quality Assurance Agency for Higher Education. Registered charity numbers and SC037786

B7: External Examining Indicators of sound practice: Information about external examiners (Indicator 8) Sharing of reports (Indicators 14, 15, 17) Meetings with students (Indicators 8, 10) QAA Concerns scheme (Indicator 18) Expectation: Higher education providers make scrupulous use of external examiners

Context Public confidence in academic standards and external examining The role of students in HE quality management

To secure widespread confidence the system should be more open, so that students know what external examiners do, what they say about programmes, and how institutions respond to this.... A clear entitlement for students to know what they can expect from external examiners regardless of subject, course or university/college. We believe there is much to be gained from a positive, confident and open approach to reports. (UUK/Guild HE, 2011)

Chapter B5: Student engagement Students as partners in quality systems to enhance their learning experience Opportunities individually and collectively Sharing of information to enable students and staff to have informed discussions Part C – information about HE provision HE providers set out what they expect of current students and what current students can expect of the HE provider (Ind 5)

Information about external examiners Indicator 8 Institutions include the name, position and institution of their external examiners in module or programme information provided to students. Transparency No direct contact – appeals mechanisms If direct contact received

Sharing reports Indicator 14 Institutions make external examiners' annual reports available in full to students, with the sole exception of any confidential report made directly, and separately, to the head of the institution. Indicator 15 Institutions ensure that student representatives are given the opportunity to be fully involved in [considering reports] enabling them to understand all the issues raised and the institution's response.

Sharing reports (2) Information provided as part of involving students in quality management Not written for students per se No requirement to publish History (HEFCE 06/45, UUK/GuildHE) HEI decides the mechanism for sharing Edited only in specified circumstances

Sharing reports (3) Involvement in considering reports and actions to be taken or not taken At programme/subject level At institutional level Consideration of general issues and themes

Meetings with students No view on whether desirable Clarification of whether entitled to meet students (Ind 10) Clarification of purpose for external and students (Ind 8, 10) QA, assessment, orientation?

Website: Sign up to QAA News alert.aspx Find out more

External Examining: in Institutional Review (IRENI) The Quality Assurance Agency for Higher Education. Registered charity numbers and SC037786

Aligned to the standards element of IRENI (expectation 2) One of the 5 expectations in this area Also has resonance in: quality of learning opportunities (CP, PGR), Information (sharing EE reports) and enhancement (use of outcomes of QA for QE)

5 pillars of academic standards (1) Each qualification (including those awarded under collaborative arrangements) is allocated to the appropriate level in the FHEQ. (2) Higher education providers make scrupulous use of external examiners. (3) Design, approval, monitoring and review of assessment strategies is effective in ensuring that students have the opportunity to demonstrate learning outcomes of the award. (4) Design, approval, monitoring and review of programmes enables standards to be set and maintained and allows students to demonstrate learning outcomes of the award. (5) Subject benchmark statements and qualification statements are used effectively in programme design, approval, delivery and review to inform standards of awards.

ExpectationsFactors ( this term used in IRENI refers to clusters of indicators) (2) Higher education providers make scrupulous use of external examiners. Reference points: Quality Code - Chapter B7: External examining Quality Code - Chapter B10: Management of collaborative arrangements (Indicators , 25-28) Quality Code - Chapter B3: Learning and teaching (Indicators 7-8, 9-12) Defining the role of external examiner The nomination and appointment of external examiners Carrying out the role of external examiner Recognition of the work of external examiners External examiners' reports Serious concerns

ExpectationsFactors (for further explanation see the reference points) (1) Higher education providers produce information for their intended audiences about the learning opportunities they offer that is fit for purpose, accessible and trustworthy. Reference points: Quality Code - Part C: Information about higher education provision HEFCE 2011/18: Table 1 and Table 2 HEFCE 2012/04 Circular There are effective institutional mechanisms for making sure that the following information is fit for purpose, accessible and trustworthy: - Information for the public about the higher education provider - Information for prospective students - Information for current students - Information for students on completion of their studies - Information for those with responsibility for academic standards and quality. The information detailed in HEFCE 2011/18, and in particular the Key Information Set (KIS) and the wider information set (WIS), is up-to-date, and accessible to the institution’s stakeholders. External examiners' reports are shared as a matter of course with the institution's student r epresentatives, for example through staff-student committees.

More equal than others? Cornerstone of confidence in management of current and likely future of academic standards difficult for a team to reach a judgment that academic standards meet UK requirements if there are doubts about the scrupulous use of external examiners

Typical issues arising in reviews Areas requiring improvementPositive features EE comments are “filtered out” as they move upwards through the reporting system Comments made by EEs have been fed directly into enhancement initiatives (comments on quality/timeliness of feedback to students on assessed work lead to an improvement in turn around times & depth of feedback) EEs are not provided with data to enable them to comment on comparability of standards or not provided with scripts on time EEs are provided with a full response to the issues they raised plus a generic response relating to the module/programme EE comments seeking improvements are not responded to year on year New EEs are mentored by existing EEs and are provided with EE reports from previous year as background information EE are not given opportunity to comment on comparative performance on modules undertaking by students in different locations EEs are invited to and attend regular updates or briefings held by the home institution

The future of reviews Higher Education Review Initial Appraisal The role of external examiners in shaping the review