External Examiners’ Briefing Day 22 February 2016

Brunel University London Presenters Professor Mariann Rand-Weaver, Pro Vice-Chancellor for Quality Assurance and Enhancement Dr Amanda Harvey, Associate Dean (Quality Assurance), College of Health and Life Sciences Mr Ruaidhri Donnelly, Head of Quality & Standards Assurance Plus: Dr Bryan Mase and Dr Jane Lawrie, Associate Deans (Quality Assurance) External Examiners’ briefing

Brunel University London Briefing outline Overview of External Examiner role External scrutiny and moderation of assessments Panels and Boards of Examiners Programme and Award Regulations External Examiner annual reports External Examiners’ briefing

Welcome

Brunel University London Brunel academic structure Three large Colleges, each led by a Dean, supported by Vice-Deans Education and Associate Deans Quality Assurance Business, Arts and Social Sciences (CBASS) Engineering, Design and Physical Sciences (CEDPS) Health and Life Sciences (CHLS) Larger Departments have ‘Divisions’ for different subject area – your appointment as an External Examiner is likely to cover a large slice of a particular Division. External Examiners’ briefing

Brunel University London Current activities and developments QAA Review taking place March 2016 New Education strategy and International strategy Major Portfolio review now concluding External Examiners’ briefing

Brunel University London Input from External Examiners Brunel greatly values your work and considers and acts on your reports We also welcome any feedback about today’s event External Examiners’ briefing

Role overview

Brunel University London Appointment Senate Regulation 4 governs the appointment of and role of External ExaminersSenate Regulation 4 At least 1 External Examiner per award bearing Programme Appointment normally for 4 years but may be extended by one further year External Examiners nominated by Heads of Department and appointed by Senate External Examiners’ briefing

Brunel University London Main functions External scrutiny of assessment-setting External moderation of summative assessments Involvement in Panels and Boards of Examiners which confirm grades and awards Other meetings with staff/students as agreed (e.g. project presentations) Annual Report (to the VC) on academic standards, assessment processes, student performance, and comparability of overall standards with those of similar programmes in other UK HEIs External Examiners’ briefing

Brunel University London Brunel’s Regulations All published at SR2 – Undergraduate programmes/awards (Levels 1, 2, 3) SR3 – Postgraduate Taught programmes/awards (Level 5) Note that “levels” are not the same numbers as FHEQ. SR4 – Assessment, Boards & Panels of Examiners External Examiners’ briefing

Scrutiny and external moderation

Brunel University London Scrutiny of summative assessment tasks All summative assessments should be scrutinised by a subgroup designated by the appropriate Panel of Examiners, consulting the External Examiner(s), before final approval by the Panel. [SR4.4] External Examiners' views should be sought on all examination papers and other assessments (e.g. proposed coursework titles) at Level 2, 3 and Masters Level, before the papers are printed or other assessments notified to students. Assessment criteria, marking schemes, and/or indicative answers should be provided (also see University Grade Descriptors) Consider whether the questions: a) are expressed clearly; b) adequately assess the intended learning outcomes; c) are in accordance with the overall assessment strategy for the programme; d) are of an appropriate standard; e) are associated with appropriate assessment criteria. Changes proposed by the External Examiner under should only be rejected with good reason after discussion with him/her. External Examiners’ briefing

Brunel University London External moderation of summative assessments External Examiners will be provided with [SR4.85]: Samples of assessed work (covering full range of student achievement; at least 10% of cohort, or 20% for UG projects / PGT dissertations; all work where class size <10) Full set of provisional grades All internal moderation or double-marking records (indicating any adjustments already made) Don’t need to study each piece of work in detail; but consider [SR4.86]: Is the work of appropriate standard and comparable with other UK HEIs? Do grades/marks assigned reflect the marking criteria, published grade descriptors and intended learning outcomes? Findings feed into Panel of Examiners consideration/confirmation External Examiners’ briefing

Brunel University London External moderation arrangements Samples of student work should be: Sent prior to the Panel of Examiners (discuss how with the programme team – e.g. electronically via Blackboard) Or, by agreement, available on the morning of the Panel meeting You should also be invited to any ‘live’ assessment events, displays, performances etc; or given opportunity to see video recordings (discuss with the programme team) You may be asked to moderate placement assessments All assessed material should be available if you want to see it, either on the day of Panel of Examiners or on request. External Examiners’ briefing

Panels of Examiners

Brunel University London Panels and Boards of Examiners – overall role Common purpose is to ensure: Fairness to all students and each student Rigour of assessment Robustness of the results and decisions Maintenance of academic standards Different types of decision-making: Panels are where collective academic judgement is exercised Boards confirm that processes have been appropriately carried out, and make decisions on things like reassessment – but extraordinary academic judgements may also be needed Chairing Boards of Examiners and Panels of Examiners 17

Brunel University London Panels of Examiners (PoE) – purpose Panels of Examiners are where collective academic judgement is employed and EEs have a vital role to play Key role: to verify and confirm the marks/grades for each module/block Do not look at individual students (unless error or additional info presented) May make specific recommendations to Boards concerning particular assessments See Senate Regulations 4.52 to 4.56 External Examiners’ briefing

Brunel University London Issues for Panels of Examiners Potential assessment design errors Academic judgment Moderation effectiveness and issues Extraordinary factors/events Re-marking/adjustment of results (see next slide) Justification of actions … and ensure recording of decisions at all stages! External Examiners’ briefing

Brunel University London If a Panel identifies a problem If the Panel is not satisfied with the fairness or integrity of assessments/marking, it should not confirm grades/marks Appropriate action to take is normally for the assessors to reconsider grade/marks for all affected students Only in very exceptional circumstances may the Panel scale grades/marks for a particular assessment; if so must then record justification and rationale Grades/marks may not be adjusted for individual students (unless wrongly recorded, or additional information is presented) [SR4.55] External Examiners’ briefing

Brunel University London Role of External Examiners at Panels Full membership of the Panel of Examiners Contribute to the collective academic decision-making Have a distinct influence on the PoE (through their independence and their overview) EEs are not empowered to change marks or grades – all decisions are made collectively May ask to inspect any documents involved in the assessment process [See SR4.93 – 4.99] External Examiners’ briefing

Boards of Examiners

Brunel University London Boards of Examiners (BoE) – purpose To receive confirmed marks/grades from Panels of Examiners To consider individual student profile of achievement To take into account mitigating circumstances and determine appropriate actions To decide on progression and re-assessment To recommend appropriate awards for each completed/withdrawn student Implement Senate Regulations and NB Boards now have few powers of discretion within the regulations – but all decisions must still be justified External Examiners’ briefing

Brunel University London Role of External Examiners at Boards Full membership of the BoE Contribute to the collective academic decision-making General responsibility for all areas covered, not just your own area of specialist focus within this Have a distinct influence on the BoE (through independence and overview) EEs are not empowered to change marks/grades or progression/classification decisions unilaterally; all decisions are made collectively Assure themselves that due process has been followed and appropriate consideration and decisions have been made by the BoE See Senate Regulations SR4.93 – 4.99 External Examiners’ briefing

Brunel University London Vice-Chancellor’s Representative (VCR) Award Boards must have a VCR present in order to make awards VCR is an experienced member of University professional staff with a good understanding of the Regulations VCR has authority to stop the Board should any irregularities come to light, and reports back to the VC on any issues VCR (and Assistant VCR, where present) take formal record of BoE decisions – you sign this! – for release to students External Examiners’ briefing

Brunel University London Issues for Boards Mitigating circumstances Progression and re-assessment Awards Extraordinary decisions – discretion Justification of decisions … and recording of decisions at all stages! External Examiners’ briefing

Brunel University London Board discretion Limited scope: Action re. Mitigating Circumstances Re-assessment timing Extraordinary situations External Examiners’ briefing

Brunel University London Mitigating Circumstances (MC Panels) “A serious or significant unforeseen event, and its consequences, which have significantly impaired the academic performance of a student in one or more assessed activities” Role of MC Panels (which don’t include EEs) is to: Accept or reject MC submissions from students (considering likely impact on studies/assessment, but with no access to academic results/profile in considering cases) Make decisions directly re coursework submission deadlines with MCs Report to Board of Examiners on all other accepted MCs (identifying serious cases) See Senate Regulations 4.26 – 4.31, plus further Guidance for Mitigating Circumstances Panels and Boards of ExaminersGuidance for Mitigating Circumstances Panels and Boards of Examiners Chairing Boards of Examiners and Panels of Examiners 28

Brunel University London BoE actions re. Mitigating Circumstances Board paperwork will show where a student’s results have been affected by MCs (as accepted by MCs Panel).  Only Chair and Externals should have sight of actual personal details (which may inform decisions on reassessment timing etc.) Actions should minimise disadvantage to students while preserving the integrity of standards and awards.  Normal BoE action should be to permit (uncapped) reassessment  Need to decide reassessment timing (esp. for severe MCs)  Other potential actions: academic decision to assign a grade based on student’s wider performance/profile; or extraordinary action if agreed with Chair of QAC Marks/grades for individual assessment elements (e.g. exam script, coursework essay), as confirmed by PoE, may not be altered. [See SR ] External Examiners’ briefing

Programme and Award Regulations

Brunel University London Programme and Award Regulations (SR2 and SR3) Principles – Transparency of standards of awards Equity of treatment De-modularised Regulations (Assessment Blocks and Study Blocks) Also note existence of ‘old’ Regulations – may still apply to some students who have previously taken abeyances External Examiners’ briefing

Brunel University London Brunel’s 17-point grade scale Grades are the primary performance measure, as used in award calculations (not marks – though mark equivalence is defined) University Grade Descriptors are the primary reference for academic standards External Examiners’ briefing Indicative Mark Band Degree class equivalent Grade Grade Point 90 and above1A* A A A B B B C C C D D D FailE FailE FailE-2 29 and belowFailF1 Non-submissionFailNS0

Brunel University London Grading and marking at point of assessment Departments define whether they are entering marks or grades at element level (by programme) SITS (student records database) is set up with details of assessment elements and weightings  For Grade entry – SITS assigns a grade point for each element and calculates weighted average of grade points.  For Mark entry – SITS finds weighted average of marks and assigns corresponding grade point for assessment block All calculations and requirements for award are then grade-based External Examiners’ briefing

Brunel University London Progression and Award rules Defined minimum grade profile requirements for progression and threshold awards “Core” assessments can be defined (must be passed at threshold level) Classification determined by simple GPA requirement “Borderline” mechanism is strictly deterministic (no BoE discretion) No compensation/condonation mechanisms other than strict borderline mechanism Standard reassessment rules across University; reassessment volume right = limit External Examiners’ briefing

Brunel University London Core assessments or assessment blocks ‘Core’ indicates an assessment block, or element within this, which must be passed with a grade of at least D- (for undergrads) or C- (for postgrads) in order for the student to progress or be awarded Core status for specific blocks/assessments must be defined in the programme specification All Masters dissertations and all UG Final Year Projects must be designated as core External Examiners’ briefing

Brunel University London Undergraduate Sandwich Placements Sandwich students who have passed the placement assessment (minimum grade D-) are eligible for a different award title – normally a “with professional development” suffix This award is calculated slightly differently:  The placement module grade is included in the Level 2 grade profile for award.  Percentage contribution of the placement module to Level 2 is defined in the Programme Specification. All students who have failed the placement module are eligible for the standard (non-sandwich) award Students who did badly in the placement and would get a better classification under the standard award should be offered the choice between them by the Board [See SR2.42 to 2.44] External Examiners’ briefing

Brunel University London UG Final Awards – Honours Bachelors Set out in SR2 Appendices. Based on Level 2 and Level 3 Weighted Grade Profile (1:2 weighting of credit between L2:L3) Requirements for Other UG Awards (Integrated Masters, Ordinary, DipHE, CertHE) are also set out in SR2 Appendices External Examiners’ briefing Class Maximum credit volume of Grade F Maximum credit volume of assessment blocks containing core assessments below D- Maximum (non-weighted) credit volume of non-core Grade Band E (E+, E, E-) Minimum weighted GPA* Level 2Level 3Level st rd NA

Brunel University London PGT Final Awards – Master’s Set out in SR3 Appendices. Requirements for PGDip and PGCert Awards are also set out in SR3 Appendices If dissertation can be included in the profile for a PGDip award, this must be set out in programme specification External Examiners’ briefing Class Maximum credit volume of Grades E+, E, E-, F Maximum credit volume of assessment blocks containing core assessments below C- Maximum (non- weighted) credit volume of non-core Grade Band D (D+, D, D-) Minimum Taught Part Weighted GPA* Minimum Dissertation Grade Distinction A- Merit B- Pass0030NAC-

Brunel University London UG Final Awards – borderline mechanism (PGT very similar) Set out in SR2 (and SR3 for PGT) Appendices External Examiners’ briefing Volume of grades (weighted by assessment block credit value and by Level weighting) in Class or better Class 1 st Minimum Weighted GPA At least 41% of grades (weighted) in Class or better At least 45% of grades (weighted) in Class or better At least 50% of grades (weighted) in Class or better At least 54% of grades (weighted) in Class or better At least 58% of grades (weighted) in Class or better NA At least 62% of grades (weighted) in Class or better11.0NA At least 66% of grades (weighted) in Class or better10.5NA

Brunel University London Entitlements to reassessment (UG) Defined rights to, and limits of, reassessment. Not at discretion of Boards of Examiners (unless MCs) Entitlement to reassessment where student fails to achieve D- (Level 1, 2, 3) or C- (Level 5) in a block or a core assessment (or fails to achieve a pass in a pass/fail assessment) Reassessment volume limit (for each Level) is maximum 60 credits at Level 1, or 40 credits at Level 2, 3 or 5 Grades for reassessments are capped at D- (Levels 1-3) or C- (Level 5) External Examiners’ briefing

Brunel University London Entitlements to reassessment (PGT) Same general principles as for UG Reassessment grades capped at C- (threshold grade) Entitlement to reassessment in taught part assessments, up to a maximum of 60 credits, where student fails to achieve C- in a block or a core assessment (or fails to achieve a pass in pass/fail assessments) Students may submit a dissertation even if they cannot achieve a Master’s degree by doing so For dissertation – may revise and resubmit for first- attempt fails between E- and D+; but no resubmission permitted for F-grade dissertation fails or non- submissions External Examiners’ briefing

External Examiner Annual Reports

Brunel University London Report submission All External Examiners submit a report annually to the Vice-Chancellor – by 31st July each year Request for reports and reminders sent out by the Quality and Standards Office Pro-forma/web form provided for the report You will be sent a formal response detailing how the University is following up if you raise any concerns External Examiners may also write in confidence to the Vice-Chancellor at any time during their period of appointment about any matter relating to their duties External Examiners’ briefing

Brunel University London Response and publication College required to respond to the comments made in the External Examiners Report both directly to the External Examiner and in their annual monitoring reports University will provide a central response to broader issues, particularly regarding regulations External Examiner reports and University responses published to current students and staff. Web page also available to External Examiners. Fees paid once EE reports are received External Examiners’ briefing

Your questions?

Brunel University London Further support resources Guidance and resources about all aspects of these processes are available at and The formal Regulations are published at External Examiners’ briefing

Thank you Please fill out one of the feedback sheets about today’s event