1. Summary Input – process – output model

2. Input Written Oral Observational

3. Written University related Course related Subject related

4. University related Regulations and codes of practice Policy documents eg on disabled students Examiner training – by institution

5. Course related Curriculum Articulation of course learning outcomes- taught curriculum and assessments eg course handbooks Curriculum overview documents including taught curriculum, marking schemes etc

6. Course related Assessment Statement of purpose Assessment regulations, timetable, dates of examiners meetings Description of assessment methods and ideally rationale Standard setting procedure Exam blue print Procedures for failing students

7. Course related Assessment evidence Question papers OSCE station scenarios Portfolios/SSCs ??performance recordings Assessment results Results spread sheets Examination profile Statistical information eg reliability etc

8. Course related Previous examination board notes Appeals and mitigating circumstances Previous external examiners reports and Schools responses/actions

9. Subject related Reports by accrediting/professional bodies eg GMC QABME reports Internal university periodic reviews Internal course review documents eg QM&E reports

10. Observational/oral information Discussion with other externals Observation of cultures/behaviours Meetings with students Meetings with staff

11. Process Are course outcomes achievable? Is course/assessment fit for purpose? Are procedures appropriate for measuring student achievement? Are the qualifications awarded comparable to other HE institutions? Is the process fair and equitable? Does it make appropriate provision for special needs Is the sample truly reflective of the cohort?

12. General activities Early involvement of externals in course and course assessment is desirable. Identify standards Discussions + email/telephone conversations Observations of assessment practice and process Observations of standard setting practice and process Reviews of data and analysis Marking and re marking scripts

13. Example of typical process for Medicine Review set exam papers – is the level appropriate? does the marking schedule relate to the questions asked? Review marking schedule Observe sample of OSCEs and practicals Review exam scripts (sample) – were questions comprehended? Quality of answers? Is level appropriate? Relate marking schedule to scripts Review and discussion with other external examiners Review range of marks

14. External examiners need to be …. Detached Critical friend Provide advice Provide arbitration Be an advocate? Able to make (negative) judgments

15. Advice Formal = written reports Oral reports at exam board Informal oral feedback to module/exam convenor

16. Advice may cover Curricular programme System of assessment Appropriateness of methods used Remedial actions and processes – supported by references/evidence Shared examples of good practice elsewhere Internal moderation – possibility of collusion Standard setting process

17. The external examiners report An ideal report: Concise Constructive Honest Aligns to the questions the institution needs to ask + also questions the external needs to ask? Strengths and weaknesses Commenting about process Commentary on action Typed and timely Provides relevant recommendations