How are question papers created and assessed? Nov. 2016 Devon EO Conference November 2016 How are question papers created and assessed?

The 1st draft question paper and mark scheme are created by the Principal Examiner around 18 months prior to the paper going live. There are a number of influences on this… Specification content SAMs The first draft Examiner’s reports Assessment objectives Past candidate performance Feedback from examining team Centre feedback

Question Paper Evaluation Committee (QPEC) This 1st draft is initially reviewed at a QPEC meeting. This is around 12 months prior the paper it going live Chief Examiner Subject Officer Principal Examiners Question Paper Evaluation Committee (QPEC) Chair of Examiners Subject Experts Scrutineer

Further drafts are circulated, checked and proof read as necessary until the final, fully checked & proof read paper is ready for printing A copy of this printed version is also scrupulously checked prior to the full print run being approved Papers are sent to centres and candidates sit them EOs return the papers to the Awarding Bodies as instructed, and these are sent out to marking teams either as hard copies or as electronic scans

The marking teams Chief Examiner (usually also a Principal Examiner for one of the papers) Principal Examiner Principal Examiner Principal Examiner Team Leader Team Leader Team Leader Team Leader Team Leader Team Leader AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE = Assistant Examiner

So how do we ensure that marking is as accurate and consistent as it can be? This is done slightly differently by the various awarding organisations, but the underlying principles apply across all of them: Senior Examining Team meets very soon after the live paper has been attempted. They scrutinise live candidate responses in order to fine-tune the mark scheme Senior Examining Team meets with Team Leaders to explain and train them in the application of the mark scheme

So how do we ensure that marking is as accurate and consistent as it can be? (cont.) Team Leaders communicate with their Assistant Examiners (either in face-to-face or on-line) to explain this mark scheme, and a further standardising process takes place. At WJEC Eduqas we do this face-to-face and call it the Examiners’ Conference Assistant Examiners send sample scripts to their Team Leaders; Team Leaders to Principals; Principals to the Chief at key points throughout the marking process in order that further checks can be carried out. (This can be achieved through random checking if on-screen marking is employed)

After the Marking Process It is now essential to ensure that fair and accurate adjustments can be made where Assistant Examiners may be too generous or too lenient in applying the mark scheme In traditional, paper marking, well-tested computer adjustments can adequately deal with any consistent straying from the standard For example: Too generous/lenient across whole paper/question(s) Too generous/lenient at top end/bottom end Essentially, anything that fits a pattern These adjustments are known as scaling

When the marking has finished The candidates’ marks are fed into the computer as the Examiners upload them throughout the process A distribution curve of raw marks is then created

Usually Present at the Award Chief Examiner Principal Examiners Usually Present at the Award Statistician/analyst Subject Officer Chair of Examiners This is a meeting (usually in July) where the key grade boundaries per paper (and therefore for the subject as a whole) are set

The award is approved Grades are applied to each candidate Provisional results are sent out to centres

That’s all there is to it!

Many thanks for listening Dave Jones Regional Representative Eastern, Central & South Western England davidr.jones@eduqas.co.uk @DavidWJECJones