1. How are question papers created and assessed?
Nov. 2016
Devon EO Conference
November 2016
How are question papers created and assessed?

2. The 1st draft question paper and mark scheme are created by the Principal Examiner around 18 months prior to the paper going live. There are a number of influences on this…
Specification content
SAMs
The first draft
Examiner’s reports
Assessment objectives
Past candidate performance
Feedback from examining team
Centre feedback

3. Question Paper Evaluation Committee (QPEC)
This 1st draft is initially reviewed at a QPEC meeting. This is around 12 months prior the paper it going live
Chief Examiner
Subject Officer
Principal Examiners
Question Paper Evaluation Committee (QPEC)
Chair of Examiners
Subject Experts
Scrutineer

4. Further drafts are circulated, checked and proof read as necessary until the final, fully checked & proof read paper is ready for printing
A copy of this printed version is also scrupulously checked prior to the full print run being approved
Papers are sent to centres and candidates sit them
EOs return the papers to the Awarding Bodies as instructed, and these are sent out to marking teams either as hard copies or as electronic scans

5. The marking teams
Chief Examiner (usually also a Principal Examiner for one of the papers)
Principal Examiner
Principal Examiner
Principal Examiner
Team Leader
Team Leader
Team Leader
Team Leader
Team Leader
Team Leader AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE AE
AE = Assistant Examiner

6. So how do we ensure that marking is as accurate and consistent as it can be?
This is done slightly differently by the various awarding organisations, but the underlying principles apply across all of them:
Senior Examining Team meets very soon after the live paper has been attempted. They scrutinise live candidate responses in order to fine-tune the mark scheme
Senior Examining Team meets with Team Leaders to explain and train them in the application of the mark scheme

7. So how do we ensure that marking is as accurate and consistent as it can be? (cont.)
Team Leaders communicate with their Assistant Examiners (either in face-to-face or on-line) to explain this mark scheme, and a further standardising process takes place. At WJEC Eduqas we do this face-to-face and call it the Examiners’ Conference
Assistant Examiners send sample scripts to their Team Leaders; Team Leaders to Principals; Principals to the Chief at key points throughout the marking process in order that further checks can be carried out. (This can be achieved through random checking if on-screen marking is employed)

8. After the Marking Process
It is now essential to ensure that fair and accurate adjustments can be made where Assistant Examiners may be too generous or too lenient in applying the mark scheme
In traditional, paper marking, well-tested computer adjustments can adequately deal with any consistent straying from the standard
For example:
Too generous/lenient across whole paper/question(s)
Too generous/lenient at top end/bottom end
Essentially, anything that fits a pattern
These adjustments are known as scaling

9. When the marking has finished
The candidates’ marks are fed into the computer as the Examiners upload them throughout the process
A distribution curve of raw marks is then created

10. Usually Present at the Award
Chief Examiner
Principal Examiners
Usually Present at the Award
Statistician/analyst
Subject Officer
Chair of Examiners
This is a meeting (usually in July) where the key grade boundaries per paper (and therefore for the subject as a whole) are set

11. The award is approved
Grades are applied to each candidate
Provisional results are sent out to centres

12. That’s all there is to it!

13. Many thanks for listening
Dave Jones
Regional Representative
Eastern, Central & South Western England
@DavidWJECJones