1. 1 Grading severity In science assessment SCORE Conference 11 Feb 2011Robert Coe

2. 2 2008 Report for SCORE Relative difficulty of examinations in different subjects –http://www.cemcentre.org/attachments/SCORE2008report.pdfhttp://www.cemcentre.org/attachments/SCORE2008report.pdf –http://www.cemcentre.org/attachments/SCORE2008summary.pdfhttp://www.cemcentre.org/attachments/SCORE2008summary.pdf Robert Coe, Jeff Searle, Patrick Barmby, Karen Jones, Steve Higgins

3. 3 Research questions Existing research –How (if at all) can the relative difficulty of different examinations be defined? –What existing studies have presented evidence about relative difficulties of UK examinations in different subjects? –To what extent do the results of existing studies converge? New analysis of GCSE and A-level data for England: –How well do different statistical methods agree? –How consistent are differences over time? –How much do estimates of relative difficulty vary for different subgroups? Integrating existing and new research: –Are STEM subjects generally more difficult than others? –What are the practical implications of any differences in difficulty? –What policy options are there?

4. 4 Results: Different methods

5. 5 Key findings Existing studies, and new analyses using different statistical methods, broadly agree on the relative severity Differences between subjects are highly stable over time and consistent across subgroups Candidates who take science subjects generally achieve lower grades in those subjects than comparable (or the same) candidates do in other subjects

6. 6 Policy options Do nothing –Comparability within subjects (over time and across boards) is more important –Statistical methods are flawed –Universities (should) require specific subjects, not treat them as interchangeable Make grades comparable across subjects –Transparent and fair –Some subjects would be too hard/easy for current entries Adjust the tariff –Fair at the point of use

7. 7 Update: 2010 A level data

8. 8 Questions Is it really true that sciences are more severely graded? Do actual or perceived differences in grading severity put students off taking sciences? Or other ‘hard’ subjects? Do they put schools off offering them? Do universities/employers recognise the scale of the differences? If there is a problem, what should we do about it?