1. 1 Perspectives on the Future of Assessment in England and Internationally Robert Coe CEM conference, 25th January 2012

2. 2  Background to CEM  Current context of assessment  Key questions about the future of assessment: o Should we be using standardised tests? o How can classroom assessment support learning? o Can assessment data identify good teachers? o How can monitoring and feedback of performance support improvement? Outline

3. 3 To help educators improve learning and other educational outcomes, through o Assessments that support learning o Monitoring and feedback systems for self- evaluation o Rigorous evaluation of the impact of different approaches o Promotion of evidence-based practices and policies CEM’s Aims

4. 4 CEM’s Achievements  Providing monitoring systems for schools for almost 30 years – we led the world from the North East of England  CEM assessments are used by o 1.1 million students each year o More than 50% of UK secondary schools o Schools in over 40 countries  Largest provider of computerised adaptive tests outside US  The largest educational research unit in a UK university

5. 5 Current context of assessment

6. 6

7. 7  Existing qualifications in England are the legacy of an out-dated, amateurish view of assessment 3 / 10 Could do better  Good: high-stakes assessments are based on what has been studied  Bad: examinations often trivialise the range of skills, knowledge and understanding that have been (should be?) taught, are poorly conceived and constructed and validity is an afterthought (at best). GCSE & A level …

8. 8 Low level skills are easier to assess Bloom’s Taxonomy –Knowledge –Comprehension –Application –Analysis –Synthesis –Evaluation SOLO Taxonomy –Pre-structural –Uni-structural –Multi-structural –Relational –Extended abstract

9. 9  NC is just a part of curriculum  Broaden at KS4  2-year Key Stages  Clarify relationship between PoS & assessment  Ensure all students are ‘ready to progress’  ‘High expectations for all’  Detailed profiles, not general levels  No change to GCSE NC Review

10. 10 “Researchers at Durham University have been particularly good at challenging the growth in grade performance. One piece of analysis from Durham concluded that between 1996 and 2007, the average grade achieved by GCSE candidates of the same ‘general ability’ rose by almost two thirds of a grade. And the rise, they argued, is particularly striking in some subjects: in 2007, pupils received a full grade higher in maths, and almost a grade higher in history and French, than pupils of the same ability when they sat the exams in 1996. Similar trends have been found at A level. Academics at Durham found that in 2007, A level candidates received results that were over two grades higher than pupils of comparable ability in 1988. And pupils who would have received a U in Maths A-Level – that’s a fail – in 1988 received a B or C in 2007.” Michael Gove on A level standards Ofqual Standards Summit, 13 Oct 2011

11. 11 Rising standards

12. 12 Grade slippage at A level

13. 13 25 point rise in PISA = +£4,000,000,000,000 GDP International surveys

14. 14 A fair UCAS points tariff?

15. 15 Key questions about the future of assessment

16. 16 Should we be using standardised tests? Cons Test only a limited part of what can be learnt Methods (eg multiple choice) are limited and constraining Focus on short, closed tasks Emphasise (fixed) ability Pros Well designed tests can cover the full range of content and methods Standardisation gives valuable reference point for performance Teacher-created assessment is generally expensive, hard to standardise, unreliable and biased

17. 17  1998: Black & Wiliam’s review – strong evidence of power of FA  Support from governments to implement  Are teachers actually doing it?  Do we know what ‘it’ is?  Have there been improvements in learning?  How do you get a teacher is not currently doing it faithfully to do so? Formative Assessment (AfL)

18. 18 What happens if you get a good teacher for several years? Tymms et al (2009)

19. 19  Having a good teacher (+1SD in VA, ie top 16%) in a single year o Raises test scores that year by 0.1 SD; about 1/3 of the gain is sustained o Raises earnings by about 1% at age 28 o Is worth paying $4,600 per child to retain  Replacing a very poor (bottom 5%) teacher with an average teacher is worth $267k to each class they teach Long-term effects of a good primary teacher Chetty et al (2011)

20. 20 www.suttontrust.com

21. 21 Overview of value for money Cost per pupil Effect Size (months gain) £0 0 10 £1000 Feedback Meta-cognitive Peer tutoring Pre-school 1-1 tutoring Homework ICT AfL Parental involvement Sports Summer schools After school Individualised learning Learning styles Arts Performance pay Teaching assistants Smaller classes Ability grouping Promising May be worth it Not worth it

22. 22 Is that it? Have we solved the problem of how to improve attainment?

23. 23  These strategies have been shown to be cost- effective in research studies  But when we have tried to implement evidence- based strategies we have not seen system-wide improvement  We don’t know how to get schools/teachers who are not currently doing them to do so in ways that are o True to the key principles o Feasible in real classrooms – with all their constraints o Scalable & replicable o Sustainable Implementation

24. 24 To help educators improve learning and other educational outcomes, through o Assessments that support learning o Monitoring and feedback systems for self- evaluation o Rigorous evaluation of the impact of different approaches o Promotion of evidence-based practices and policies CEM’s Aims