1. The Power of e-Assessment Robert Coe Qualifications Wales, Cardiff, 14 March 2016

2. ∂  Language and number for children at the start of school –PIPS Baseline: 3m pupils since 1993 –BASE: new in 2015: 100,000 pupils  Stopping rule –Questions are progressively more difficult. Stop when the child gets three wrong in a row 2 An adaptive assessment

3. ∂ 3

4. ∂ Diagnostic assessment, eg reading  Language comprehension –Background knowledge –Literacy knowledge –Vocabulary –Language structure –Verbal reasoning  Word reading –Phonological awareness –Decoding (phonics) –Word (sight) recognition 4

5. ∂ E-Assessment Pros  Automatic marking  Instant results & feedback  Engaging for pupils  Smarter assessment –Adaptive –Dynamic –Rich environments –Simulations/games –Integration with learning  Cheap to use Cons  Bandwidth  Availability of devices  Different platforms may not be comparable  Vulnerable to technical IT failure  Security problematic for high-stakes  Expensive to develop 5

6. ∂ Fully adaptive test 6 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

7. ∂ Fully adaptive test 7 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

8. ∂ Fully adaptive test 8 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

9. ∂ Fully adaptive test 9 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

10. ∂ Fully adaptive test 10 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

11. ∂ Fully adaptive test 11 ‘Ability’ scale ‘Difficulty’ scaleHardEasy HighLow Item bank Ability estimate

12. ∂ Convergence and Test-Retest

13. ∂ Adaptive assessment Pros  Efficient: same info in <50% time  Accurate across the range, esp extremes  Better experience: challenge matched to ability  Neighbours see different items (can’t copy) Cons  Expensive to develop –Large item banks –Extensive pre-testing  Limited to auto-marked items  Cannot review / go back  Problematic with time limits 13

14. ∂ Adaptive assessment issues  Calibration of items  Item selection algorithm  Termination rules  Item exposure  Content balance  Correlated sections 14