1. Introduction to CEM Secondary Pre-16 Information Systems Neil Defty Secondary Systems Programme Manager

2. Ensuring Fairness

3. Principles of Fair Analysis : 1.Use an Appropriate Baseline 2.Compare ‘Like’ with ‘Like’ 3.Reflect Statistical Uncertainty

4. CEM Systems Principle of Fair Analysis No1 : Use an Appropriate Baseline

5. The Assessments MidYIS - Year 7, 8 or 9 Developed ability assessments: Vocabulary Maths Non-verbal (Cross-sections, Block Counting and Pictures) Skills (Proof Reading and Perceptual Speed & Accuracy) SOSCA (“INSIGHT Core”) – Year 9 Curriculum-based assessments: Reading (Passage Comprehension, Text Comprehension and Speed Reading) Maths (Number & Algebra, Handling Data and Space, Shapes & Measures) Science (Biology, Chemistry and Physics) INSIGHT – Year 8* or 9 ( * depending on numbers) Developed ability assessments (‘MidYIS’) plus Curriculum-based assessments (‘SOSCA’) Yellis - Year 10 or 11 Developed ability assessments: Vocabulary Maths Patterns (Matching and Reflections & Rotations) 3 sessions - Computer adaptive 4 sessions - Computer adaptive 1 session – Computer adaptive or paper-based 1 session - Computer adaptive or paper-based

6. Computer AdaptivePaper-based Number of Students per Test Session Limited by number of computers available – multiple testing sessions Can test all students in a single session (in hall or in form groups) or in more than the one session Cost Roughly 30% cheaper than paper-based test Standard cost Processing of Baseline Feedback Baseline feedback available within a couple of hours of testing Takes around 2-4 weeks for papers to be marked Preparation Must be able to install the software or access the internet version of the test No pre-test set up Student Experience “Tailored” assessmentAll students see all questions, irrespective of suitability Computer-adaptive vs Paper-based testing

7. The Analysis

8. . 50100150 -ve VA +ve VA Regression Line (…Trend Line, Line of Best Fit) Outcome = gradient x baseline + intercept Correlation Coefficient (~ 0.7) Residuals Subject X Linear Least Squares Regression

9. Some Subjects are More Equal than Others…. Principle of Fair Analysis No2 : Compare ‘Like’ with ‘Like’

10. The Assessments

11. Maths

12. Vocabulary

13. Non-verbal

14. Skills

15. Maths

16. Science

17. Reading

18. Baseline Assessment and Predictive Feedback

19. Individual Pupil Recordsheets (IPRs)

20. Band Profile Graphs

21. Predictions

22. English Language - band D 2 10 23 31 24 9 1 00 0 10 20 30 40 50 UGFEDCBAA* Grade Percent English Language - band C 1 2 6 20 35 29 7 1 0 0 10 20 30 40 50 UGFEDCBAA* Grade Percent English Language - band B 00 2 7 24 40 21 5 0 0 10 20 30 40 50 UGFEDCBAA* Grade Percent English Language - band A 000 1 8 26 35 23 7 0 10 20 30 40 50 UGFEDCBAA* Grade Percent Chances Graphs

23. Value Added Feedback

24. Burning Question : What is my Value-Added Score ? Better Question : Is it Important ? Principle of Fair Analysis No3 : Reflect Statistical Uncertainty

25. The Scatter Plot Baseline Score Grade Points Equivalent Look for Patterns… General under- or over-achievement ? Do any groups of students stand out ? – high ability vs low ability ? – male vs female ?

26. Value Added Feedback Year 7 Pupil Level Residuals to GCSE

27. Standardised Residuals Graph

28. Statistical Process Control (SPC) Chart

29. Attitudinal Surveys

30. Attitudinal Feedback Your data is above the Yellis average Your data is below the Yellis average Your data is about the same as the Yellis average

31. Attitudinal Feedback

32. Secondary Pre-16 Contact Details Tel:0191 334 4255 Email:midyis@cem.dur.ac.uk sosca@cem.dur.ac.uk insight@cem.dur.ac.uk yellis@cem.dur.ac.uk Web:www.cemcentre.org