1. Interpreting Feedback from Baseline Tests – Predictive Data Course: CEM Information Systems for Beginners and New Users Day 1 Session 3 Wednesday 17 th October 2012 Peter Hendry: CEM Consultant Peter.Hendry@cem.dur.ac.uk

2. The word ‘PREDICTION’: Quite probably the most contentious term that is used!! Concerns include: A prediction for GCSE at start of year 7? What has the baseline test got to do with my subject? I know my pupils! The predictions are too low: not valid!!! And what about my professional judgement?

3. BASELINE SCORE GRADE * ** *** ******************* * ** *** ******************************** * ** *** ********************************* *** ** * * ** *** ************************************ *** ** * * ** *** ******************************** **** ** * * ** ***************************** *** ** * * ** *** ******************* ** * * * ** *********** ** * How is a ‘prediction’ generated? C A* A B 50 100 150

4. 3 key points are: The higher the baseline score the higher the final grade Any one grade is achievable from a range of baseline scores From any baseline score, a range of grades are possible

5. BASELINE SCORE GRADE * ** *** ******************* * ** *** ******************************** * ** *** ********************************* *** ** * * ** *** ************************************ *** ** * * ** *** ******************************** **** ** * * ** ***************************** *** ** * * ** *** ******************* ** * * * ** *********** ** * Subject National trend line (regression line) How is a ‘prediction’ generated? 50% on or above the trend line 50% on or below the trend line ‘PREDICTION’ (expected grade) C A* A B

7. ‘Predictions’…...are based on Average performance by similar pupils in past examinations The problem with the word ‘prediction’ is…? An alternative is ‘expected’ grade

8. Predictions 4 points = D 6.6 points = A/B Trend line

9. Comments? The graph below shows the middle 2/3 of some subject trend lines

10. Some Subjects are More Equal than Others…. E D C B A C Grade BAA* Average GCSE Physics Maths Psychology Sociology Latin Photography English Lit >1 grade

11. FACTORS THAT WILL INFLUENCE RELIABILITY OF PREDICTIONS: Knowledge of student Parental support/home life Peer influences/social life Student attitude, interest, language Expectations of staff Department/institution ethos Resources Quality of teaching and learning: pace of lessons Understanding how children learn……… And the reliability of the predictions......

12. 12 Correlation = 1 Result

13. Correlation = 0 Correlation = 0.7

14. Point and grade ‘predictions’ to GCSE Compare the predictions for English and Mathematics. What pattern do you notice? Look at Art and Design, Biology and French. Comments?

15. Comments? The graph below shows the middle 2/3 of some subject trend lines

16. Prediction/expected grade: 5.1 grade C Most likely grade

17. to place school at 75 th percentile of VAD

19. Or insert own values and click adjust

22. Prediction/expected grade: 6.4 grade A/B Most likely grade

24. Prediction/expected grade: 5.9 grade C Most likely grade Independent Sector

26. Not a label for life...just another piece of information The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score. In working with students these graphs are more useful than a single predicted or target grade Chances graphs show what can be achieved: –By students of similar ability –By students with lower baseline scores

27. Yellis predictive data: baseline score 103 (55%)

29. Student 1

30. Student 2

31. Why is the ‘predicted’ grade not always equal to the highest bar ? Most likely grade Predicted (‘expected’) grade AT WHICH POINT WILL THE SEE- SAW BE BALANCED? i.e. the lower grades ‘pull’ the prediction to the left

32. Student 3

33. Student 4

34. Student 4 - IPR

36. Basing Targets on Prior VA – One Methodology from an Alis School Discuss previous value added data with each HoD Start with an agreed REALISTIC representative figure based, if available on previous (3 years ideally) of value added data add to each pupil prediction, and convert to grade (i.e. in-built value added) Discuss with students, using professional judgment and the chances graphs, adjust target grade calculate the department’s target grades from the addition of individual pupil’s targets

38. Key Questions for Target Setting What type of valid and reliable predictive data should be used to set the targets? Should students be involved as part of the process (ownership, empowerment etc.)? Should parents be informed of the process and outcome?

39. Key points to consider might include: Where has the data come from? What (reliable and relevant) data should we use? Enabling colleagues to trust the data: Training (staff) Communication with parents and students Challenging, NOT Demoralising, students……. Storage and retrieval of data Consistency of understanding what the data means and does not mean