#TrainWithEase

School Performance and Accountability 16th November 2016 Our Speakers Jamie Pembroke – School Data Expert Paul Charman -FFT James Burch – Angel Solutions Sean Harford - Ofsted Nick Capstick – White Horse Federation

Key Performance Measures: an in-depth guide T: @jpembroke W: www.sigplus.co.uk E: info@sigplus.co.uk

data

1: Floor standards

Primary floor measures 65% or more meet expected standard in reading, writing and maths combined, or Pupils make ‘sufficient’ progress in reading (score >= -5), and Pupils make ‘sufficient’ progress in writing (score >= -7), and Pupils make ‘sufficient’ progress in maths (score >= -5)

Secondary floor measure (Progress 8) -0.5

This is what should happen Ditch the attainment component Remove writing TA from progress measures Have single combined progress measure for Reading and Maths This is what the floor standard should be based on

alternative floor standard (combined CVA measure) -4

2: The coasting measure

(below 85% & any progress threshold) Coasting 2014 (below all) 2015 2016 (below 85% & any progress threshold) Attainment L4+ RWM 85% EXS RWM Progress 2LP R 94% VA R -2.5 2LP W 96% 97% VA W -3.5 2LP M 93% VA M * Coasting if below all 4 measures in 2014 and 2015, and below 85% EXS and any one progress threshold in 2016. If below just one threshold in 2016, that threshold must be sig- to be defined as coasting.

Alternatively, remember these? If school plots ‘significantly’ in this zone 3 years running then it’s coasting

3: The use of teacher assessment in high stakes accountability measures

Reading test vs writing TA at KS2 2016

Use of writing results in floor standards In 2016 only, if a school's performance at KS2 has dropped below the floor standard based on performance in writing alone, and in the absence of any other factors, the local authority or RSC should not issue a warning notice, except where the extent of the change in performance cannot be explained by the impact of the changes to primary assessment arrangements in this transitional year. (Schools causing concern guidance) As this is the first year of schools working with the new interim assessment frameworks, the Minister for Schools has written to the Chief Inspector asking him to ensure that Ofsted inspectors take into account national performance and contextual factors when considering a school's performance in writing at KS2, which is used as part of the floor standard (Writing clarification document)

Writing TA is not used as part of the progress 8 baseline for KS4; only reading and maths tests scores are.

4: The writing progress measure

4.5: Setting targets based on these Prior Attainment Group (PAG) KS1 average points score Average KS2 Reading Score for PAG Average KS2 Writing Score for PAG Average KS2 Maths Score for PAG 1 >0 to <2.5 77.48 77.39 78.23 2 >=2.5 to <2.75 80.79 80.11 81.53 3 >=2.75 to <3 82.92 82.05 84.14 4 >=3 to <6 85.58 84.01 87.50 5 >=6 to <9 87.87 85.94 90.05 6 >=9 to <10 90.76 89.14 92.11 7 >=10 to <12 93.96 92.60 95.55 8 >=12 to <13 95.77 94.21 97.69 9 >=13 to <14 97.26 96.69 98.33 10 >=14 to <14.5 98.38 98.22 99.77 11 >=14.5 to <15 99.23 100.66 12 >=15 to <15.5 100.62 100.75 101.50 13 >=15.5 to <16 102.46 101.67 102.14 14 >=16 to <16.5 102.60 102.06 103.58 15 >=16.5 to <17 104.13 102.78 104.50 16 >=17 to <18 105.56 104.10 104.97 17 >=18 to <19 106.81 104.74 106.33 18 >=19 to <20 107.96 105.77 107.54 19 >=20 to <21 109.04 106.11 109.41 20 >=21 to <21.5 111.58 108.68 110.57 21 >= 21.5 115.70 110.40 114.51 4.5: Setting targets based on these

Nominal Scores for teacher assessments Not applied to HNM and EXS in reading and maths unless special consideration granted (80 points) PKS codes: BLW: 70 PKF: 73 PKE: 76 PKG: 79 Writing codes: WTS: 91 EXS: 103 GDS: 113 NB: Only applied for purposes of progress measures

VA for all possible KS2 Writing outcomes Prior Attainment at KS1 KS2 Estimate VA for all possible KS2 Writing outcomes Prior Attainment Group (PAG) KS1 average points score Average KS2 Writing Score for PAG BLW PKF PKE PKG WTS EXS GDS 70 73 76 79 91 103 113 1 >0 to <2.5 77.39 -7.39 -4.39 -1.39 1.61 13.61 25.61 35.61 2 >=2.5 to <2.75 80.11 -10.11 -7.11 -4.11 -1.11 10.89 22.89 32.89 3 >=2.75 to <3 82.05 -12.05 -9.05 -6.05 -3.05 8.95 20.95 30.95 4 >=3 to <6 84.01 -14.01 -11.01 -8.01 -5.01 6.99 18.99 28.99 5 >=6 to <9 85.94 -15.94 -12.94 -9.94 -6.94 5.06 17.06 27.06 6 >=9 to <10 89.14 -19.14 -16.14 -13.14 -10.14 1.86 13.86 23.86 7 >=10 to <12 92.60 -22.60 -19.60 -16.60 -13.60 -1.60 10.40 20.40 8 >=12 to <13 94.21 -24.21 -21.21 -18.21 -15.21 -3.21 8.79 18.79 9 >=13 to <14 96.69 -26.69 -23.69 -20.69 -17.69 -5.69 6.31 16.31 10 >=14 to <14.5 98.22 -28.22 -25.22 -22.22 -19.22 -7.22 4.78 14.78 11 >=14.5 to <15 99.23 -29.23 -26.23 -23.23 -20.23 -8.23 3.77 13.77 12 >=15 to <15.5 100.75 -30.75 -27.75 -24.75 -21.75 -9.75 2.25 12.25 13 >=15.5 to <16 101.67 -31.67 -28.67 -25.67 -22.67 -10.67 1.33 11.33 14 >=16 to <16.5 102.06 -32.06 -29.06 -26.06 -23.06 -11.06 0.94 10.94 15 >=16.5 to <17 102.78 -32.78 -29.78 -26.78 -23.78 -11.78 0.22 10.22 16 >=17 to <18 104.10 -34.10 -31.10 -28.10 -25.10 -13.10 -1.10 8.90 17 >=18 to <19 104.74 -34.74 -31.74 -28.74 -25.74 -13.74 -1.74 8.26 18 >=19 to <20 105.77 -35.77 -32.77 -29.77 -26.77 -14.77 -2.77 7.23 19 >=20 to <21 106.11 -36.11 -33.11 -30.11 -27.11 -15.11 -3.11 6.89 20 >=21 to <21.5 108.68 -38.68 -35.68 -32.68 -29.68 -17.68 -5.68 4.32 21 >= 21.5 110.40 -40.40 -37.40 -34.40 -31.40 -19.40 -7.40 2.60

5: the ostracism of CVA

DfE view of CVA in 2011 Contextual Value Added (CVA) goes further than simply measuring progress based on prior attainment by making adjustments to account for the impact of other factors outside of the school’s control which are known to have had an impact on the progress of individual pupils e.g. levels of deprivation. This means that CVA gives a much fairer statistical measure of the effectiveness of a school and provides a solid basis for comparisons.

Why we need CVA Fair, like for like comparisons Compare achievement of your pupils with that of similar pupils in similar schools Pupils with low start point: the SEN vs EAL issue FFT use 13 different factors in their CVA model By comparing like-for-like, CVA gets closer to the school effect on pupil progress.

6: the progress loophole of despair

Included in Average score? Included in progress measure Teacher Assessment Test Status Score Included in Average score? Included in progress measure PKS Did not take test 70-79 No Yes Did take test but not enough marks to achieve scaled score Took test and enough marks to achieve scaled score 80+ HNM Took test but not enough marks to achieve scale score No score

7: the one about getting rid of expected progress measures

the ‘[expected progress] measure has been replaced by a value-added measure. There is no ‘target’ for the amount of progress an individual pupil is expected to make.’ Primary Accountability Technical Guide, October 2016

Hmmmm

8: Progress measures in 2020

Because…… The DfE didn’t collect scaled scores Fewer outcomes than under levels Therefore fewer prior attainment groups (possibly 13?) containing more pupils All pupils in specific group have same ‘expectations’ Schools should consider alternative, more robust progress measures to counter negative messages from future data.

9: colour codes in RAISEonline

This is statistically significant (sort of) These are statistically significant based on 95% confidence interval These are statistically significant based on 95% confidence interval + an arbitrary threshold

And this isn’t! Diff (no. of pupils) Presents the gap from national average as a number of pupils. E.g. If you have 20 pupils, each pupil is worth 5%. Therefore a gap of 15% equates to 3 pupils. Gap 18% = 3 pupils (1 pupil = 5%): dark green Gap 4% = 1 pupil (1 pupil 3%): light green Gap -16% = 1 pupil (1 pupil 14%): light red Gap -9% = 3 pupils (1 pupil = 3%): dark red

On the subject of statistical significance Simply identifies a deviation from the mean that probably didn’t happen by chance You cannot infer cause i.e. sig+/- is not necessarily due to school effect (CVA is a better indicator) Red does not necessarily mean the school has done something wrong Green does not necessarily mean the school has done something right It’s just a threshold - a 0.01 point shift is all it takes for data to become ‘significant’ or not.

At least it’s the last year of RAISE

10: recreating levels

Levels were removed because…… They were best-fit so pupils could have serious gaps but still be placed within a level They led to a focus on getting pupils across the next threshold Progress became synonymous with moving on to the next level, rather than developing deeper or wider understanding They told us nothing about what a pupil could and couldn’t do Pupils at opposite sides of level boundary could have more in common that pupils within a level.

These are levels! Band % Objectives Step/Point Y1 Secure 67-100% 3 Y2 Emerging 0-33% 4 Y2 Developing 34-66% 5 Y2 Secure 6 Y3 Emerging 7 Y3 Developing 8 Y3 Secure 9 Y4 Emerging 10

and this is just weird Band % Objectives Step/Point Y2 Emerging 0-25% 4 Y2 Developing 26-50% 5 Y2 Secure 51-90% 6 Y2 Mastery 90-110% 7 Y3 Emerging Please note: mastery is not a thing that applies to the ‘most able’ that happens at the end of the year.

11: this!

‘KS2 progress was not significantly below average ‘KS2 progress was not significantly below average* overall or for any prior attainment group in any subject. *and not below -3’

and relax

data