How Data and Analysis techniques inform effective Teaching and Learning ■ Powerful things that schools can now do with their data... Within School Variation project website at:

Should a school ever be surprised by its Ofsted report? Question for discussion Consensus view of group: A school that knows itself well would not be surprised by the findings of a school inspection

“Satisfactory was no longer good enough” – HMCI Gilbert “Half of schools are failing” - Guardian October 06 11% outstanding, 48% good, 34% satisfactory, 8% inadequate - Ofsted Annual Report Are schools doing better or worse than they used to? Nearly 60% of schools are good or better Headline figures about ‘standards’ are not always helpful

The data used to judge schools because: School inspections are too brief to add much to these judgements A school’s own self-evaluation evidence provided in the SEF tends to be descriptive rather than quantitative e.g. “We do lots of lesson observations” RAISEonline (ex-PANDA) data will form the basis of the main judgements about a school’s standards

Issues to do with using National Data to judge schools It can come too late in the year to be useful rather than simply judgemental Ofsted global figures are good for Government stats but less useful for school improvement Contextual Value Added adjustments are not a precise way to compare one school with another

CVA Contextual Value Added adjustment discounts one or more variables (ethnicity, gender etc) so that comparisons can be made on a fairer basis It is a statistical technique that really only works with large sample sizes. To make CVA accurate enough to make exact comparisons between one school and another it needs to discount every external factor known to affect a pupil’s performance, to leave only those attributable to the influence of the school. It has no meaning at class level. A good teacher will aim to do the best for every pupil - and not expect some individual pupils to do less well because of their ethnicity or gender etc. but…

CVA Where inspectors say that CVA data shows the school should be doing better, what independent, pupil-level source of measures can a school use to put this judgement into context? A growing number of schools are able to provide secure alternative evidence arising from their own pupil-level analysis and research

There is over 4 times the variation in provision within schools than between schools - source OECD Reducing negative variation would raise standards by 10% - source DfES Within School Variation (WSV) "We have always known that there is a difference in performance between schools. But what can make a bigger difference is the experience that children have within one school. So a child can do really well in one subject and not do well in another subject. And that can make an even bigger difference to children's life chances than differences between schools." - Jane Creasy, Assistant Director of Research, NCSL

RAISEonlineSchool self analysis available late in the Autumn term available the day the exam results arrive inspection could be based on information that is out of date inspection would be based on the most up-to-date performance information ownership and control in the hands of Government ownership and control in the hands of schools offers an external evaluation of a school’s performance offers teachers an opportunity to make their own evaluation evaluates pupils’ performance with national contextual data evaluates pupils’ performance with school contextual data forms the basis of Ofsted judgments on the school provides measures to support the school's own view of its performance judges leadership on basis of CVA comparisons with other schools allows a school to show that good leadership equates with successfully tackling negative variation How would school self-analysis complement RAISEonline data?

What is the better basis on which school leadership should be judged? High Standards High Achievement and Low Variation 5 A*-CsHow well every pupil achieves Some Children MatterEvery Child Matters Headline figures can hide significant pockets of underachievement Headline figures should show how well every child achieves Good Leadership = high attainment Good Leadership = evidence of doing the best for every pupil League table positionMeasures of Within School Variation Performance analysis is done by the few and passed to the many All teachers are involved with analysing their pupils’ performance Judged by official data Secure school-level evidence of effectiveness School Improvement = more 5A-Cs School Improvement = less negative variation

“Schools that are proactive in showing inspectors the evidence of their own pupil-level analysis and research tend to do better in their inspection.” - Dr. Mike Treadaway, Fischer Family Trust, Naace ‘Making Information Work’ Conference

Jack is a C level pupil with an F in this subject – should Jack or his teacher try harder?

An analysis of variation can shine a light on the achievement of different groups of pupils

The ‘Data Confident School’ Toolkit The use of pupil-level data analysis and research is relatively underdeveloped at this point in time Good schools will be those that can show that they know themselves well, are targeting negative variation, and can show evidence of improvement “The Data Confidence Toolkit is a good starting point for schools” - Dr Mike Treadaway

Should every school be expected to prove that for them ‘Every Child Matters’ - rather than just those pupils who will get 5 A*-C grades? Question: