1. How Data and Analysis techniques inform effective Teaching and Learning■
Powerful things that schools can now do with their data
Mike Bostock, New Media Learning

2. Should a school ever be surprised by its Ofsted report?
Question for discussion
Should a school ever be surprised by its Ofsted report?
Consensus view of group: A school that knows itself well would not be surprised by the findings of a school inspection

3. “Satisfactory was no longer good enough” – HMCI Gilbert
Are schools doing better or worse than they used to?
11% outstanding, 48% good, 34% satisfactory, 8% inadequate - Ofsted Annual Report
Nearly 60% of schools are good or better
“Satisfactory was no longer good enough” – HMCI Gilbert
Only 50% of schools are average!
“Half of schools are failing” - Guardian October 06
Headline figures about ‘standards’ are not always helpful

4. The data used to judge schools
RAISEonline (ex-PANDA) data will form the basis of the main judgments about a school’s standards
because:
School inspections are too brief to add much to these judgements
A school’s own self-evaluation evidence provided in the SEF tends to be descriptive rather than quantitative
e.g. “We do lots of lesson observations”

5. Issues to do with using National Data to judge schools
It can come too late in the year to be useful rather than simply judgemental
Ofsted global figures are good for Government stats but less useful for school improvement
Contextual Value Added adjustments are not a precise way to compare one school with another

6. CVA
Contextual Value Added adjustment discounts one or more variables (ethnicity, gender etc) so that comparisons can be made on a fairer basis
but…
It is a statistical technique that really only works with large sample sizes.
To make CVA accurate enough to make judgements between one school and another it would need to be refined by discounting every factor known to affect a pupil’s performance
It has no meaning at class level. A good teacher will aim to do the best for every pupil - and not expect some individual pupils to do less well because of their ethnicity or gender etc.

7. CVA
Where inspectors say that CVA data shows the school should be doing better, what independent, pupil-level source of measures can a school use to put this judgement into context?
A growing number of schools are able to provide secure alternative evidence arising from their own pupil-level analysis and research

8. How would school self-analysis complement RAISEonline data?
School level analysis
available Spring Term the following year
available the day the exam results arrive
inspection could be based on information that is a year out of date
inspection would be based on the most up-to-date performance information
ownership and control in the hands of Government
ownership and control in the hands of schools
offers an external evaluation of a school’s performance
offers teachers an opportunity to make their own evaluation
evaluates pupils’ performance with national contextual data
evaluates pupils’ performance with school contextual data
forms the basis of Ofsted judgments on the school
provides measures to support the school's own view of its performance
judges leadership on basis of CVA comparisons with other schools
allows a school to show that good leadership equates with successfully tackling negative variation

9. Within School Variation (WSV)
There is over 4 times the variation in provision within schools than between schools - source OECD
Reducing negative variation would raise standards by 10% - source DfES
"We have always know that there is a difference in performance between schools. But what can make a bigger difference is the experience that children have within one school. So a child can do really well in one subject and not do well in another subject. And that can make an even bigger difference to children's life chances than differences between schools." - Jane Creasy, Assistant Director of Research, NCSL

10. High Achievement and Low Variation
What is the better basis on which school leadership should be judged?
High Standards
High Achievement and Low Variation
5 A*-Cs
How well every pupil achieves
Some Children Matter
Every Child Matters
Headline figures can hide significant pockets of underachievement
Headline figures should show how well every child achieves
Good Leadership = high attainment
Good Leadership = evidence of doing the best for every pupil
League table position
Measures of Within School Variation
Performance analysis is done by the few and passed to the many
All teachers are involved with analysing their pupils’ performance
Judged by official data
Secure school-level evidence of effectiveness
School Improvement = more 5A-Cs
School Improvement = less negative variation
Ofsted Global measures are useful to know where to target the Education budget but not much use to schools

11. “Schools that are proactive in showing inspectors the evidence of their own pupil-level analysis and research tend to do better in their inspection.”
- Dr. Mike Treadaway, Fischer Family Trust, Naace ‘Making Information Work’ Conference

12. Jack is a C level pupil with an F in this subject – should Jack or his teacher try harder?

13. An analysis of variation can shine a light on the achievement of different groups of pupils

14. The Data Confidence Toolkit
The use of pupil-level data analysis and research is relatively underdeveloped at this point in time
Good schools will be those that can show that they know themselves well, are targeting negative variation, and can show evidence of improvement
“The Data Confidence Toolkit is a good starting point for schools” - Dr Mike Treadaway

15. Question:
Should every school be expected to prove that for them ‘Every Child Matters’ - rather than just those pupils who will get 5 A*-C grades?