1. Using research to get the best value from the Pupil Premium Steve Higgins, School of Education, Durham University s.e.higgins@durham.ac.uk @stig_01 National Middle Schools’ Forum Conference 2013 21 st October 2013 Stratford Manor Hotel, Stratford on Avon

2. Sutton Trust/EEF Teaching and Learning Toolkit Why we wrote it Best ‘buys’ on average Key messages for spending the Pupil Premium Currently used by about a third of schools http://educationendowmentfoundation.org.uk/toolkit

3. The pupil premium Aims: to close the attainment ‘gap’ between the highest and lowest achieving to increase social mobility to enable more pupils from disadvantaged backgrounds to get to the top Universities to provide additional resource to schools to do this Was £600 in 2012-13 for fsm 1 pupils; increased to £900 in 2013-14 and £1300 in 2014-15. 1 any child registered for fsm in the last six years and all looked after children, smaller premium for children of Service families

4. Resources and learning Above a minimum threshold – no simple link Conclusion: spending more won’t guarantee improvement - no simple solution More money ≠ more learning

5. Smaller classes? Complex evidence- no clear link with class size and achievement Experimental trials suggest Classes need to be less than about 17 for 0.2 effect size… And teachers need to change the way they teach… But support from teaching assistants not as effective The maths: £900 x 25 pupils x 3 classes with 50% on fsm = £33,750 = 1 extra teacher per 3 classes Class size reduction from 25 to 19 – expensive for little gain

6. One-to-one tuition Highly effective I hour/ day over at least 6 weeks Support for class teacher to re-integrate Effect size 0.44 The maths… 6 weeks x 5 days x 1 hour = 30 hours 4 days teacher time (more effective with an experienced teacher) Approx £700 (ECC models less costly) Expensive but very effective Consider using pairs or triplets?

7. What should the Pupil Premium ‘buy’? Secondary £5,200 per pupil Primary £4,284 Wide variation Secondaries £4,000 to £9,000 Primaries £3,000 and £8,000 Middle Schools £3,300 - £8,000 (median £4,100) Excludes Academies and Free Schools Will £1,300 buy an extra three or four months learning for each pupil eligible for the Pupil Premium? (In England, data from 2009-10: https://www.gov.uk/government/publications/funding-for-primary-and-service-childrens-education-schools )https://www.gov.uk/government/publications/funding-for-primary-and-service-childrens-education-schools

8. The Bananarama Principle It ain’t what you do it’s the way that you do it… So how do you spend to “get results”? Or, what does the evidence say is a good investment or a poor investment for learning? It ain’t what you spend it’s the way that you spend it…

9. What we tried to do Summarise the evidence from meta-analysis about the impact of different strategies on learning (tested attainment) As found in research studies These are averages Apply quality criteria to evaluations: rigorous designs only Estimate the size of the effect Standardised Mean Difference = ‘Months of gain’ Estimate the costs of adopting Information not always available

10. Toolkit Best ‘buys’... New entry

11. Toolkit Good ‘buys’... New entry

12. Toolkit Low range...

13. Worst ‘buys’... (on average) Toolkit New entry Revised entry

14. Summaries What is it? How effective is it? How secure is the evidence? What are the costs? What do I need to know?

15. The methodology Systematic searches for meta-analyses of interventions and quantitative estimates of impact Effect sizes aggregated (fixed effect model) Converted to months progress Moderator variables summarised Costs estimated in terms of outlay for schools Variation within toolkit themes greater than variation between them: it ain’t what you do… Technical appendix available on EEF website

16. Toolkit

17. Homework (Secondary) Overview of value for money Cost per pupil Effect Size (potential months gain) £0 0 1.0 £1000 Meta-cognition Peer tutoring EY intervention 1-1 tutoring Digital technology Parental involvement Summer schools After school Individualised learning Performance pay Teaching assistants Smaller classes Ability grouping Promising Could be worth it Needs careful thought Feedback Phonics

18. Key messages Some things that are popular or widely thought to be effective are hard to make work well in terms of tested attainment Ability grouping (setting); After-school clubs; Deployment of teaching assistants; Performance pay Some things look more ‘promising’ Effective feedback; Meta-cognition and self regulation strategies; Early years intervention; Peer tutoring; Small group/intensive tuition; Parental involvement and engagement

19. Issues and limitations Based on meta-analysis – averages of averages Conversion to ‘months progress’ is a rough estimate Intervention research is compared with ‘normal’ practice which is varied Not ‘what works’ but what has worked – ‘good bets’ to support professional enquiry

20. The challenges (1): implementation These strategies have been shown to be cost-effective in research studies But when we have tried to implement evidence-based strategies we have not seen system-wide improvement (e.g. AfL, SureStart) We don’t know how to get schools/teachers who are not currently doing them to do so in ways that are True to the key principles Feasible in real classrooms – with all their constraints Scalable and replicable Sustainable

21. The challenges (2) : context This is what has worked (on average) Where is there leverage for improvement in a school or class? Will it build capacity? (i.e. is it efficient?) For learners? For teachers? Is it inclusive?

22. For disadvantaged /struggling learners… One ‘intervention’ won’t be enough Identify areas of greatest need Clear focus on improving learning, not (just) behaviour Mid and high attaining learners can be disadvantaged too! Effects will need to be cumulative What will build learning capacity and capability? Need to track and evaluate – our best guesses are not always good enough

23. Evaluate impact EEF’s DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

24. Establishing a baseline Teacher assessment vs standardised tests? Individual progress vs comparison group? Test score increases or age-standarised progress? Evidence suggests Teacher Assessment tends to over-estimate progress BUT reliance on standardised tests narrows the curriculum and encourages test preparation

25. Feedback Meta-cognition Self regulation Peer tutoring Small group tuition Phonics TA support (Parent involvement) (Early years intervention) One-to-one SEAL Behaviour

26. But remember…. “It ain’t what you do, it’s the way that you do it… that’s what gets results” Effective identification Adopting solutions, not bandwagons High quality intervention/support Evaluate to ensure learning progress Quality counts

27. Some Links The full report can be found on the EEF’s website: http://educationendowmentfoundation.org.uk/toolkit/ http://educationendowmentfoundation.org.uk/toolkit/ The toolkit is recommended by the Department for Education: http://www.education.gov.uk/schools/pupilsupport/premium/b00200492/ppstrat egies http://www.education.gov.uk/schools/pupilsupport/premium/b00200492/ppstrat egies Official information about the Pupil Premium and LA allocations is available at: http://www.education.gov.uk/schools/adminandfinance/financialmanagement/sc hoolsrevenuefunding/a00200697/pupil-premium-2012-13 http://www.education.gov.uk/schools/adminandfinance/financialmanagement/sc hoolsrevenuefunding/a00200697/pupil-premium-2012-13 Ofsted’s report is available at: http://www.ofsted.gov.uk/resources/pupil-premiumhttp://www.ofsted.gov.uk/resources/pupil-premium Find out how much each school gets: http://educationendowmentfoundation.org.uk/toolkit/pupil-premium-calculator DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy- evaluation-guide http://educationendowmentfoundation.org.uk/toolkit/pupil-premium-calculatorhttp://educationendowmentfoundation.org.uk/library/diy- evaluation-guide

28. Questions…

29. Using research to get the best value from the Pupil Premium: workshop session Steve Higgins, School of Education, Durham University s.e.higgins@durham.ac.uk @stig_01 National Middle Schools’ Forum Conference 2013 21 st October 2013 Stratford Manor Hotel, Stratford on Avon

30. ‘Best value’ Identify target group Improve effectiveness of provision/support Evaluate impact Track target pupils Adjust plan in response to tracking Use funding to achieve wider outcomes?

31. Low cost High Impact Low Impact High cost

33. Preparing for Ofsted Communicate your strategy Teachers Governors Website Track target pupils Record data on spending Evaluate accurately Improve plan Justify/argue!

34. Tracker links Ofsted resources: http://www.ofsted.gov.uk/resources/pupil-premium- analysis-and-challenge-tools-for-schoolshttp://www.ofsted.gov.uk/resources/pupil-premium- analysis-and-challenge-tools-for-schools Leading Learner blog:http://leadinglearner.me/2013/10/10/pupil-premium- analyser-and-tracker/http://leadinglearner.me/2013/10/10/pupil-premium- analyser-and-tracker/ NAHT: http://www.naht.org.uk/welcome/advice/advice-home/governance- and-infrastructure-advice/pupil-premium-reporting-2012-2013/http://www.naht.org.uk/welcome/advice/advice-home/governance- and-infrastructure-advice/pupil-premium-reporting-2012-2013/ TES: http://www.tes.co.uk/teaching-resource/Track-Impact-and-Spend-of- Pupil-Premium-6121277/http://www.tes.co.uk/teaching-resource/Track-Impact-and-Spend-of- Pupil-Premium-6121277/ Deepings School (CfBT Academy): http://www.deepingschool.org.uk/162/pupil- premiumhttp://www.deepingschool.org.uk/162/pupil- premium http://www.deepingschool.org.uk/uploads/asset_file/How%20are%20the%20P upil%20Premium%20pupils%20doing.pdf

35. For every complex problem there is a solution that is simple, neat… and WRONG! H.L. Mencken 1880-1956