How do you choose a washing machine? Using evidence to inform practice Dr Jonathan Sharples www.educationendowmentfoundation.org.uk.

How do you choose a washing machine? Using evidence to inform practice Dr Jonathan Sharples www.educationendowmentfoundation.org.uk

The EEF approach Summarise the existing evidence Make grants Evaluate projects Share and promote the use of evidence

Stand on the shoulders of previous progress i.e. healthcare, engineering Research is seen as something done to, not with, or for, or by the profession – culture change “We must give educators and politicians the information they need to make wise decisions for children” Estelle Morris

What is evidence-informed practice? How do we generate useful evidence for practice? What can be done to make research more accessible to busy professionals? How might we build capacity in professional settings to use research evidence at scale? Make abstract concepts a bit more tangible... www.alliance4usefulevidence.org

Current moves towards more evidence-informed policy/practice… Cabinet Office ‘What works’ centres Royal College of Teachers ResearchED

Need for good evidence is increasingly agreed Weak link is the coupling of evidence to practice, at scale – ‘Knowledge mobilisation’ Challenges – Fragmentation in the system, weak links to training, OfSTED?

What are the challenges in accessing and using reliable research evidence? Sifting reliable research conclusions from the rest Not skilled to judge the validity of claims Can be a conflict with existing school practices Knowing where to look for useful information Too much information is available Senior Leadership Teams Time, time, time!

Researchers Policy makers Practitioners What would an evidence-using and creating culture look like? What do we need to get there? 2013/14 - Events series on knowledge mobilisation in practice www.cebenetwork.org

The EEF by numbers 83 evaluations funded to date 3,000 schools participating in projects 34 topics in the Toolkit 16 independent evaluation teams 600,000 pupils involved in EEF projects 14 members of EEF team £220 m estimated spend over lifetime of the EEF 6,000 heads presented to since launch 10 reports published

Applying evidence in practice External evidence summarised in the Toolkit can be used to inform choices. Step 2: Identifying possible solutions Evaluate the impact of your decisions and identify potential improvements for the future. Step 4: Did it work? Mobilise the knowledge and use the findings to inform the work of the school to grow or stop the intervention. Step 5: Securing and spreading change Applying the ingredients of effective implementation. Step 3: Giving the idea the best chance of success Identify school priorities using internal data and professional judgement. Step 1: Decide what you want to achieve 10

Applying evidence in practice Generate a question using data, professional judgement and values. How can we engage and support struggling readers in Year 7? Step 1: Decide what you want to achieve

Applying evidence in practice Ensure that you start from the best position by seeking internal and external knowledge. What has been shown to be effective in raising outcomes for struggling readers? Step 2: Identify possible solutions

A Pupil Premium Scenario – Struggling Readers What do you decide to spend the money on? How do you make the most of that investment? Number on Roll%FSMPupil Premium Allocation 75010%£67,500 Parents Class size reduction Head of English One to one tuition SENCO Employ more Teaching Assistants

Teaching and Learning Toolkit The Toolkit is an accessible, teacher-friendly summary of educational research. ‘Which?’ for education Practice focused: tries to give schools the information they need to make informed decisions and narrow the gap. Based on meta-analyses conducted by Durham University.

Effective classroom strategies for closing the gap in educational achievement for children and young people from poor backgrounds, including white working class boys

Teaching and Learning Toolkit The Toolkit is an accessible, teacher-friendly summary of educational research. ‘Which?’ for education Practice focused: tries to give schools the information they need to make informed decisions and narrow the gap. Based on meta-analyses conducted by Durham University.

The Toolkit is a starting point for making decisions

Overview of value for money Cost per pupil Effect Size (months gain) £0£0 0 10 £1000 Feedback Meta-cognitive Peer tutoring Pre-school 1-1 tutoring Homework ICT Outdoor learning Parental involvement Sports Summer schools After school Individualised learning Learning styles Arts Performance pay Teaching assistants Smaller classes Ability grouping Phonics Independent learning

Using the Toolkit Use the evidence as a starting point for discussion. Dig deeper into what the evidence actually says Understand the ‘active ingredients’ of implementation

Example: Teaching Assistants

Better magazine and Best Evidence in Brief Three/year. Free for first year – www.betterevidence.org Free fortnightly research digest – iee@york.ac.uk

Applying evidence in practice ? Is there disruption to other learning? How will you organise the tuition during classtime? How much training do TA ’ s need? Implementation matters: have you thought about what the approach means for teaching and learning? What are the ‘active ingredients’ for making best use of teaching assistants for struggling readers? Step 3: Give the idea the best chance of success

Implementation matters: how is as important as what the evidence says Shortage of practical vehicles (interventions, CPD training) to help get evidence working in practice, at scale and with rigour (eg AfL) In the US, healthcare workers failure to wash hands effectively is major cause of death – $billions Researcher created a checklist for surgical teams. Trial showed 66% reduction in infection rates, ~1500 lives in 18 months. Packaged the principles of handwashing into a practical intervention.

EEF Projects We are working to fund, develop and evaluate projects that: Build on existing evidence. Will generate significant new understanding of ‘what works’. Can be replicated cost effectively if proven to work. Examples: One-to-one support with teaching assistants, lesson observation, using mobile devices for feedback,

Switch on Reading RCT in 19 schools with 300 pupils Attainment measured using standardised literacy measures Independent evaluation by Durham University Observations and interviews to inform how and why the approach might be working http://educationendowmentfoundation.org.uk/projects/category/primary One to one literacy intervention with children in Year 7 who are struggling with literacy (not achieving level 4 at KS2) Based on Reading Recovery. Delivered by teaching assistants, 20mins/day over 10 weeks. Previous research shows a positive effect (inc. Reading Recovery).

Switch On Reading Group Number of pupils Effect size* Estimated months’ progress All pupils308+0.24+3 Lower attainers 156+0.39+5 FSM-eligible98+0.36+4 SEN reported225+0.31+4

Switch On Reading - Conclusions Key Conclusions 1. Switch-on Reading appears to be effective for weak and disadvantaged readers at the stage of transition to secondary school. 2. It can be delivered by teaching assistants after two-days of training, and full training and support is required for all relevant staff. 3. Challenges to successful implementation may include timetabling and the availability of age-appropriate texts. 4. There is a tendency for some staff to stray away from the explicit schedule and this is likely to reduce the programme’s impact. Regular monitoring will increase fidelity but may also increase cost.

Catch Up Numeracy One to one intervention with children in Years 2 to 6 who are struggling with numeracy Two 15 minute sessions with TAs per week for 30 weeks Group Number of pupils Effect size Estimated months’ progress Catch Up Numeracy vs. control 108+0.21+3 Equivalent time one to one support vs. control 102+0.27+4

Catch Up Numeracy - Conclusions Key Conclusions 1. Within this trial, one-to-one support by TAs led to a significant gain in numeracy skills. 2. Catch Up Numeracy makes similar significant gains, but there is little evidence that it provided any additional gains in numeracy outcomes over and above those from one to one teaching itself. 3. Schools can find it challenging to run two 15 minutes sessions per week, due to timetabling and other issues. 4. Structured interventions, such as Catch Up Numeracy, should be planned into the timetable from the start of the new school year to ensure they are given priority and status.

What have we learned? Teaching assistants, given the right support and training, can make a significant contribution to pupil attainment Schools should take care to understand how specific programmes are having an impact in their school. Implementation matters: brief, well-structured 1-2-1 sessions over a sustained period, with appropriate support and training

Not all positive findings though….

IEE Evidence for Impact database… coming soon

Applying evidence in practice Did the approach work, what made it work, and how can it be improved next time? Can we demonstrate that our readers are making progress? Is it worth the effort? Step 4: Put energy into evaluation

The Toolkit is a starting point for making decisions

Overview of value for money Cost per pupil Effect Size (months gain) £0£0 0 10 £1000 Feedback Meta-cognitive Peer tutoring Pre-school 1-1 tutoring Homework ICT Outdoor learning Parental involvement Sports Summer schools After school Individualised learning Learning styles Arts Performance pay Teaching assistants Smaller classes Ability grouping Promising May be worth it Requires careful consideration Phonics Independent learning