1. The rise of RCTs in education: statistical and practical challenges Kevan Collins

4. The EEF Summarise the existing evidence Make grants Evaluate projects Share and promote the use of evidence Rigorous, independent evaluations Founded with a grant from the DfE with an aim to spend over £220 million over 15 years Focus on projects with a good evidence-base 115 grants made to date EEF and Sutton Trust Toolkit, reports, scale up campaigns

5. EEF’s Approach to Evaluation Rigorous, independent evaluations Independent evaluation Robust counterfactual Focus on attainment outcomes Impact and process evaluations

6. A typical EEF evaluation Recruit schools Pre-test / pupil data / consent Post- test Set-up / Planning Intervention delivery Randomise Process evaluation When RCT not possible RDD, propensity score matching, synthetic analysis 94/115 projects evaluated using RCTs Keen to include non-cognitive outcomes Focus on attainment (end of KS, standardised commercial tests) 115 grants made to date

7. All EEF evaluation data will be submitted to a national data archive This data archive is still being established. The aim is to allow… ….the EEF to: track the impact of projects longitudinally look at the cumulative impact of projects understand better its target group ….the research community to: verify the results of EEF evaluations conduct further analysis on subgroups and interventions link to other datasets for research purposes Overarching evaluators need access Research community needs access

8. Some lessons from our first 4 years… Schools are willing to take part in RCTs. When we first set up, some people were sceptical But getting them to do what you need them to is difficult. Testing, passing on data, sticking to the intervention, not contaminating the control group… Effect sizes are often smaller than anticipated. Maybe due to the independence of the evaluation; the “real world” nature of the trials; intention to treat analysis This means that trials need to be very large to detect an impact. Therefore we are increasingly reliant on NPD data rather than testing 1,000s of pupils. Next big questions: Will schools listen to the evidence? And policy makers? Will researchers use the data archive?