Session 3: Analysis and reporting Collecting data for cost estimates Jack Worth (NFER) Panel on EEF reporting and data archiving Peter Henderson, Camilla Nevill, Steve Higgins and Andrew Bibby

Data collection for cost estimates Jack Worth, NFER EEF London Evaluators Workshop 6 th June 2014

Summary Reporting cost is an important part of evaluations Collecting the right information can be theoretically and practically challenging Evaluators should be sharing experiences and best practice

Effectiveness

Cost effectiveness

Costs to consider collecting Direct costs – how much would it cost a school or parent to buy the intervention? –how much did it cost may differ Indirect costs – do staff have to put in more hours than usual? –always think: what is the counterfactual? Needs to be quantitative information

Collecting cost information Impact or process evaluation? –cost effectiveness relates to impact... –...but process methods e.g. surveys, case studies, often better suited to cost data Planning and communication –across evaluation project team –with development partner

Reporting cost information Present the average cost to compare with average effectiveness Cost per pupil or per school? –may depend on the specific intervention –cost per pupil is comparable to other interventions Present the assumptions made and data sources used

Sharing best practice Recommend agreeing principles for a common approach among evaluators Questions?

NFER provides evidence for excellence through its independence and insights, the breadth of its work, its connections, and a focus on outcomes. National Foundation for Educational Research The Mere, Upton Park Slough, Berks SL1 2DQ T: F: E:

EEF reporting and data archiving Peter Henderson (EEF) Camilla Nevill (EEF) Steve Higgins (Durham) - Chair Andrew Bibby (FFT)

The reporting process and publication of results on EEF’s website Peter Henderson(EEF)

Reporting process Evaluation team Dissemination team

Classifying the security of findings from EEF evaluations Camilla Nevill (EEF)

Example Appendix: Chatterbooks

Combining the results of evaluations with the meta-analysis in the Teaching and Learning Toolkit Steve Higgins (Durham)

Andrew Bibby Archiving EEF project data

1.Include permission for linking and archiving in consent forms 2.Retain pupil identifiers 3.Label values and variables 4.Save Syntax or Do files Prior to archiving…