ESF Evaluation Plan England 2014-2020 Bruce Byrne, Head of England ESF evaluation team

Development of the evaluation plan January to March 2014 Develop ideas Development of the evaluation plan April Workshop May First draft June Consultation Ex ante evaluators Timing to be agreed Second draft Timing to be agreed Management Approval Submission to Monitoring Committee November/December

Principles for where to evaluate Requirements - what is “the impact”? - how/why does it work? - how can the provision be improved? - how do our different contracting arrangements help provision to be a success? Data availability - E.g. counterfactual impact evaluations need rich data to ensure you have a genuine counterfactual. Expertise - Where this is for evaluation methods and knowledge of the data. Budgets - Where these come from What we already know about what works - Less need for evaluation where we already have good evidence

Other considerations Role of Local Enterprise Partnerships We don’t think (most) local bodies have the capacity to add value by conducting their own evaluations. Do need their buy-in, to help providers see the importance of evaluation. Do need to ensure that results are feeding into local bodies to aid decisions about what provision they want in their area. Oversight of the plan 2007-13 programme has oversight from the Evaluation sub-committee (and to a lesser extent directly from the National Monitoring Committee) For 2014-2020 we are looking at ways to make this role more strategic by: - making this a joint ERDF/ESF group. - adding independent evaluation expertise to the group.

Improving how we use data All participants should sign consent for their data to be linked for evaluation purposes All providers should make available participant names and contact details for the leavers survey. All participants should sign consent to share these for evaluation purposes Data requirements for evaluation need to be embedded at the beginning of the programme Requirements for evaluation should be built into contracts in the same way that requirements for management information currently are

Progress (1) Good progress on: Learning lessons from 2007-13 Establishing oversight Arrangements for improving data Timings Outstanding work to do on: What the evaluation team will look like The approach to evaluation e.g. big projects looking across priorities and delivery areas or a series of more focused studies followed by a synthesis of findings? Budgets required Specifying individual evaluations

Progress (2): Early thoughts on individual evaluations 2015 YEI Implementation evaluation 2016 ESF Implementation evaluations Set up leavers survey 2017 First year of fieldwork for leavers survey Start of counterfactual impact evaluation 2018 Final year of fieldwork for leavers survey Conclusion of counterfactual impact evaluation Theory based impact evaluation YEI impact evaluation

Summary: Key points in building the England ESF Evaluation Plan Consulting on the plan is going to be particularly important as we are moving to a new organisational set up Ensuring clarity of purpose on what each evaluation will add and how we will use it Doing more counterfactual impact evaluations can require a lot of preparation, ensuring: the understanding of the data; the expertise in methods; and the right resources are in place. Getting the data sorted out now so that when it comes to doing the evaluations everyone’s life will be a lot easier. Securing early approval from the National Monitoring Committee so that we can put in place data collection and start the YEI evaluation