1. ESF Evaluation Plan England 2014-2020
Bruce Byrne,
Head of England
ESF evaluation team

2. Development of the evaluation plan
January to March 2014
Develop ideas
Development of the evaluation plan
April
Workshop
May
First draft
June
Consultation
Ex ante evaluators
Timing to be agreed
Second draft
Timing to be agreed
Management Approval
Submission to Monitoring Committee
November/December

3. Principles for where to evaluate
Requirements
- what is “the impact”?
- how/why does it work?
- how can the provision be improved?
- how do our different contracting arrangements help provision to be a success?
Data availability
- E.g. counterfactual impact evaluations need rich data to ensure you have a genuine counterfactual.
Expertise
- Where this is for evaluation methods and knowledge of the data.
Budgets
- Where these come from
What we already know about what works
- Less need for evaluation where we already have good evidence

4. Other considerations Role of Local Enterprise Partnerships
We don’t think (most) local bodies have the capacity to add value by conducting their own evaluations.
Do need their buy-in, to help providers see the importance of evaluation.
Do need to ensure that results are feeding into local bodies to aid decisions about what provision they want in their area.
Oversight of the plan
programme has oversight from the Evaluation sub-committee (and to a lesser extent directly from the National Monitoring Committee)
For we are looking at ways to make this role more strategic by:
- making this a joint ERDF/ESF group.
- adding independent evaluation expertise to the group.

5. Improving how we use data
All participants should sign consent for their data to be linked for evaluation purposes
All providers should make available participant names and contact details for the leavers survey. All participants should sign consent to share these for evaluation purposes
Data requirements for evaluation need to be embedded at the beginning of the programme
Requirements for evaluation should be built into contracts in the same way that requirements for management information currently are

6. Progress (1) Good progress on: Learning lessons from 2007-13
Establishing oversight
Arrangements for improving data
Timings
Outstanding work to do on:
What the evaluation team will look like
The approach to evaluation
e.g. big projects looking across priorities and delivery areas or a series of more focused studies followed by a synthesis of findings?
Budgets required
Specifying individual evaluations

7. Progress (2): Early thoughts on individual evaluations
2015
YEI Implementation evaluation
2016
ESF Implementation evaluations
Set up leavers survey
2017
First year of fieldwork for leavers survey
Start of counterfactual impact evaluation
2018
Final year of fieldwork for leavers survey
Conclusion of counterfactual impact evaluation
Theory based impact evaluation
YEI impact evaluation

8. Summary: Key points in building the England ESF Evaluation Plan
Consulting on the plan is going to be particularly important as we are moving to a new organisational set up
Ensuring clarity of purpose on what each evaluation will add and how we will use it
Doing more counterfactual impact evaluations can require a lot of preparation, ensuring: the understanding of the data; the expertise in methods; and the right resources are in place.
Getting the data sorted out now so that when it comes to doing the evaluations everyone’s life will be a lot easier.
Securing early approval from the National Monitoring Committee so that we can put in place data collection and start the YEI evaluation