1. ESF monitoring and evaluation in 2014-2020 Draft guidance
Directorate General Employment, Social Affairs and Inclusion
Evaluation and Impact Assessment Unit
Brussels
16 March 2012

2. Purpose of the presentation
Present the draft guidance on monitoring and evaluation of the ESF by focusing on new elements compared to current situation
Create a common understanding around certain concepts
Caution: the guidance remains a draft and will be revised in order to reflect the final Regulation

3. Programming Link to Europe 2020 is essential
Intervention logic to address challenges from NRP, country specific recommendations
Drafting the strategy shall not start from a blank sheet of paper but shall be based on previous analytical work in the context of Europe 2020

4. Monitoring Monitoring has two sides:
1. Continuous and systematic process of generating quantitative data on implementation
The guidance will focus on various aspects related to generating these data
2. Examination and discussion of these data
EC recommends to regularly present to and discuss with MC updated monitoring data.

5. Indicators Financial, output and result indicators are possible
Reported at the level of investment priorities
Indicators relate to partially and fully implemented operations
Indicators should be clearly defined, the chosen measurement unit shall be indicated and they shall be periodically measured.

6. Common indicators Capture output and results
Monitoring shall always use all common indicators although they might report zeros.

7. Common output indicators
Relate to persons (participants) and entities
A participant is reported ONLY when ALL personal data is recorded, i.e. gender, employment status, age – as applicable – education
One person can be counted more than ones BUT only when this person benefitted from the support of a separate operation
Only entering participants are reported, no leaving
EC recommends to enter the data as early as possible

8. Common immediate result indicator
Relate to persons only
Collected through MI system
Certain combinations of immediate result indicators for one person are possible

9. Common longer-term result indicators
Data reported only in 2019 and 2023
Data normally not collected through normal MI system
Data collected based on a representative sample
Representativeness established at the level of priority, socio-economic characteristics of participants as captured by output targets
Representativeness also for the regional dimension
Minimum: two non-overlapping samples

10. Programme-specific indicators
For the Member States to establish
Member States may want to set additional indicators which do not form part of the programme
Indicators should be underpinned by clear and easy to understand definitions
Definitions need to be made available from the start of programme implementation
Beneficiaries need to be aware of the definitions

11. Baseline and targets Baselines for result indicators only
Cumulative quantified target values for 2022
Targets expressed in absolute numbers
Targets set at the level of IP
Targets for a limited number of common and programme specific output and result indicators
Targets translate a political intention for change in a certain direction into a quantified value

12. Data collection and storage
All data to be transmitted electronically
Member States shall provide a computerised system for recording, filing and transmission of data
Processing of individual participants data
The Regulations establish a legal obligation to process personal data
How to process the data is subject to national law

13. Data collection and storage
The Commission will set up a backstopping contract to support Member States in their efforts to adapt their systems, if needed
EC recommends to verify as early as possible whether the data collection and storage systems are up to the task.

14. Implementation reports
The first AIR in 2016, covering 2014 and 2015
Simplified reports in 2016, 2018, 2020, 2021, 2022
Reports of a more strategic nature in 2017, 2019 & 2023
Annual data reported at IP level, cumulative values are calculated automatically
All data needs to be there in order for a report to be admissible

15. Evaluation
Evaluations shall be carried out to improve the quality of the design and implementation of programmes, as well as to assess their effectiveness, efficiency and impact
Ex-ante evaluation
carried out for each OP
has the main role of improving the quality of programming
submitted to the Commission at the same time as the OP

16. Evaluation plan
As a general rule: each OP should be covered by one evaluation plan
when a single MC covers more programmes, an evaluation plan should cover all programmes concerned
The evaluation plan should be approved by the MC
EC recommends that the plan is approved in the first or second MC meeting
The evaluation plan should be reviewed by the MC
EC recommends that the MC reviews the implementation of the plan at least once a year

17. Evaluation during the programming period
At east once during the programming period, an evaluation shall assess how support from the CSF Funds has contributed to the objectives for each priority
Evaluations can be horizontal – covering one or several programmes, priority axes, themes across programmes
Evaluations shall be carried out by experts who are functionally independent of the authorities responsible for programme implementation

18. Impact evaluation
MA can decide which methods or combination of methods could be used in order to capture the impacts of ESF interventions
two broad categories of impact evaluation are generally recognised: theory-based and counterfactual impact evaluations
DG EMPL will produce guidance on counterfactual impact evaluation
DG EMPL encourages MS to use the existing experience and further develop MA capacity to launch such evaluations

19. Summary report
Submitted to the Commission by December 2020 for each OP
Shall provide
a summary of the findings of evaluations carried out during the programming period
qualitative assessment of the main outputs and results

20. Ex post evaluation
Under the responsibility of the EC in close cooperation with the MS and MA
Shall be completed by 31 December 2023

21. Transparency
Evaluations and their follow-up shall be examined by the MC
All evaluations must be sent to the Commission in electronic format
All evaluations must be made public in their entirety