ESF monitoring and evaluation in 2014-2020 Draft guidance Directorate General Employment, Social Affairs and Inclusion Evaluation and Impact Assessment Unit Brussels 16 March 2012

Purpose of the presentation Present the draft guidance on monitoring and evaluation of the ESF by focusing on new elements compared to current situation Create a common understanding around certain concepts Caution: the guidance remains a draft and will be revised in order to reflect the final Regulation

Programming Link to Europe 2020 is essential Intervention logic to address challenges from NRP, country specific recommendations Drafting the strategy shall not start from a blank sheet of paper but shall be based on previous analytical work in the context of Europe 2020

Monitoring Monitoring has two sides: 1. Continuous and systematic process of generating quantitative data on implementation The guidance will focus on various aspects related to generating these data 2. Examination and discussion of these data EC recommends to regularly present to and discuss with MC updated monitoring data.

Indicators Financial, output and result indicators are possible Reported at the level of investment priorities Indicators relate to partially and fully implemented operations Indicators should be clearly defined, the chosen measurement unit shall be indicated and they shall be periodically measured.

Common indicators Capture output and results Monitoring shall always use all common indicators although they might report zeros.

Common output indicators Relate to persons (participants) and entities A participant is reported ONLY when ALL personal data is recorded, i.e. gender, employment status, age – as applicable – education One person can be counted more than ones BUT only when this person benefitted from the support of a separate operation Only entering participants are reported, no leaving EC recommends to enter the data as early as possible

Common immediate result indicator Relate to persons only Collected through MI system Certain combinations of immediate result indicators for one person are possible

Common longer-term result indicators Data reported only in 2019 and 2023 Data normally not collected through normal MI system Data collected based on a representative sample Representativeness established at the level of priority, socio-economic characteristics of participants as captured by output targets Representativeness also for the regional dimension Minimum: two non-overlapping samples

Programme-specific indicators For the Member States to establish Member States may want to set additional indicators which do not form part of the programme Indicators should be underpinned by clear and easy to understand definitions Definitions need to be made available from the start of programme implementation Beneficiaries need to be aware of the definitions

Baseline and targets Baselines for result indicators only Cumulative quantified target values for 2022 Targets expressed in absolute numbers Targets set at the level of IP Targets for a limited number of common and programme specific output and result indicators Targets translate a political intention for change in a certain direction into a quantified value

Data collection and storage All data to be transmitted electronically Member States shall provide a computerised system for recording, filing and transmission of data Processing of individual participants data The Regulations establish a legal obligation to process personal data How to process the data is subject to national law

Data collection and storage The Commission will set up a backstopping contract to support Member States in their efforts to adapt their systems, if needed EC recommends to verify as early as possible whether the data collection and storage systems are up to the task.

Implementation reports The first AIR in 2016, covering 2014 and 2015 Simplified reports in 2016, 2018, 2020, 2021, 2022 Reports of a more strategic nature in 2017, 2019 & 2023 Annual data reported at IP level, cumulative values are calculated automatically All data needs to be there in order for a report to be admissible

Evaluation Evaluations shall be carried out to improve the quality of the design and implementation of programmes, as well as to assess their effectiveness, efficiency and impact Ex-ante evaluation carried out for each OP has the main role of improving the quality of programming submitted to the Commission at the same time as the OP

Evaluation plan As a general rule: each OP should be covered by one evaluation plan when a single MC covers more programmes, an evaluation plan should cover all programmes concerned The evaluation plan should be approved by the MC EC recommends that the plan is approved in the first or second MC meeting The evaluation plan should be reviewed by the MC EC recommends that the MC reviews the implementation of the plan at least once a year

Evaluation during the programming period At east once during the programming period, an evaluation shall assess how support from the CSF Funds has contributed to the objectives for each priority Evaluations can be horizontal – covering one or several programmes, priority axes, themes across programmes Evaluations shall be carried out by experts who are functionally independent of the authorities responsible for programme implementation

Impact evaluation MA can decide which methods or combination of methods could be used in order to capture the impacts of ESF interventions two broad categories of impact evaluation are generally recognised: theory-based and counterfactual impact evaluations DG EMPL will produce guidance on counterfactual impact evaluation DG EMPL encourages MS to use the existing experience and further develop MA capacity to launch such evaluations

Summary report Submitted to the Commission by December 2020 for each OP Shall provide a summary of the findings of evaluations carried out during the programming period qualitative assessment of the main outputs and results

Ex post evaluation Under the responsibility of the EC in close cooperation with the MS and MA Shall be completed by 31 December 2023

Transparency Evaluations and their follow-up shall be examined by the MC All evaluations must be sent to the Commission in electronic format All evaluations must be made public in their entirety