1. European Social Fund Evaluation in Italy Stefano Volpi Roma, 03 maggio 2011 Isfol Esf Evaluation Unit Human Resources Policies Evaluation Area Rome, Corso D’Italia 33

2. ISFOL is a research institute on human resources development counselling the Italian Ministry of Labour The Human Resource Policies Evaluation Area deals with policy and programme evaluation in the vocational training, labour, educational and social inclusion sectors. Inside this Area, the National European Social Fund (ESF) Evaluation Unit represents the core of the National Monitoring and Evaluation system for the assessment of the European Social Fund programme in Italy. It has been analysing and assessing the effects and added value of the ESF co-financed policies in Italy since 1995. The main objective of the evaluation activities is to provide ongoing assistance to national and regional policy makers in their choices for developing human capital, providing continual information on the main effects produced by the co-financed interventions and on the factors that could influence them. ISFOL National ESF Evaluation Unit: structure and goals

3. Approach to Evaluation Our Evaluation Strategy is founded: on a multidimensional approach…. 1. Multilevel: different levels of programming – central and local 1. Multifocus: input, process, output and outcomes 1. Multidisciplinary: different disciplinary fields 1. Participatory: involvement of stakeholders and policy makers

4. … based on three key tools… 1. A central/local monitoring system 2. A common set of indicators 3. A classification system for policy- based interventions Approach to Evaluation … entailing a three- stage process 1. Preparatory actions for constructing the monitoring and evaluation system 2. Implementing specific evaluations 3. Actions for disseminating an evaluation culture

5. The activities undertaken The National ESF Evaluation Unit operates at two levels: STRATEGIC  defining common standards to produce coherent physical indicators at central and local level  creating basic knowledge at central and local level to monitor programmes and to assess the effects generated by their implementation  defining evaluation and monitoring methods on some Esf priorities  developing and disseminating the evaluation culture on a national and European level OPERATIONAL  implementation of the National Evaluation System ex italian QSN  implementation of the National Monitoring System managed by the Ministry of Economy  managing National Working Groups with italian Regions on Esf priorities, such as physical monitoring of Esf interventions, evaluation of occupational impacts, quality assessment for the vocational training system  evaluation of Esf interventions to support workers and enterprises  regular dissemination of the main results of the acrivities undertaken (both for policy makers and public opinion)

6. Area’s activities in ESF 2000-2006 Evaluation of the European Social Fund Mid-term and final evaluation of the Objective 3 Community Support Framework (Northern and Central Italy) Evaluation of occupational outcomes of ESF-supported interventions for preventing and remedying unemployment Impact assessment of ESF-supported training for workers Evaluation of interventions for promoting equal opportunities between men and women Evaluation of quality of the vocational training system Evaluative survey on outcomes of higher training courses in Southern Italy Analysis of progress of ESF cofinanced policies nationwide Monitoring of public calls for tender and announcements issued by central, regional and provincial administrations Participation in the National Evaluation System promoted by the Ministry of the Economy and Finance (southern Italy)

7. SOME RESULTS OF FINAL EVALUATION CSF OB. 3 Differences among beneficiaries of the interventions Gender Equal proportion of men and women participating in the interventions Less women participated in the social inclusion policies and in the policies for fostering the competitiveness of employers and employees. Other features of the target-groups (age, level of education, occupational status) The European Social Fund gives opportunities to different users, such as: People less than 19 years old to over 44 People with a low level of education Students, unemployed and workers People over 44 participate less

8. SOME RESULTS OF FINAL EVALUATION CSF OB. 3: ACHIEVEMENT OF STRATEGICAL PRIORITIES Achievement of equal opportunities and social equity (target analysis) European Social Fund interventions have benefited 16% of people looking for a job and 24% of young people risking dropping out (coverage rate) 66 % of people (and 63 % of women) who participated in ESF interventions are employed in the labour market after one year from the end of the interventions (analysis of placement) Increased job opportunities for people involved in ESF interventions (first impacts)

9. Main results at system level  Operational evaluation system at both a central and local level  Evaluation system tailored to policies and effects to be considered (input, output, outcomes, impacts)  Evaluation activities based on the partnership principle (national working groups)  Promotion of an evaluation culture among all actors involved

10. ESF Programming Period 2007-2013 The National ESF Evaluation Unit participates in national working groups dealing with monitoring and evaluation: the National Evaluation System as indicated in the National Strategic Framework (QSN), common indicators, ESF monitoring, Evaluation design and ESF Permanent Table, etc. The National Operational Programme (PON): - puts much emphasis on monitoring and evaluation of ESF as well as of all human resources development policies. - entrusts ISFOL with the technical coordination and methodological proposals and tools in the field of monitoring and evaluation as well as evaluation activities of ESF cofinanced interventions.

11. Evaluation functions during the 2007-2013 period There are no rigid deadlines for evaluations and it is not necessary to evaluate all OPs. On-going evaluations will be carried out according to the Managing Authorities and the European Commission needs. The two main functions are: Strategic: evaluating the contribution of OP’s to reaching the social and economic cohesion objectives (policy objectives) Operational: “evaluation linked to monitoring”, when monitoring systems will identify problems.

12. The National Evaluation System according to the QSN In the paragraph on “Organisation of the methodological support to the quality of the evaluation process” the QSN states that… The National Evaluation System (NES) towards a joint regional policy:  supports the quality of the evaluation processes in order to build an evaluation capacity and to held disseminate an evaluation culture;  offers guidelines on the evaluation and self-evaluation organisation and processes, on the evaluation plans and on the dissemination of results through meetings, debates, and document production;  supports the definition, revision and implementation of the evaluation plans and sustains the evaluation and self-evaluation processes upon request of the administrations and of their institutional or social partners;  monitors the evaluation processes and reports on the results of those activities on a regular basis;  promotes, commits, carries out and pursues studies on the evaluation themes;  stimulates an high level debate between actors responsible for regional policies and for their evaluations together with other actors (public, private and non governmental) operating in the national and international debate on evaluation and development.

13. Thanks for your attention Stefano Volpi Isfol – Human Resources Policies Evaluation Area Tel. +39 06 85447216 e-mail s.volpi@isfol.it