Post-secondary vocational training courses: are they effective for Italian unemployed youth with a high school diploma? Paolo Severati Scientific Coordinator of the Project and Head of Training Policies Evaluation Unit -ISFOL – Institute for the Development of Vocational Training for Workers ESF Evaluation Partnership Meeting Brussels, 13th November 2013

Outline General and specific objectives of the project The design of evaluation Governance of the project Expected results and their use Work programme

General and specific objectives of the project

Premise The project I will illustrate is a national shared project Many different actors participate in the project

The principal intent of this National Project To use the project to trigger a more-wide reaching effort to induce and enable regional ESF Managing Authorities to rigorously evaluate labour market policies

Who are the actors in this national project Ministry of Labour and Social Policies (applicant and provider of the co-financing of the project) ISFOL - Institute for the Development of Vocational Training for Workers (in-house research organization) ASVAPP & IRVAPP (subcontractors) Coordinamento delle Regioni (National Coordination Committee of the Italian Regions) 5 Regions: Piedmont, Trento, Veneto, Lazio, Apulia

Motivation Youth unemployment is at the top of the agenda of the Italian Government (youth unemployment rose to 32% on average in 2012, three times the overall rate) Robust and credible quantitative evidence on ‘what works and for whom’, in Italy is lacking, with a few exceptions While there is a strong presumption among policy-makers that financing job training programmes with public expenditure is necessary and useful, the empirical evidence is quite mixed. Whether training is cost-effective is a matter of dispute; and the very nature of training programmes makes such learning even more difficult.

The design of evaluation

Target population The study will focus on ‘post-diploma’ (PD) training courses targeted mainly to 20-29 year old unemployed individuals with a high school degree (diploma) This segment of the array of the training programmes co-financed by ESF in Italy is relatively homogenous respect to any other segment

The impact of the programme It will be estimated for two outcomes: the probability of obtaining a job with an open-ended contract at any point after the training; the number of weeks with a job in the following 12-month period after the programme; eventually a third outcome could be annual earnings (if data from the Italian Social Security Agency [INPS] will be available)

Geographical scope Five Regions: one from the North-West (Piedmont), two from the North-East (Trento and Veneto), one from the Centre (Lazio) and the last one from the South (Apulia) Trento is a small Region, the remaining four are large (only Apulia belongs to the Convergence Objective) 2007-2013 ESF Governance Total: 24 National and Regional Programmes Italy has 21 ESF Regional Managing Authorities (19 Regions and 2 Autonomous Provinces) 5 of them belong to the Convergence Objective (Basilicata, phasing out)

Time coverage PD courses started in 2007, 2008 and 2009 (from four to six year of coverage, assuming that outcome data until the end of 2013 will be available) PD courses last from less than one to two years Lock-in effect (generally absorbed after at most two years)

Table 1: Number of enrolees in post-secondary training courses, by year and region (source: Monit 2000-2006 and Monit 2007-2013  

Methodology Propensity score matching method(s) Balancing: past labour market history Large variety of matching procedures

Data sources, accessibility and content Three archives will be used and merged for the analysis: Regional archives on training policies (co-funded by ESF or not); the Public Employment Service Archives; the COB (Comunicazioni Obbligatorie) Archive, containing the information that every private employer or its agent is legally bound to communicate on-line to the COB archive in order to initiate, modify or terminate any work-related contract

The data archives needed for CIE of training programmes

Governance of the Project

Two different levels National Level Management and Methods Coordinating Committee (MMCC), five persons Scientific Committee (SC) Regional Level 5 Local Working Groups (LWG)

Tasks of the MMCC Maintaining and further developing a common approach to be followed by the 5 LWG; monitoring the work being conducted by the LWG synthesizing the findings; disseminating the results and lessons learned in using CIE

Scientific Committee It includes 6 of the best national experts in the fields of non-experimental evaluation methods It will provide comments and will meet through conference calls

Local Working Groups Members: Components of the project research team Regional managers and officials Tasks: sharing the methodology and the data following the whole evaluation process discussing the principal results emerging from the evaluation

Expected results and their use

Expected results Robust evidence on the impact of the ESF interventions Increased awareness and knowledge of counterfactual analysis Enhanced capacity to effectively implement counterfactual analysis

Dissemination and learning Production of accessible, non-technical writing on methods and the results; dissemination of this material to the staff on the Managing Authorities as well the top management of the regional governments, including member of the regional assemblies; training opportunities for the staff on Managing authorities

Work Programme

TASK Signing of the contract Kick-off Meeting in Italy Oct.-13 Nov.- 13 Dec.- 13 Jan.- 14 Feb.- 14 Mar.- 14 Apr.-14 May-14 Jun-14 July-14 Aug-14 Sept.-14 Oct.-14 Nov.-14 Dec.-14 Jan.-15 Feb.-15 TASK   Signing of the contract Kick-off Meeting in Italy M Meeting in Brussels with European Commission Institution of local working groups Data Acquisition and first round of impact estimates Mid-way assessments Second round of impact estimates Dissemination activities Final Report

Thank you for your attention Contacts: p.severati@isfol.it