1. Post-secondary vocational training courses: are they effective for Italian unemployed youth with a high school diploma?
Paolo Severati
Scientific Coordinator of the Project and
Head of Training Policies Evaluation Unit -ISFOL – Institute for the Development of Vocational Training for Workers
ESF Evaluation Partnership Meeting Brussels, 13th November 2013

2. Outline General and specific objectives of the project
The design of evaluation
Governance of the project
Expected results and their use
Work programme

3. General and specific objectives of the project

4. Premise The project I will illustrate is a national shared project
Many different actors participate in the project

5. The principal intent of this National Project
To use the project to trigger a more-wide reaching effort to induce and enable regional ESF Managing Authorities to rigorously evaluate labour market policies

6. Who are the actors in this national project
Ministry of Labour and Social Policies (applicant and provider of the co-financing of the project)
ISFOL - Institute for the Development of Vocational Training for Workers (in-house research organization)
ASVAPP & IRVAPP (subcontractors)
Coordinamento delle Regioni (National Coordination Committee of the Italian Regions)
5 Regions: Piedmont, Trento, Veneto, Lazio, Apulia

7. Motivation
Youth unemployment is at the top of the agenda of the Italian Government (youth unemployment rose to 32% on average in 2012, three times the overall rate)
Robust and credible quantitative evidence on ‘what works and for whom’, in Italy is lacking, with a few exceptions
While there is a strong presumption among policy-makers that financing job training programmes with public expenditure is necessary and useful, the empirical evidence is quite mixed. Whether training is cost-effective is a matter of dispute; and the very nature of training programmes makes such learning even more difficult.

8. The design of evaluation

9. Target population
The study will focus on ‘post-diploma’ (PD) training courses targeted mainly to year old unemployed individuals with a high school degree (diploma)
This segment of the array of the training programmes co-financed by ESF in Italy is relatively homogenous respect to any other segment

10. The impact of the programme
It will be estimated for two outcomes:
the probability of obtaining a job with an open-ended contract at any point after the training;
the number of weeks with a job in the following 12-month period after the programme;
eventually a third outcome could be annual earnings (if data from the Italian Social Security Agency [INPS] will be available)

11. Geographical scope
Five Regions: one from the North-West (Piedmont), two from the North-East (Trento and Veneto), one from the Centre (Lazio) and the last one from the South (Apulia)
Trento is a small Region, the remaining four are large (only Apulia belongs to the Convergence Objective)
ESF Governance
Total: 24 National and Regional Programmes
Italy has 21 ESF Regional Managing Authorities (19 Regions and 2 Autonomous Provinces)
5 of them belong to the Convergence Objective (Basilicata, phasing out)

12. Time coverage
PD courses started in 2007, 2008 and 2009 (from four to six year of coverage, assuming that outcome data until the end of 2013 will be available)
PD courses last from less than one to two years
Lock-in effect (generally absorbed after at most two years)

13. Table 1: Number of enrolees in post-secondary training courses, by year and region (source: Monit and Monit  

14. Methodology Propensity score matching method(s)
Balancing: past labour market history
Large variety of matching procedures

15. Data sources, accessibility and content
Three archives will be used and merged for the analysis:
Regional archives on training policies (co-funded by ESF or not);
the Public Employment Service Archives;
the COB (Comunicazioni Obbligatorie) Archive, containing the information that every private employer or its agent is legally bound to communicate on-line to the COB archive in order to initiate, modify or terminate any work-related contract

16. The data archives needed for CIE of training programmes

17. Governance of the Project

18. Two different levels National Level
Management and Methods Coordinating Committee (MMCC), five persons
Scientific Committee (SC)
Regional Level
5 Local Working Groups (LWG)

19. Tasks of the MMCC
Maintaining and further developing a common approach to be followed by the 5 LWG;
monitoring the work being conducted by the LWG
synthesizing the findings;
disseminating the results and lessons learned in using CIE

20. Scientific Committee
It includes 6 of the best national experts in the fields of non-experimental evaluation methods
It will provide comments and will meet through conference calls

21. Local Working Groups Members: Components of the project research team
Regional managers and officials
Tasks:
sharing the methodology and the data
following the whole evaluation process
discussing the principal results emerging from the evaluation

22. Expected results and their use

23. Expected results
Robust evidence on the impact of the ESF interventions
Increased awareness and knowledge of counterfactual analysis
Enhanced capacity to effectively implement counterfactual analysis

24. Dissemination and learning
Production of accessible, non-technical writing on methods and the results;
dissemination of this material to the staff on the Managing Authorities as well the top management of the regional governments, including member of the regional assemblies;
training opportunities for the staff on Managing authorities

25. Work Programme

26. TASK Signing of the contract Kick-off Meeting in Italy
Oct.-13
Nov.- 13
Dec.- 13
Jan.- 14
Feb.- 14
Mar.- 14
Apr.-14
May-14
Jun-14
July-14
Aug-14
Sept.-14
Oct.-14
Nov.-14
Dec.-14
Jan.-15
Feb.-15
TASK
Signing of the contract
Kick-off Meeting in Italy M
Meeting in Brussels with European Commission
Institution of local working groups
Data Acquisition and first round of impact estimates
Mid-way assessments
Second round of impact estimates
Dissemination activities
Final Report

27. Thank you for your attention Contacts: p.severati@isfol.it