1. ESF Expert Evaluation Network Evaluation Partnership Meeting Herta Tödtling-Schönhofer & Isabel Naylon

2. Objective
To set up an Expert Evaluation Network with one national expert in each Member State which collects, describes and analyses all evaluations related to ESF Operational Programmes carried out by the Member States in the programming period
Main task
To produce Inventories of evaluations and Country Reports for all 27 Member States and a Synthesis Report at EU level summarising the evaluation findings

3. 1. Evaluations

4. Evaluations
By July 2011, national experts had identified 559 evaluations, whereby distribution is very uneven across the Union
Ongoing evaluations have been included in the inventories but not in the analysis
Only finalised evaluations (final reports) were screened for findings except in the case where interim evaluations had the character of final evaluations (were on specific measures, would not be updated etc.)
Most evaluations are carried out at OP level

5. Evaluation methodologies
In keeping with the aim of the evaluations (operational),
55% were exclusively process evaluations,
18% exclusively impact,
10% were both impact and process,
17% were other
122 were impact evaluations (of which 44 were both impact and process); of these
45 were theory based
26 used a counterfactual approach
24 were classified as ‘other’ and
27 as unclear
10 MS have used the counterfactual approach so far
Most evaluations carried out at OP level

6. Key figures 112 Evaluation Plans 559 Evaluations … of which
117 Planned
84 Ongoing
 330 Finalised
 28 Unclear
233 Process evaluations (exclusively)
78 Impact evaluations (exclusively)
139 Strategic evaluations
307 Operational evaluations
Total findings 1651
Methodology
Approach

7. Evaluations by Member State

8. 2. Evaluation Findings sorted by…..

9. Findings by evaluation criteria

10. Findings by intervention type

11. Findings by target group

12. Findings by individual target groups

13. Findings by policy field

14. 3. Evaluation Findings are telling us….

15. a) Increased adaptability /1
Findings focus on the:
establishment of new enterprises
growth of SMEs
organizational learning, global management quality
development of innovation
The findings concern
employed people (40%)
self-employed people (22%),
older people (10%),
unemployed (8%),
other disadvantaged groups (9%)
and to a lesser extent inactive people, young people, women and people with disabilities

16. Increased adaptability /2
Support is provided through:
training activities (65% of all findings)
start-up incentives (10%) and
education (9%)
and to a lesser extent through LM services, employment incentives and social inclusion specific support, etc.
Evaluations provide a positive assessment of the relevance and effectiveness of ESF interventions in this field, in particular in the context of the financial crisis
The ESF fosters the development of innovative and more productive forms of work

17. b) Enhancing access to employment /1
Findings mostly concerned with unemployed people (31%) and young people (20%),
to a much lesser extent self-employed, inactive and older people, people with disabilities,
Few evaluations to date focus on migrants and Roma
The financial crisis has increased the relevance of ESF interventions in this field but has also hampered their effectiveness, in particular for the most disadvantaged target groups

18. Enhancing access to employment /2
The evidence concerning the modernisation and strengthening of labour market institutions is rather mixed
The most effective interventions for the unemployed are close to the labour market and have employer involvement
Proximity to the users is a success factor
Barriers to participation remain, in particular for target groups most distanced from the labour market
Actions fostering gender mainstreaming seem to be successful in raising women’s participation rates

19. c) Reinforcing social inclusion
Some target groups are not reached by the programme despite targeting by the ESF due to ‘creaming off’ strategies
Evaluations so far show that success factors in the field of social integration include:
integrated strategies across policy sectors (health, social affairs, education)
Local interventions and partnerships at local level
Personalised services
The effectiveness of social inclusion measures ranges from economic returns to the individual well-being of participants

20. d) Improving human capital
The ESF interventions contribute to reforms in education and training systems
The findings indicate good progress in lifelong learning and training, in particular in the introduction of new products in enterprises
In particular in the light of the financial crisis are measures fostering the development of human capital found to be relevant

21. e) Partnerships and networking
The evaluations so far hardly deal with the promotion of partnerships and networking (only 2% of the findings address this policy field)
Two main strands emerge when analysing the results attributed to partnerships across Member States:
Partnerships have an important function in increasing awareness of the local dimension
Partnerships and networking allow diversification across fields and actors involved
A number of challenges are identified in the capacity of Member States to develop partnerships, e.g.: concerning the involvement of the private sector and the development of clear goals

22. f) Strengthening institutional capacity
According to the Regulation 1081/2006, the ESF supports actions to strengthen institutional capacity by promoting mechanisms for improving good policy and programme design, monitoring and evaluation at national, regional and local level, capacity building in the delivery of policies and programmes
Evaluations looked at both the administrative capacity actions and at weaknesses in administrative capacity in the implementation of Operational Programmes
This lead to some confusion as to which findings strictly belong to the policy field and which do not but have been put here for want of a better category

23. Strengthening institutional capacity /2
Where it was clear that the findings were not linked to the Policy Field ‘Strengthening institutional capacity’, they were allocated to ‘All Policy Fields’ – again for want of a better solution
Findings that can be linked to this policy field stem to a large extent from evaluations implemented in Poland (121 evaluations and 235 findings attributed to this policy field) due to the administrative reforms introduced in Poland

24. 4. Synthesis and ConcLusions….

25. Conclusions /1
The manner in which Member States have used the flexibility of the new regulations and the evaluation approach they have adopted vary considerably
Evaluations in some Member States, especially in the EU12, tend to focus on implementation and operational aspects
The sum total of the evaluations at the end of the period may not give a coherent overall picture of the effects and effectiveness of the ESF across the EU
Evaluations in the current period may tend to focus more on the deficiencies of programmes co-funded by the ESF than on the benefits of the ESF as such

26. Conclusions /2
The ESF is a useful financial instrument, especially for those Member States in which the ESF represents a significant proportion of funding for LMP and in the current context of economic downturn
The ESF contributes to the adaptability of workers to the labour market mainly through training activities
The ESF contributes to enhance access to employment for the unemployed and young people but disadvantaged groups still face barriers to participation

27. Conclusions /3
The ESF contributes to social inclusion but effects on the most disadvantaged groups are not yet evaluated in all the Member States
The ESF contributes to public administration reforms in Convergence Member States and regions
The contribution of partnerships and networking to the improvement of reforms and labour market policies in the MS is little evaluated
The ESF supports the MS in addressing horizontal priorities (gender mainstreaming, sustainable development, and transnationality) but these have hardly been evaluated so far

28. Conclusions /4
CAV and sustainability of ESF interventions have not yet been evaluated
There are still weaknesses in the ESF monitoring and evaluation:
link between monitoring and evaluation
data quality
data collection methods
information about beneficiaries
indicator systems with target levels
link between projects and OP

29. Thank You Benno Savioli, Feedback Bremen
Herta Toedtling Schoenhofer & Isabel Naylon, Metis GmbH
Hélène Génin & Irene Basile, Euréval
Benno Savioli, Feedback Bremen