1. ESF EVALUATION PARTNERSHIP MEETING – 15 MARCH 2013
ESF EXPERT EVALUATION NETWORK: SOCIAL INCLUSION KEY FINDINGS AND ISSUES
ISABEL NAYLON AND ALAN MCGREGOR
ESF EVALUATION PARTNERSHIP MEETING – 15 MARCH 2013

2. OVERVIEW
The ESF Expert Evaluation Network was set up in 2010 with the objective of collecting, analysing, synthesising and aggregating evaluation studies carried out in the Member States
The network produces:
an inventory of evaluations in each Member State (693 end 2012)
2 country reports a year per Member State
two synthesis reports a year
ad hoc notes on request
2 network meetings a year
The structure of the network is a management team consisting of the Training and Employment Research Unit of the University of Glasgow and Metis GmbH, a core team of six experts with different fields of expertise, 27 national experts (one in each Member State)
Themes for country and synthesis reports are given by the Commission in consultation with the network based on the amount of evaluation material available and on strategic importance

3. OVERVIEW
The themes so far have been: all Policy Fields/Priority Axes of the ESF Regulations, Access to Employment, Social Inclusion, and Young People and Women
The final theme this year will be an update of all Policy Fields/Priority Axes across all Member States
Process:
Template for country reports developed by core team
Expert network meeting to brief experts
Experts write reports on the basis of AIRs, evaluations and relevant studies
Synthesis reports on the basis of the 27 country reports (this time ten long and 17 short reports)
Result: Overview of evaluation evidence in the Member States on the given theme
Presentations at Evaluation Partnership Meetings, today Social Inclusion

4. QUANTITATIVE FINDINGS
Across 27 Member States, Social Inclusion major part of ESF activity
Almost €24 billion committed Public Sector Cost
Over 14.5 million final recipients supported, but some multiple counting
Of final recipients:
Over 2.4 million unemployed
Over 1.3 million migrants
Over 1 million young people
Under-estimates due to data deficiencies
Despite substantial activity relatively little information on results – but 330,000 employment results based on 10 Member States

5. INTERVENTION TYPES
Prevention
Complementary activities
Activities targeted at specific groups
Confronting discrimination
Providing supported employment
Access to employment
Integrated pathways

6. TARGET GROUPS
People with disabilities – all Member States
Young people – all but 1 Member State
Migrants – all but 2 Member States
Unemployed and long term unemployed – sometimes focussed on specific groups for SI activities
Economically inactive
Less frequently mentioned but important
Roma and other minorities
Offenders and ex-offenders
People with drug and alcohol misuse issues
Early school leavers

7. TARGET GROUPS (CONT)
Relatively infrequently mentioned
Low skilled at risk of social exclusion
People with mental health issues

8. COMMUNITY ADDED VALUE (CAV)
Limited evaluation evidence on CAV but qualitative assessments
CAV volume effects reported for many countries, with ESF helping to maintain significant SI activity with demand growing and national budgets declining
CAV scope effects reported in a number of countries principally in relation to support for SI activity for specific groups that would otherwise receive only limited activity
CAV role effects were identified in some countries in terms of promoting innovation in SI such as more integrated service delivery
CAV process effects included more rigorous approaches to target setting and monitoring, for example, adopted by Member State agencies

9. IMPLEMENTATION CONSTRAINTS
Impact of economic crisis on scale of problem and available employment opportunities
Management and administrative issues associated with ESF generally
Problems with design and delivery of interventions
Difficulty in engaging social inclusion target groups
Challenges of partnership working
Organisational capacity issues

10. SUPPORTING IMPLEMENTATION
Finding effective recipient engagement mechanisms
Integrating interventions and services
Customising activities to individual needs
Designing interventions that tackle key issues
Intervening at an early stage
Responding to changing problems
Creating effective organisations
Building on local strengths

11. GOOD PRACTICE
Integrated approaches
Effective engagement
Early intervention – but not enough examples
Complementary activities
Employment and training interventions
Raising awareness – but not enough examples
Changing attitudes and behaviours – but not enough examples

12. POSITIVE MESSAGES
High level of SI activity reaching large number of final recipients
Good targeting on more disadvantaged groups already socially excluded or at risk of social exclusion
Evidence for integrated pathways in 26 Member States – central design requirement for SI interventions
Some – but limited – evaluation evidence that identifies effective interventions for specific groups (e.g. those at risk of leaving school early)
Consistent qualitative evidence, backed up by some evaluation evidence, of CAV associated with SI activity
Qualitative evidence, with some evaluation underpinnings, identified factors supporting effective implementation, plus good practice

13. SOME CHALLENGES
On monitoring and evaluation evidence base:
Country experts struggled to assemble simple monitoring data partly due to SI being horizontal priority
Despite importance of SI, evaluation evidence often fragmented and lacking robustness
There is very limited evidence on hard results
There is very limited evidence on soft results – ‘distance travelled’
Given amount of budget deployed and need to tackle social exclusion effectively – we need to know what is happening
Very limited evidence on ‘what works for which target group’ – we need to know more to design and re-design interventions
Counterfactual evaluations difficult for SI – but desperately needed
SI effort heavily concentrated in a small number of Member States:
But growing problem across Member States
Potential long term costs for economies and societies if not tackled

14. SOME CHALLENGES (CONT)
Many Member States now see employment as key to tackling social exclusion – but report suggests job entry for only around 6% of final recipients
Need to do better on this or
Do a reality check on this policy approach
Limited evidence on nature and effectiveness of work with employers (and other organisations) – whose direct or indirect discrimination key causal factor in social exclusion
Changing attitudes and behaviour requires significant investment
..... but legacy effects and long term returns can make this pay