ESF EVALUATION PARTNERSHIP MEETING – 15 MARCH 2013 ESF EXPERT EVALUATION NETWORK: SOCIAL INCLUSION KEY FINDINGS AND ISSUES ISABEL NAYLON AND ALAN MCGREGOR ESF EVALUATION PARTNERSHIP MEETING – 15 MARCH 2013

OVERVIEW The ESF Expert Evaluation Network was set up in 2010 with the objective of collecting, analysing, synthesising and aggregating evaluation studies carried out in the Member States The network produces: an inventory of evaluations in each Member State (693 end 2012) 2 country reports a year per Member State two synthesis reports a year ad hoc notes on request 2 network meetings a year The structure of the network is a management team consisting of the Training and Employment Research Unit of the University of Glasgow and Metis GmbH, a core team of six experts with different fields of expertise, 27 national experts (one in each Member State) Themes for country and synthesis reports are given by the Commission in consultation with the network based on the amount of evaluation material available and on strategic importance

OVERVIEW The themes so far have been: all Policy Fields/Priority Axes of the ESF Regulations, Access to Employment, Social Inclusion, and Young People and Women The final theme this year will be an update of all Policy Fields/Priority Axes across all Member States Process: Template for country reports developed by core team Expert network meeting to brief experts Experts write reports on the basis of AIRs, evaluations and relevant studies Synthesis reports on the basis of the 27 country reports (this time ten long and 17 short reports) Result: Overview of evaluation evidence in the Member States on the given theme Presentations at Evaluation Partnership Meetings, today Social Inclusion

QUANTITATIVE FINDINGS Across 27 Member States, Social Inclusion major part of ESF activity Almost €24 billion committed Public Sector Cost Over 14.5 million final recipients supported, but some multiple counting Of final recipients: Over 2.4 million unemployed Over 1.3 million migrants Over 1 million young people Under-estimates due to data deficiencies Despite substantial activity relatively little information on results – but 330,000 employment results based on 10 Member States

INTERVENTION TYPES Prevention Complementary activities Activities targeted at specific groups Confronting discrimination Providing supported employment Access to employment Integrated pathways

TARGET GROUPS People with disabilities – all Member States Young people – all but 1 Member State Migrants – all but 2 Member States Unemployed and long term unemployed – sometimes focussed on specific groups for SI activities Economically inactive Less frequently mentioned but important Roma and other minorities Offenders and ex-offenders People with drug and alcohol misuse issues Early school leavers

TARGET GROUPS (CONT) Relatively infrequently mentioned Low skilled at risk of social exclusion People with mental health issues

COMMUNITY ADDED VALUE (CAV) Limited evaluation evidence on CAV but qualitative assessments CAV volume effects reported for many countries, with ESF helping to maintain significant SI activity with demand growing and national budgets declining CAV scope effects reported in a number of countries principally in relation to support for SI activity for specific groups that would otherwise receive only limited activity CAV role effects were identified in some countries in terms of promoting innovation in SI such as more integrated service delivery CAV process effects included more rigorous approaches to target setting and monitoring, for example, adopted by Member State agencies

IMPLEMENTATION CONSTRAINTS Impact of economic crisis on scale of problem and available employment opportunities Management and administrative issues associated with ESF generally Problems with design and delivery of interventions Difficulty in engaging social inclusion target groups Challenges of partnership working Organisational capacity issues

SUPPORTING IMPLEMENTATION Finding effective recipient engagement mechanisms Integrating interventions and services Customising activities to individual needs Designing interventions that tackle key issues Intervening at an early stage Responding to changing problems Creating effective organisations Building on local strengths

GOOD PRACTICE Integrated approaches Effective engagement Early intervention – but not enough examples Complementary activities Employment and training interventions Raising awareness – but not enough examples Changing attitudes and behaviours – but not enough examples

POSITIVE MESSAGES High level of SI activity reaching large number of final recipients Good targeting on more disadvantaged groups already socially excluded or at risk of social exclusion Evidence for integrated pathways in 26 Member States – central design requirement for SI interventions Some – but limited – evaluation evidence that identifies effective interventions for specific groups (e.g. those at risk of leaving school early) Consistent qualitative evidence, backed up by some evaluation evidence, of CAV associated with SI activity Qualitative evidence, with some evaluation underpinnings, identified factors supporting effective implementation, plus good practice

SOME CHALLENGES On monitoring and evaluation evidence base: Country experts struggled to assemble simple monitoring data partly due to SI being horizontal priority Despite importance of SI, evaluation evidence often fragmented and lacking robustness There is very limited evidence on hard results There is very limited evidence on soft results – ‘distance travelled’ Given amount of budget deployed and need to tackle social exclusion effectively – we need to know what is happening Very limited evidence on ‘what works for which target group’ – we need to know more to design and re-design interventions Counterfactual evaluations difficult for SI – but desperately needed SI effort heavily concentrated in a small number of Member States: But growing problem across Member States Potential long term costs for economies and societies if not tackled

SOME CHALLENGES (CONT) Many Member States now see employment as key to tackling social exclusion – but report suggests job entry for only around 6% of final recipients Need to do better on this or Do a reality check on this policy approach Limited evidence on nature and effectiveness of work with employers (and other organisations) – whose direct or indirect discrimination key causal factor in social exclusion Changing attitudes and behaviour requires significant investment ..... but legacy effects and long term returns can make this pay