Monitoring and Evaluation of Roma projects and policies, Brussels, 30/11/2010 Evaluating the European Social Fund support to Roma inclusion: processes, challenges and identified results of interventions Agnieszka Sternik European Commission DG Employment, Social Affairs and Equal Opportunities

Overview of the presentation  Evaluation objectives, scope and deliverables  Methodological challenges  Main findings of the evaluation in terms of inputs, outputs and results of the ESF interventions for Roma  Lessons learned and recommendations 1 Evaluation of ESF Support for Enhancing Access to the Labour Market and the Social Inclusion of Migrants and Ethnic Minorities

Evaluation objectives, scope and deliverables Evaluation questions: What are the intervention logic of support and types and scope of ESF measures? What are the effects in terms of outputs, results and impacts of ESF measures? What are the links between and programmes and complementarity of ESF with national and other EU funded interventions? What European added value can be attributed to ESF support? Scope: ESF in the and OP + EQUAL Community Initiative Programme 15 MS – representative sample: AT, DE, ES, FIN, FR, GR, HU, LT, IT, NL, PL, SE, SK, RO, UK Deliverables: EU-level synthesis report Roma thematic report 2

Methodological challenges Definitional differences between MS Difficulties in obtaining reliable baseline data: demographics, social and labour market indicators Difficulties in obtaining quantitative data for inputs, outputs and results of ESF interventions –data on participation disaggregated by target group ( ) –Annex XXIII of the ESF Implementing Regulations ( ) –limited experience in survey-based follow-up of beneficiaries Diversity of interventions targeting Roma: ‘specific actions’ vs ‘general measures’ Measuring of ‘soft outcomes’ not integrated into monitoring and evaluation of the ESF interventions 4

Main findings: inputs and outputs (quantitative) For the sample of 15 countries: More attention was given to the socio-economic integration of the Roma in compared with More explicit targeting of Roma in through specific actions than in , in particular by addressing needs of sub-groups, e.g. Roma women Increase in % of ESF funds allocated to measures targeting Roma (exclusively or among other disadvantaged groups) in compared with Increase in Roma participation rates in compared with , in particular in Roma specific measures 6

Main findings: outputs (qualitative) Most common types of interventions targeted at Roma: Pre-training and integration support - support to advise on different training and employment possibilities, obtaining a driving licence, social skills and initial job search Employability – individual training and employment plans, career counselling and advisory services, job placements, language learning Social inclusion - training social workers within Roma communities, measures to strengthen mutual trust and cooperation between the Roma and public institutions Education - projects to dismantle segregated schooling, classroom support to Roma children in mainstream schools, measures to reduce the no. of school drop-outs Training and lifelong learning – language skills and basic IT skills, training measures requiring specialisation (e.g. arts and crafts, housekeeping and domestic work, finance and accounting) Entrepreneurship – measures to encourage micro enterprise creation, creating self- employment opportunities to allow the Roma to capitalise on their existing skills e.g. small arts and crafts Measures to combat discrimination – awareness-raising about discrimination, information about Roma culture and language

Main findings: results (qualitative) ESF interventions targeting Roma contributed to: Improving individual skills, qualification and motivation of Roma participants Improving cooperation and partnership arrangements at national, regional and local levels Strengthening capacity building among public institutions working on Roma issues Developing of integrated approaches to Roma integration Innovation and developing new instruments to tackle Roma issues (EQUAL CI) Strengthened awareness about discrimination against the Roma Synergies with other EU funds Despite ESF interventions: Persistent disparities in employment and educational attainment between the majority population and Roma Limited sustainability of project outcomes Insufficient focus on capacity building of Roma organisations 9

Lessons learned and recommendations Horizontal issues: Ensure involvement of Roma organisations at all stages of ESF programming and implementation cycle Ensure reliable monitoring of inputs, outputs and results, including ‘soft outcomes’, of the ESF interventions targeting Roma Programming level: Establish detailed baseline of the Roma situation and needs Ensure synergy between the ESF funded interventions and national programmes targeting Roma Strike balance between general measures (equality mainstreaming) and specific actions for Roma in the ESF OPs Apply in the ESF interventions an integrated approach to tackling the multiple factors linked to the social exclusion of Roma Promote concept ‘explicit but not exclusive’ targeting in the ESF interventions 17

Implementation level: Support creation of new networks and organisational structures at national, regional and local levels Involve Roma organisations in management and implementation arrangements Continue strengthening the capacity of employment and social services to identify individual needs of Roma and better tailor the ESF support Ensure phase-out for projects targeting Roma to strengthen their sustainability and transfer of lesson learned Beneficiaries'’ level: Involve Roma organisations in project design and delivery Use global grants approach, which reduces administrative burdens and facilitates access of grassroots organisations to the ESF Pursue capacity building for Roma organisations in ESF implementation, in particular for monitoring and evaluation Involve Roma participants in implementation of ESF projects Lessons learned and recommendations

Thank you for your attention Contact: