1. Evaluation of ESF Support for Enhancing Access to the Labour Market and the Social Inclusion of ROMA
(May 2011, Centre for Strategy & Evaluation Services)
Directorate General Employment, Social Affairs and Inclusion
(Mrs Amparo Roca, unit A3)
21 March 2012 Joint REGIO-EMPL COCOF Technical Meeting

2. Purpose of the presentation: KEY FINDINGS of Roma Thematic Report
Evaluation objectives and scope. Methodological constraints Assessment of targeting approaches Outputs, results and impacts of the ESF interventions. Added value and complementarity of the ESF. Success factors and obstacles for effective ESF interventions. Conclusions and Recommendations.
KEY FINDINGS of Roma Thematic Report
Evaluation of ESF Support for Enhancing Access to the Labour Market and the Social Inclusion of MIGRANTS AND ETHNIC MINORITIES 2

3. Evaluation objectives
• Assess the ESF’s contribution to enhancing access to the labour market and the social inclusion of the Roma; EFFECTIVENESS, EFFICIENCY, RELEVANCE, SUSTAINABILITY • Identify the most effective means of tackling barriers to social/ LM integration of Roma; • Provide guidelines on the implementation of ESF OPs in the current period to target beneficiaries effectively; • Make recommendations looking ahead to the role of the ESF in promoting ROMA access to the LM post-2013 3

4. Evaluation questions:
What is the intervention logic of ESF support for ROMA?
What are the types and scope of ESF measures?
What are the outputs, results and impacts of ESF measures?
What are the links between and in objectives for ESF support ?
What European added value can be attributed to ESF support in implementing national and EU policies? 4

5. Evaluation SCOPE
ESF in the and programming periods
(data availabilitty until 2009!!)
AND…
EQUAL Community Initiative Programme
(role in promoting innovative approaches to ROMA inclusion)
15 countries – representative sample: AT, DE, ES, FIN, FR, GR, HU, LT, IT, NL, PL, SE, SK, RO, UK
Good balance in geographical coverage: old/new MS, size of ROMA population , types of ESF programmes, targeting approach -no specific measures in Germany, Sweden, the Netherlands and the UK-

6. Evaluation scope: 15 EU MS

7. Methodological constraints
LACK OF MONITORING AND EVALUATION FOCUS ON ROMA + LEGAL DIFFERENCES BETWEEN MS ON ROMA TREATMENT :
Difficulties in obtaining reliable baseline data: demographics, social and LM indicators.
Lack of accurate data on ESF funding and participation by ROMA (Annex XXIII of the ESF Implementing Regulations); non self-declaration of participants, legal constraints on ethnic data, targeting: specific actions vs. general measures)
Absence of results data (lack of aggregation at programme level of project results).
‘Soft outcomes’ – population too distant from LM integration –difficult to quantify progress
Methodology: extensive interview programme (220 stakeholders at EU and MS level)
(Some MS target ROMA explicitly through ‘specific actions’, others through ‘general measures’ in which multiple disadvantaged groups are addressed ) 7

8. Targeting ROMA
Specific actions – ESF measures solely focused on the Roma; ES, GR, HU, PL, RO, SK
‘Explicit but not exclusive’ targeting – measures that mainly address the needs of the Roma, but in which other disadvantaged groups may also be targeted ES, GR, IT, HU, RO, SK
General measures – in which the Roma are among the disadvantaged groups being targeted, but are not the primary focus of an ESF intervention; AT, FI, FR (only ROMA migrants)
Limited or no targeting of the Roma – (country with small Roma population, or because wishing to avoid ‘labelling’ particular ethnic groups) DE, NL, SE, UK
Most MS: combination of targeting approaches (although some MS only one (general measures, no targeting) PLUS Equality mainstreaming
. While there is growing attention to the Roma in many ESF programmes, not all countries target the Roma directly.
With regard to the merits of a mainstreaming approach to Roma inclusion versus promoting their integration through specific actions, the two are not mutually exclusive and both approaches may be needed. A key finding was that the specific needs of the Roma risk becoming overlooked within the general social inclusion Priorities of ESF Programmes unless there are also specific actions.

9. ESF Interventions for ROMA2007-2013: TARGETING APPROACH

10. KEY FINDINGS on TARGETING
ESF : Transition to an explicit but non inclusive targeting (often linked to territorial approach –e.g. HU rural areas, HU and SK micro-regions-)
But also Specific actions highly effective for enabling tailoring support for some ROMA problems: (SK: school desegregation of ROMA children; ES: early school leaving, personalized integration pathways).
It remains scope for strenghening ROMA mainstreaming in ESF general measures: need of resources, know how/instruments to implement and political commitment.
Integrated measures - effective in helping Roma minority. Need of long-term integrated approach to ensure effectiveness of ESF interventions (ACCEDER programme - ES).
The most effective use of ESF to promote social inclusion: combination of specific actions with mainstreaming

11. Financial inputs For the sample of 15 countries:
Increase in ESF funds to integrate ROMA in the labour market in compared with
: Expenditure ESF on specific ROMA actions estimated at €172m, only for the first 2 years of implementation, concentrated in 6 Member States (the Czech Republic, Lithuania*, Poland, Romania, Slovakia and Spain):
: Estimations for ESF: €228m targeted directly at the Roma through specific actions, with a further €806m through general measures that also targeted Roma.
The EQUAL Community Initiative provided funding for Roma integration: The BBI study estimated expenditure between €35m and €68m.
Additional EQUAL measures addressed at the Roma but also targeting other vulnerable groups between € 40m and € 87m
(Although expenditure on specific actions targeted at migrants and minorities has decreased, for ROMA the opposite holds).
Funding to promote equal opportunities varied in from 8% to 25% by MS
Some MS devoted considerable funding (FR, DE, NL, UK).
Approximately 20-25% EQUAL funds dedicated to measures promoting equal opportunities and combating racial discrimination
€26 billion (31%) out of €84 billion of total ESF expenditure in 11

12. Outputs
Participation in the ESF by ROMA people has increased between programming periods:
: 2.1m people with migrant or minority background received ESF support through specific actions(EU25) , among them approximately 100,000 people were ROMA participants. :
Expected increase to 4.2m with circa 600,000 participants with a migrant or minority background receiving ESF support each year (EU27).
Among them, in 2007 and 2008: 110,000 Roma benefited from ESF (targeted actions)
The number is undercounted due to reluctance of many ROMA participants to provide data on their ethnicity.
No reliable data for measures not exclusively addressed to ROMA.
Lack of comparability of participation data among MS

14. Results
Employability - language learning, improving qualifications, job placements/ apprenticeships, vocational training.
Personalised employment pathways versus ‘one size fits all’ approach: package of support
Pre-training and pre-integration support: job coaching, initial counseling/motivation. Ease access to mainstream support services
Basic literacy, numeracy and ICT skills . Informal learning : Moving ROMA closer to the labour market - Soft skills
Entrepreneurship micro-enterprise creation, self-employment, social economy
Education: early-school leaving, desegregation of ROMA schools, second chance schooling, youth transition to employment. Concern: SUSTAINABILITY
Social inclusion and community development: activation, cultural mediators, engagement with ROMA communities, key factor for success of other measures
Health: Integrated approach. GR: 31 Health-Social centres.
Combating discrimination: role of EQUAL, changing attitudes among employers
package of support. For example, many projects included a combination of career counselling and advisory services, job search assistance, vocational training, together with job placements.

15. Impacts: individual level
POSITIVE results in moving ROMA towards the labor market:
Strengthened access to the labour market , but training not always materialize in job opportunities (increase in LM participation rates).
no big impact on employment: 5-30% of ROMA get a job upon completion of an ESF intervention.
Basic skills improvement (literacy, ICT, national language)
Soft outcomes (self-confidence, communication skills, motivation for job search)
Improved access to education.
Upskilling and professional training.
Social inclusion; Access to services "one-stop shop"
HOWEVER: external barriers-persistent discrimination; spatial factors (isolation, rural vs. urban integration) undermine impacts.
(Personalised integration pathways appear to be very effective in promoting labour market access

16. Impacts: institutional/organisational
Educational reform e.g. desegregation of Roma school children
STRENGTHENED GOVERNANCE of ROMA support, in particular:
Strengthened partnership working: horizontal and vertical coordination mechanisms at different governance levels (BUT still improvements needed)
Greater capacity of public institutions to implement integrated approach to ROMA inclusion
Greater capacity of public employment services tailored to ROMA needs
Greater capacity of NGOs and self-organised ROMA to implement projects themselves and provide social support
Mainstreaming at programme level, but still room for improvement at implementation level.
Transfer/ adaptation of practices between countries - e.g. personalised employment pathways (ACCEDER)

17. Added value of ESF interventions
Volume effects - funding increase for Roma (EU12), overcome national funding limitations.
Scope effects –. Influence on national policy-making: Targeting (explicit but non exclusive, mainstreaming). Social economy: social enterprises work with ROMA.
Demonstration effects – raising attention about the needs of particular target groups, promoting partnership, stronger cooperation between governance levels, transnational learning.
Process effects – improving the way in which ROMA are addressed by public employment services; integration approach
The ESF has helped to achieve results difficult to achieve through national actions alone (funding limitations and lack of active LM programmes addressing structural barriers for ROMA. 17

18. Complementarity of ESF interventions
1. Complementarity with EU programmes:
ESF and ERDF - maximising SF impacts on the Roma; integrated approach (housing, infrastructure, education) reform of ERDF . Rural Development Fund (EAFRD)
Progress addressing statistical gaps, transnational learning and exchange of experiences -e.g. Peer Review on Roma integration (GR)-
New structures at regional/ national levels / improved coordination and partnership working arrangements
HOWEVER
Ensuring effective coordination across different funding programmes (e.g. ESF, ERDF, national programmes) requires adequate staffing, resources and political commitment. 18

19. Complementarity of ESF interventions
2. Complementarity and coordination with national programmes:
ESF broadly complementary with national programmes and social and employment policies.
EU coordinates policy framework through social and employment OMC.
Coordination should be improved between ESF and national programmes targeting ROMA
Strengthened visibility of ROMA in national employment and social inclusion programmes.
Develop a national strategy for Roma integration and ensure maximum use of the full potential of the ESF (most MS refer to structural funds in their NRIS -March 2012 report-)

20. Obstacles for effective ESF interventions
Programming:
Lack of involvement of NGOs in many EU countries
Lack of political commitment and continuity in policy approaches
Absence of baseline assessment and missing data
Lack of sufficient attention to equality mainstreaming
Implementation:
Lack of understanding among some PES / public authorities about training needs of ROMA and difficulties in accessing labour market
Capacity issues - NGOs and public authorities
Inadequate monitoring of achievements
Beneficiaries (project level):
External barriers are difficult to address through ESF e.g. societal attitudes and discrimination
Top-down approach without including professionals with a ROMA background in project delivery.
Final beneficiaries insufficiently involved 20

21. Success factors for effective ESF interventions
Programming:
Consult with NGOs as early as possible in programme design
Conduct detailed baseline of the needs of ROMA
Target through specific actions as well as equality mainstreaming
Promote concept of ‘explicit but not exclusive’ targeting
Implementation:
Involve NGOs closely in management and implementation arrangements
Tailor employment services to individual needs
Adopt integrated and long-term approach to tackling labour market obstacles
Adopt holistic approach to partnership working (engagement of Roma community)
Beneficiaries (project level):
Involve NGOs closely in project delivery
Involve ROMA themselves in implementing measures 21

22. Main conclusions (1)
Increased attention to promoting LM integration and social inclusion of ROMA compared with (expenditure, participation by the ROMA).
ESF has made a positive contribution to promoting Roma inclusion, HOWEVER the nature and scale of the results and longer-term impacts are difficult to assess (weak monitoring+soft outcomes)
In spite of the positive ESF contribution to increased labour participation, no substantial effect in reducing ROMA persistently high unemployment (40-90% -depending on MS-).

23. Main conclusions (2)
ESF has promoted a more integrated and long-term approach to the social and LM integration of ROMA (multiannual programming)
Partnership (vertical and horizontal) has been promoted - more holistic approach to tackling LM barriers involving multiple stakeholders.
ESF has strengthened employability and encouraged entrepreneurship among these target groups; HOWEVER lack of sustainability of job creation once the project is finished.
Discrimination remains a major problem, important to tackle "external barriers" to LM and social inclusion and to reinforce integrated approach.
Despite ESF interventions, persistent huge disparities in employment and educational attainment between the majority population and ROMA

24. Recommendations to European Commission
1. Promote awareness among ESF Managing Authorities and wider stakeholders about the relevance of the 10 Common Basic Principles for Roma Integration-EU Roma Platform- (‘explicit but not exclusive’ targeting)
2. Produce and disseminate good practice guidance (effective policies and instruments to promote Roma inclusion; how to measure soft outcomes)
3. Ensure that synergies are maximised between the ESF and ERDF (links between the two funds in promoting social inclusion and an integrated approach.
4. Amend Annex XXIII of the ESF Implementing Regulations (MS required to report on participation in the ESF by the Roma separately from ethnic minorities).
5. Require the Member States to report on the outcomes achieved through ESF support to the Roma at programme level (outputs and results indicators).

25. Recommendations to Member States
1. EU Member States should develop a national strategy for Roma integration and ensure that this makes maximum use of the full potential of the ESF to promote a holistic approach to Roma integration

26. Recommendations to MS: Programme planning and design (I)
2. Take into consideration the 10 Common Basic Principles for Roma Integration, esp. mainstreaming and ‘explicit but not exclusive’ and commitment to ‘integrated approach’ .
3. Further ESF investment to tackle low educational attainment among Roma youth and adults (distance learning courses).
4. Inclusion of Roma children in mainstream schools and complete their basic school education: ensure sufficient funding once ESF projects finish (e.g. classroom support and social assistants).
Measures to promote employability among Roma living in segregated Roma camps require a different approach. Likewise, differences between rural and urban settings should be better reflected in thinking through ‘what works’.

27. Recommendations to MS: Programme planning and design (II)
5. ESF MAs should place a stronger emphasis on tackling external barriers to Roma’s socio-economic inclusion/LM access.
6. Spatial and territorial factors should be taken into account. (e.g. employability in segregated require a different approach).
Measures to promote employability among Roma living in segregated Roma camps require a different approach. Likewise, differences between rural and urban settings should be better reflected in thinking through ‘what works’.

28. Recommendations to MS Management and implementation arrangements
8. Greater consideration should be given by ESF MAs as to how to derive maximum synergies between the ESF and other EU funding (ERDF) and (EARDF).
9. Strengthen vertical and horizontal coordination mechanisms to promote Roma integration to maximise potential synergies between ESF and the ERDF.
Measures to promote employability among Roma living in segregated Roma camps require a different approach. Likewise, differences between rural and urban settings should be better reflected in thinking through ‘what works’.

29. Recommendations to MS Implementation level and beneficiaries
10. Greater use should be made of micro-project grant schemes to enable smaller Roma NGOs to take part in ESF projects and thereby to develop and strengthen their capacity.
11. Promote stronger involvement of Roma NGOs across the ESF programme implementation lifecycle to improve the effectiveness of ESF support.
12. A bottom-up approach to addressing the social inclusion (grassroots and local, better understanding of realities on the ground).
Measures to promote employability among Roma living in segregated Roma camps require a different approach. Likewise, differences between rural and urban settings should be better reflected in thinking through ‘what works’.

30. Recommendations to MS: Monitoring and Evaluation
13. Improve quality of monitoring data on ESF and the Roma and report on systematically in ESF AIR.
data at programme level: (i) Expenditure on ESF support to Roma; (ii) participation by the Roma and (iii) outputs and results.
14. ESF MAs should actively promote a culture of evaluation and impact assessment in ESF support to the Roma.
15. Use Methodological tools to assess soft outcomes
There is currently in adequate assessment of what measures benefiting the Roma have achieved.

31. Thank you !!!!!!

32. Here you can find the whole evaluation and the Country Reports: