1. The external evaluation of the Islington Refugee Integration Strategy Linda Pearce Liz Saunders Murray Saunders 1

2. Overview of the context Aspects of the evaluative research design The evaluative research process in low trust environments Issues and challenges in considering voice, participation and building social capital 2

3. Islington’s Refugee Integration Strategy  Developed in response to the ‘Best Value Review’ of Islington Council’s services for refugees & asylum seekers and subsequent audit commission report of July 2004  Defines Islington Council’s corporate commitment to refugees & aligned with ‘One Islington’ vision 2005-2008; a key driver of services  Provides a means of mainstreaming services for refugees by feeding into Islington’s Local Area Agreement, where refugees are identified as key priority group  Presents the vision for refugee integration delivered through clearly targeted and managed action plans, based on strategic priorities across main indicators of refugee integration; Community Safety, Community Integration, Education, Employment, Housing, and Health 3

4. During the evaluation 95 refugees & asylum seekers were interviewed directly. These were drawn from:  Refugee Community Organisations  Education Providers  Refugee Centres  Cafes & Restaurants  Events, Forums, Consultations & Working Parties People groups interviewed included:  Congolese  Turkish  Latin American  North African  Eritrean  Iranian 4

5.  The provision of formative independent evidence of the overall strategy to the community and principally IRIS and the RSPB and its partners: 2007-2008  An account and an on-going analysis of the experience of the strategy from the perspective of key stakeholders  A design which will be responsive and flexible enough to capture unintended outcomes and unanticipated effects  The provision of an overall summary of progress of the strategy, highlighting strengths, weaknesses and areas of development for use at the various review moments within the two years 5

6.  Capturing refugee voices through evaluative research: building informal social capital  Using an ‘inclusive approach’  Evaluation as part of a democratic vision of civic participation  The search for an authentic voice  Problems of trust 6

7.  Focus 1. Primary integration: basic indicators of service experience in health, accommodation, education, work or benefits and safety (hard indicators)  Focus 2 Secondary integration: based on routine social practice and experience (soft indicators of cultural change) Contribution individuals and groups make to community life Friendship groups and networks Associations and active local citizenship Neighbourhood tone  The priority actions will form the ‘mechanisms’ that will be the focus for the evaluation and will be the ‘theory in action’ of the IRIS approach 7

8.  Interviews with provider stakeholders (IRIS team, policy makers and providers)  Focus on expectations, process issues, communication and partnership, connectivity, improvements, ‘sticky spots’, futures  sweep of refugee experience  Focus on soft and hard indicators with a sample of interviews from refugee groups (looking at differences in experience by gender, country of origin, ethnicity, age)  Focus on primary and secondary integration through sampled interviews and informal methodology  Interviews with refugee group representatives for experience of IRIS 8

9.  Meta concerns with differences between service areas, partnership working, refugee communities, ‘sticky spots’ and futures, gender, ethnicity  Community safety (experience of victim support, experience of routine crime, cross community collaboration, street ‘tone’)  Community integration (soft indicators, participation in planning, identification of ‘success’ indicators, experience of capacity building, patterns of association)  Employment (skill assessments, training, changing employers attitudes 9

10.  Education (existing data and needs assessment practices, achievements, associations, networks, participation)  Health (experience of being a patient, access to services, partnerships, awareness of specific needs based on refugee community, voluntary agency support)  Housing (needs assessment processes, indefinite leave provision, experience of housing advice, voluntary sector practices) 10

11. The research persona:  Persistence  Searching for locations (informal interviews, observations etc)  Sensitivity to fear and uncertainty  Tolerance to timing issues  Flexibility 11

12. Practicalities:  Ethics (gender / safety)  Ownership  Confidentiality  Legality  Cross-culture awareness  Evaluator’s role –personal clarity and understanding  Negotiating boundaries 12

13.  Finding the voices in collaboration with:  IRIS team  Refugee Strategic Planning Board  Voluntary and statutory organisations  Refugees themselves 13

14.  The identification of people who could be seen as gatekeepers to facilitate meetings with refugee community members so that voices on the ground could be heard:  central to unlocking access  ‘snowball’ effects  hearing authentic voices 14

15.  The Methodology –  Getting to real voices requires ‘informal methods of data collection’  Focus groups  Semi-structured group interviews  Shadowing  Questionnaires  Event observations  Informal cafe and restaurant interviews See Vignettes of refugee experience 15

16.  Apparent authenticity: the complexity of community voices – speaking with one voice addressed by: breadth of representation from participating communities: moderating findings across participants range of interpreters from different backgrounds 16

17.  ‘mediated voices’ (community leaders with a good grasp of English and knowledge of English culture acted as ‘key informants’ for their communities)  self appointed mediators: ‘natural assumption’ of a spokesperson role  elderly male community leaders for example may not speak ‘authentically’ (in the sense we outline above) with the voice of either the women or the youth of their community. 17

18.  worked on the basis that ‘authenticity’ was evident when: the experiences described were those of the interviewee the experiences described were not mediated by an external party where an external party was acting as a spokesperson, they were given authority by those they were representing, to ‘capture’ a collective experience  18

19.  Can evaluative research construct the views of the interviewees authentically?  What is the status of ‘explanatory interpretations’ offered by researchers?  The methodology required the capture of ‘voices’ at each stage ; to capture and foreground the ‘voices’ of refugee community members themselves as the intended recipients of strategy Examples of interpretive feedback 19

20. Voice is ‘heard’ by: involving participants in identifying and using key questions, indicators or issues being part of an ethically justifiable process (a concern with evaluation ethics). making sure their experience is ‘faithfully’ reported even under political pressure (a concern with declamatory platforms). making sure evaluation outputs enter into a public debate (a concern with evaluation as part of a democratic process and as a way of promoting democratic participation). 20

21. If participation does not lead to change, the sense of value initiated will stagnate and breed frustration – is this the challenge for the future of this kind of evaluation approach.......? Is the biggest challenge for the future to ensure that the political context of participative approaches to evaluation can live up to its promise......if no, rather than building social capital, we will be dismantling it. 21