Evaluating Open Dialogue in the NHS: the ODDESSI research programme Outline and Progress to date Professor Stephen Pilling PhD Research Department of Clinical, Educational and Health Psychology University College London, UK

UK Research Programme “Open Dialogue – Development and Evaluation of a Social Network Intervention for Severe Mental IIlness (ODDESSI)” 5 year programme, NIHR Programme Grant for Applied Research Comprehensive evaluation with 5 workpackages, including a multi-centre cluster RCT 5+ NHS Trusts across UK signed up as study sites Majority of OD staff teams, including peer support workers, will be trained in 2017/2018 Programme milestones started July 2017 review December 2018 completion October 2022

Programme aims Develop a protocol acceptable to clinicians and service users for OD in the NHS for the management of mental health crises Assess the clinical and cost-effectiveness of OD compared to usual care: does OD reduce time to relapse and improve quality of social network? Assess the organisational changes required to support OD implementation in the NHS: can we organise services to deliver OD and develop a sustainable model? Understand service user and their family and wider networks’ experience of OD compared to usual care

5 Workpackages (WP) From bid: Months 12 24 36 48 60 WP1: Intervention development WP2: Feasibility study WP3: Multicentre cluster RCT and economic evaluation WP4: Implementation and organisational change process evaluation From bid: (a) developing and refining the OD intervention, (b) subjecting it to feasibility testing, (c) evaluating whether OD is clinically superior and cost-effective to usual NHS care, (d) assessing service user and staff experience of OD, and (e) evaluating whether OD can be implemented as a sustainable service model for routine care in the NHS. To achieve these aims we will develop and test a new in-service model of training for OD, develop and refine a peer support worker role, test the clinical and cost-effectiveness of OD, describe and assess its delivery in the programme and model the implementation of a catchment area OD service. WP1 develops the OD model for use in the NHS. WP2 tests the feasibility of evaluating OD and determines if a large-scale evaluation of OD is possible. The outcomes of WP1 and WP2 will inform modifications to the intervention, trial design and outcome measures and progression to WP3. WP3 will assesses the clinical and cost-effectiveness of OD through a multicentre cluster RCT. WP4 run in parallel with WP2 and WP3 evaluates key elements of OD intervention to inform its implementation in the NHS. This includes the peer support worker (PSW) role, the experience of staff (including PSWs), the development of an in-service training programme and the impact of OD on shared decision-making. This work will inform a final phase of WP4 to model a catchment area OD service. WP5 is a parallel process of service user and carer experience of the OD intervention. WP5: Evaluation of service user, family and network experience of OD

WP1 – Intervention Development 1) Refine and establish the intervention OD teams start operating across all sites ✓ and ? Develop operational protocol with core and flexible OD model functions and staffing structure, including peer support ✓ and ? Stakeholder interviews/focus groups to support NHS implementation - how does the model vary across Trusts? ✓ 2) Develop Peer Support Worker component Integral members of OD team ✓ Role: assist service user to develop and maintain social network, attend network meetings, team meetings and supervision ✓ and ? From bid: Operational protocol: It includes the expected staffing levels needed to discharge these functions, suggested participants flows and the relationship of the OD team to other community and in-patient mental health services. Specifically, it includes for each site: the key functions and organisational structure of OD teams (including PSWs) the referral pathways, caseload capacity and team composition of the OD service the nature of the relationship between OD and standard inpatient, crisis and community services the support, supervision and governance arrangements in each site WP aims: The results of this analysis will also inform the scope of the process evaluation which will run parallel to the trial (WP4). This will be designed to assess how the intervention operates and whether assumed mechanisms of action function as intended. If the process of translation is shown to involve modification of any intervention components thought to be critical, then the process evaluation will seek to assess whether it has a positive, negative or neutral impact.

WP2 – Feasibility Trial 9 month feasibility trial of 2 OD teams and 2 TaU teams in 2 Trust sites – to start March/April 2108 Can we satisfy the following stop-go criteria so we can progress to the main trial: Can we recruit 10% of trial sample? Can we retain 80% of participants at 3 month follow up? Can we collect primary outcome data from 85% at 3 months? Can we achieve the expected consent rate? Can all sites establish OD teams and the geographical clusters they serve and can all OD teams operate to protocol? Can all OD teams achieve adequate adherence and fidelity? From bid: If there is a shortfall by a modest amount (10% to 15%) we will review the reasons for this and look to potential remediation of any identified problems and discuss progression to the full trial with NIHR. If there is a significant shortfall on key outcomes and no obvious remediation can be identified, we will discuss with NIHR not proceeding to the full trial.

WP3 – Multicentre cluster RCT Pragmatic two-arm cluster RCT and cost-effectiveness evaluation of OD versus usual care (routine NHS crisis care, CRTHTT, and longer-term community care) 28 clusters randomized to deliver OD or usual care (23 per cluster) n=644 recruited over 12 months, 24 month follow up Include: 18+ years, service user in crisis, within 48 hours of CRHTT referral or discharged to CRHTT, ICD MH diagnosis Primary outcome: time to relapse (case-note method) Secondary outcomes: social network quality and size, hospitalisation, recovery, satisfaction with care, QoL, carer burden of care and shared decision making

WP4 – Implementation and organisational change process evaluation Stakeholder consultation – staff, NHS managers, service users ✓ Assess OD team adherence and fidelity to intervention and service model across sites ✓ and ? Explore staff experience of OD training and of delivering OD and usual care ✓ and ? Develop an in-service staff OD training programme ✓ Model a catchment-wide ‘model’ OD service in an NHS site following main trial

WP5 – Evaluation of service user, family and network experience of OD Service user, family member and/or social network members and practitioner experience of ‘doing’ OD together – what works, how does it work and what’s different to usual care? Comparative case study approach: interviews with OD and usual care families Topics could include experience of process and change, contextual factors affecting access and outcomes, perceived change in relationships, social engagement, empowerment

Challenges in Delivering the ODESSI Programme Recruiting 28 clusters (and 644 participants!) NELFT, KMPT, CIFT, BEH, DPT (?? Others ) Maintaining staff training throughout the programme (300 trained, 100 on ’new model’) Development of the peer support worker role Managing case loads and expectations in OD and TAU services Maintaining adherence/fidelity to the model throughout the programme (for OD and TAU) Keeping in contact with 644 people Surviving the changes in the NHS

Challenges in Delivering the ODESSI Programme Implementing the ODESSI Protocol Protocol based on core OD principles Essential components of OD team specified Definition of crises Definition of and referral pathways to clusters Links to crisis and in-patient care Avoiding ‘contamination’ - OD ‘leakage’ Variation of importance but has limits