ComQuol: Service Focused Outcomes Professor Doulas MacInnes, Professor of Mental Health Chief Investigator (ComQuol) Canterbury Christ Church University

Overview Examination of each of the service focused outcomes at the three time points User satisfaction with services Social climate of the ward Levels of disturbance Health economic analysis

Satisfaction with Services Satisfaction is a multidimensional concept Satisfaction is context specific and satisfaction factors vary between different health areas (Langle et al, 2010) Coffey (2006) questioned validity of forensic satisfaction measures and the reliability of study findings Common finding was 90+% satisfaction rates (Baker, 2003; Ford et al, 1999)

(strongly disagree to strongly agree or not relevant) Outcome Assessment Patient Satisfaction Assessed using Forensic Satisfaction Scale (MacInnes et al, 2010) 60-item questionnaire rated on a 5-point Likert scale (strongly disagree to strongly agree or not relevant) Greater mean score is indicative of high levels of satisfaction Scored generated seven domains and a total score Staff interaction Rehabilitation Milieu Communication Finance Safety Overall Total score

Domain Mean (range 1-5) (SD) Satisfaction Scores   Intervention Group Control Group  Domain Mean (range 1-5) (SD) Base line 6-months 12- months 6- months Staff Interaction 3.2(0.3) 3.2 (0.3) 3.2 (0.4)  3.1 (0.1)  3.2 (0.1)  3.2 (0.2) Rehabilitation 3.4 (0.1) 3.6 (0.2) 3.6 (0.4) 3.5 (0.1) 3.6 (0.1)   3.5 (0.1) Communication 3.1 (0.2)  3.1 (0.2) 2.9 (0.2) 3.1 (0.0) Milieu 3.0 (0.2) 3.0 (0.3) 2.9 (0.0) 3.0 (0.0) Finance 3.2 (0.1) 3.3 (0.2) Safety 3.8 (0.1) 3.9 (0.2) 3.7 (0.2) 3.7 (0.4) Overall Care 3.3 (0.4)  3.3 (0.7) 3.4 (0.1)  3.3 (0.1) Total score 3.2 (0.2) 3.3(0.2) 3.3 (0.3) 3.2 (0.0)

Treatment Effects - Satisfaction   Treatment effect (intervention – control) and confidence interval 6 months 12 months Staff Interaction 0.0 (-0.5 to 0.5) -0.1 (-0.8 to 0.7) Rehabilitation 0.0 (-0.3 to 0.4) 0.1 (-0.5 to 0.7) Communication -0.1 (-0.3 to 0.2) Milieu 0.0 (-0.4 to 0.3) 0.0 (-0.7 to 0.7) Finance 0.0 (-0.3 to 0.3) 0.0 (-0.4 to 0.5) Safety 0.2 (-0.2 to 0.5) 0.2 (-0.5 to 0.8) Overall Care -0.1 (-0.8 to 0.6) -0.1 (-1.2 to 1.1) Total score

Social Climate of Ward Schalast and Redies (2005) the interaction of the material, social and emotional conditions of a ward which.. over time… influence the mood, behaviour and self confidence of the persons involved Significant relationship between social climate, users’ behaviour and wellbeing, staff morale and treatment outcomes.

Outcome Assessments Social Climate of Ward Assessed using the Essen Climate Evaluation Schema (EssenCES) (Schalast et al, 2008) 15-item questionnaire rated on 5-point Likert scale Items divided equally into three subscales Patients’ cohesion Mutual support among the patients Experienced safety Perceived tension and threat of violence Therapeutic hold Climate is supportive of patent's needs

Social Climate Scores Domain Mean (range 0-20) (SD) Baseline 6-Months Control Patient cohesion 10.0 (0.5) 10.6 (0.2) 9.3 (0.7) Experienced safety 10.1 (1.5) 16.3 (2.3) 15.4 (2.7) Therapeutic hold 12.1 (0.1) 11.7 (1.0) 12.2 (0.5) Intervention 10.1 (0.7) 8.8 (1.0) 12.7 (1.4) 15.4 (1.2) 16.3 (2.4) 12.4 (1.0) 10.7 (1.5) 11.6 (1.2)

Treatment Effects – Social Climate Treatment effect (intervention – control) and confidence interval 6-Months 12-Months Patient cohesion -1.7 (-3.3 to -0.2) 0.0 (-1.6 to 1.5) Experienced safety -0.9 (-5.1 to 3.2) 0.9 (-4.9 to 6.6) Therapeutic hold -1.1 (-3.9 to 1.8) -0.6 (-2.8 to 1.6)

Outcome Assessment Disturbance monitoring form Any disturbed behaviour involving the patient taken from ward incident forms and progress notes Recorded on a monthly basis from three months prior to the assessment till the 12 month follow up (15 time points)

Disturbance Monitoring Form   Intervention Group Control Group Disturbance (Number of ) Pre- ComQuol Post- Seclusions 9 37 Hours of seclusion 231 150 328 758 Physical restraint 8 22 35 Suicide attempts 1 2 7 17 Self-harm attempts 18 47 10 93 Violent acts on others 21 50 23 96 Violent attacks on inanimate objects 48 81 15 76 Attempted absconding/escapes 3 Actual absconding/escapes 4 11 Abusive/racial language 201 427 94 313

Economic Evaluation Examined the cost-effectiveness of the intervention compared to standard treatment The total cost for each group was calculated as the sum of the following: The cost of stay in the facility Cost of incidents Cost of intervention

Cost-Consequences Analysis Resource use and costs Intervention group Control group Total cost of intervention £30,413 £0 Cost of intervention per patient including nurse training £529 - 576 Av no of days in the facility over 12 months, mean (SD) 341 (56) 338 (37) Av cost of stay in the facility (bed-day cost) over 12 months £166,064 £164,506 Total cost of incidents £23,697 - £38,354 £51,222 - £92,340 Cost of incidents per patient £456 - £738 £985 - £1,776 Av treatment cost (intervention + stay + incidents) £167,049 - £167,378 £165,491 - 166,282

Conclusions Satisfaction – Little difference between intervention group vs. control group scores Social Climate – Little difference in scores except higher therapeutic hold sub-scale scores for the control group Disturbance – Much lower in most categories in intervention group Economic Evaluation – Overall costs similar, incident costs much lower in intervention group

Answers to Study Aims Establish the feasibility of the trial design as the basis for determining the viability of a large full-scale trial The trial design appears viable. The procedures seem to function well. The response rates were good with low patient withdrawal rates. Determine the variability of the outcomes of interest The variability of the outcomes of interest was all within normal limits. The estimated treatment effect of the primary outcome is clinically important. A full trial would be justified to estimate the effect with greater certainty.

Answers to Study Aims Estimate the costs of the intervention Incidents are costly, as associated with significant use of NHS resources and police. Real cost of incidents may be even higher when analysed using patient-level data. Refine the intervention following the outcome of the study based upon the experiences of the nurses and patients Responses suggest general satisfaction with the approach. Number of nurses lost to follow up questions including nursing outcomes. The reasons for higher dropout rates for women and whether to offer ongoing support. Examining the incident costs for longer may give a better indication of on-going costs.

Thank you for listening  For further information, please contact Douglas MacInnes – Chief Investigator douglas.macinnes@canterbury.ac.uk Jacqueline Mansfield – Research Assistant jacqueline.mansfield@canterbury.ac.uk Catherine Kinane – Principle Investigator (KMPT) catherine.kinane@kmpt.nhs.uk Janet Parrott – Prinicple Investigator (Oxleas) janet.parrott@oxleas.nhs.uk Ian Marsh – Service User Experience ian.marsh@canterbury.ac.uk