1. Feeding back Clinical Outcomes to Frontline Teams
UKRCOM
22nd January 2015

2. Outcomes Analyses
Have wanted to embed the routine measurement, analysis and feeding back of clinical outcomes to frontline teams for several years
Improve clinical effectiveness through reflective practice, shared learning, identifying gaps in service, training needs etc
Have had to deal with re-organisations and loss of data, changing priorities etc

3. CQUINs
Have used CQUINs to promote the use of outcome data – has ensured the Trust devoted resource from the Information team to develop analyses
CQUINs initially required recording of HoNOS scores at certain events eg acceptance to service, admission, discharge, CPA review
This year’s CQUIN required evidence that analyses of outcomes were actively fed back to teams

4. 2014/15 CQUIN for CNWL & WLMHT Numerator
Sample of 50 patients’ paired scores per CCG, across a range of care clusters per month - the patients chosen will be from within the same group of clusters i.e. non-psychosis ( 1-8); psychosis (10-17) organic (18-21); cluster groups may vary by individual CCG in agreement with commissioners
Denominator
All patients paired scores.
Rationale for inclusion
If ratings are incorporated into care plans then objectives can be quantified
- By using Outcome Measures, individual clinicians / teams can build up a picture over time of their service-users patterns of response to interventions and events that might not be easy to achieve without measurement.
-Sharing outcome analyses with frontline staff helps to improve accuracy of scoring, as staff see analyses of their interventions on their patients, and clinical effectiveness through reflection
-Managers can examine differences between outcomes between different teams and interventions on similar service-user groups.
-Commissioners can move from a purely activity/structured approach to a more rational purchasing model involving health gain.

5. CQUIN Milestones Date/period milestone relates to
Rules for achievement of milestones (including evidence to be supplied to commissioner)
Date milestone to be reported
Milestone weighting (% of CQUIN scheme available)
Quarter 1
Develop an Audit tool to allow for analysis of paired HoNOS /CROM scores
30th June 2014
15%
Quarter 2
Undertake the Audit - with different teams using CROM or HoNOS
30th September 2014
25%
Quarter 3
Review the outcome of the Audit and feedback to clinicians
Provide evidence that services are routinely sharing clinical outcomes analyses with frontline teams
Develop an action plan based on the Audit
Implement the Action Plan
31st December 2014
35%
Quarter 4
Re-Audit using paired HoNOS / CROM scores
28th February 2015

6. The Presentations: “How much do we help our patients?”
How do we know whether the interventions we provide are effective? Eg:
An antipsychotic?
A “brief intervention” eg course of CBT?
An admission to an acute ward?
A 2 year admission to a rehabilitation/ forensic unit?
How interested are we in whether we make a difference to our patients health and quality of life?

7. Why would we want to know if we were being clinically effective?
Delivering clinically effective interventions is arguably the most important thing we do for patients!
GPs, patients, NHSE, Monitor, CQC all want to know whether we provide a good (effective) service to patients
Commissioners want to know that actual clinical outcomes for patients using our services do improve
Measurement and analysis of outcomes provides this evidence
Really importantly, there is also clinical utility to measuring outcomes:
Systematic analyses of outcomes provide evidence of teams’ clinical effectiveness
Enrich clinicians & managers understanding of morbidity in their locality

8. Is there any evidence we make a difference?
For several years we have been recording HoNOS scores at key times during patients’ pathway through our services:
At first assessment
When there is a significant change in need eg admission
At CPA
At discharge
Comparing a patient’s scores from eg point 1 to point 4 gives us a measure of our effectiveness

9. Outcomes analyses
After years of collecting HoNOS scores, we now want to analyse these at a team level, identifying how we are doing
What are we doing well?
In what areas are we providing the most help for our patients?
Where are we doing less well and could benefit from further training, different staff mix etc
We have several analyses of outcomes scores in different formats and are really interested in your views as to which (if any) you find most helpful to understand whether you are delivering clinically effective care

10. METHODOLOGY
Paired HoNOS scores for selected Service-lines per CCG covering the period
April 2013 to September 2014 were analysed using the following method:
Scores extracted from JADE at point 1 and point 2 for each selected patient
For new patients, Point 1 consisted of first assessment scores. For existing patients, Point 1 consisted of scores at the start of a new cluster episode.
Point 2 will be scores on discharge to GP or at start of new cluster episode
There are four potential pathway scores:
New to Discharge
New to Review
Review to Discharge
Review to Review
Sufficient paired HoNOS scores were found for pathways 1 and 4.

11. DATA ANALYSES
Analysis of HoNOS scoring will consist of comparing aggregated mean scores for
patients at point 1 and point 2 using:
Mean total HoNOS scores at point 1 and point 2 and the difference for each sub sample.
HoNOS Four factor model showing differences in scores between point 1 and point 2 – for each sub-sample.
The HoNOS Categorical Change model. HoNOS scales were rated - 0 to 2 as LOW and 3 to 4 as HIGH. Scores were then classified as follows, from point 1 to 2:
- Low score to Low score
- Low score to High score
- High score to Low score
- High score to High score
Mean individual HoNOS scores at point and point 2 – for each sub-sample.

12. HoNOS SCALE FOUR FACTOR MODEL 1.
Overactive, aggressive, disruptive or agitated behaviour 2.
Non-accidental self-injury 3.
Problem drinking or drug taking 4.
Cognitive problems 5.
Physical illness or disability problems 6.
Problems associated with hallucinations and delusions 7.
Problems with depressed mood 8.
Other mental and behavioural problems 9.
Problems with relationships
10.
Problems with activity of daily living
11.
Problems with living conditions
12.
Problems with occupation and activities
FOUR FACTOR MODEL
Personal Well Being 4. 5.
10.
12.
Cognitive Problems
Physical illness or disability or disability problems
Problems with activities of daily living
Problems with occupation and activities
Emotional Well Being 2. 7. 8.
Non-accidental self injury
Problems with depressed mood
Other mental and behavioural problems
Social
Well Being 3. 9.
11.
Problem-drinking or drug taking
Problems with relationships
Problems with living conditions
Severe Disturbance 1. 6.
Overactive, aggressive, disruptive or agitated behaviour
Problems associated with hallucinations and delusions
Note: The four factor score is derived using the sum of the items in each factor/dimension. Note item 12 (problems with occupations and activities) appears in both personal and social wellbeing factors. This is because this item contributes equally to both factors.

13. SAMPLE CCG SERVICE LINE/TEAM CLUSTERS PATHWAY SAMPLE SIZE
NHS West London (K&C)
ABT
1 - 5 1 93
6 - 8 19 28
NHS Central London (Westminster)
Recovery 12 4 50 23 10 11 34 6
NHS Brent
Rehab 52
NHS Hillingdon
Acute
3 - 5 45
7 - 8 80
243
NHS Harrow
OPHA
238 20

14. MEAN TOTAL HoNOS SCORES

15. FOUR FACTOR CHANGE
PWB: Personal Wellbeing EWB: Emotional Wellbeing SWB: Social Wellbeing SD: Severe Disturbance

16. ABT [K&C] CATEGORICAL CHANGE

17. ABT [K&C] HoNOS PROFILE CHANGE

18. How do we compare with other ABTs?
Different demographics, but in comparison with the other ABTs, how are we doing?

19. SAMPLE CCG SERVICE LINE/TEAM CLUSTERS PATHWAY SAMPLE SIZE
NHS West London (K&C)
ABT
1 - 5 1 93
6 - 8 19 28
NHS Central London (Westminster)
113 42 50
NHS Brent 46 12 36
NHS Hillingdon 67 11
10, 11, 13, 15 (no 12 or 14 in sample) 10
NHS Harrow 94
7, 8 (no 6 in sample) 14
10 – 14 (no 15 in sample) 22

20. Paired HoNOS Categorical Change: CLUSTERS 1-5

21. Paired HoNOS Categorical Change: CLUSTERS 6-8

22. Paired HoNOS Categorical Change: CLUSTERS 10-15

23. Conclusion of Presentations:
Providing interventions which make a genuine, positive contribution to our patients’ lives is (or should be!) our top clinical priority
It is not always easy to determine how successfully we are achieving our aims
Systematic measurement and analysis of outcomes can help us to understand where we as individuals and teams are doing well and where we might need more development
Please let us have your thoughts on the utility of outcome measurement, so we can improve how scores are analysed & fed back to teams in the future

24. Staff were asked to evaluate the sessions:
How useful was it to receive an analysis of team outcomes using the 4 models?
Which model was most helpful?
Are there alternative ways of presenting outcomes which might be more useful?
How often should outcomes analyses be presented to teams?
Which other staff might benefit from being fed back outcomes analyses?

25. Results of Evaluation (n=26)
How interested are you in finding out whether the patients you treat get better?
Very Interested /26
Interested /26
Not sure/ Not Interested - 0/26
How useful was it to receive an analysis of team outcomes using the 4 models?
Very Useful /22
Useful /22
Not Sure /22
Not Useful /22

26. Which models were useful in helping you
understand changes in patients’ symptoms?
Type of analysis
Not
Useful
Very Useful
Total HoNOS score 1 2 11 8
4 factor model 3 5 9
Categorical Change model
Profile Change 12

27. Evaluation
How else could outcomes analyses be presented? (free text response)
Benchmarking against other teams, to identify service or demographic differences, or to highlight where teams are doing well/ less well.
Compare results with patient / carer responses
Use GP feedback
Undertake further analysis for patients whose scores remain high despite treatment
Separate out by diagnosis (as well as cluster) eg do personality disorder patients do differently?
Function on Jade to produce individual change graphs which can be shown to patients
Use case studies alongside outcomes analyses
 Who else would benefit from attending presentations on outcomes analyses? (free text)
Whole of the team including admin, managers / Senior management team
Service user groups...Commissioners

28. Conclusions
Results showed all staff who completed the feedback forms were interested in knowing whether the patients they treated improved as a result of their interventions.
The outcomes analyses that were shared with teams looked at paired HoNOS scores using 4 different models. All but one responder found the HoNOS analyses useful or very useful.
Most staff were unfamiliar with the models before the presentation. However responders found all four of the models either useful or very useful in helping them understand their outcomes (Total score change 83%, 4 factor model 65%, categorical change model 87%, profile change 88%). 4 factor was the model with the highest proportion of staff being unsure or finding it not useful (35%)

29. Conclusions
Responders preferred feedback to be given at either 3 monthly intervals (40%) or six monthly intervals (36%). Although numbers were relatively small, rehab staff had a preference for longer periods between presentations (6-12 monthly)
Additional outcomes analyses which staff thought would be useful including patient completed measures. Triangulation with PROMS would help add validity to clinician rated measures
Added contextual information such as diagnosis and demographics was thought to be helpful
Staff thought all members of the team including admin staff and senior managers should be presented analyses of outcomes. Some also supported outcomes analyses to be presented to commissioners

30. Next Steps
Roll out outcomes analyses to all frontline mental health teams during the course of 2015.
Analyses should be actively presented to all members of teams (eg for 30 minutes during an MDT), by outcomes leads who understand the models and can facilitate discussions on what analyses mean.

31. Next Steps
Each Divisional Medical Director to identify an outcomes leads for their mental health teams. The role of the leads will include:
To liaise with the Information Team to ensure the correct analyses are being prepared for their allocated clinical teams
To attend training on the models and on how to facilitate a feedback session.
To develop a programme to deliver presentations to each of their allocated teams during the course of 2015.

32. Questions? Advice?