Aligning clinician and patient reported outcomes Tales from the Beautiful South July Liz Vernon-Wilson

How do we know what we’re doing well? –With an individual with our caseload? –Or as a team/service/provider? Do we understand case mix, resource use and effect? Businesses use data from different sources to assess their market, and evaluate how well they’re meeting market demands. Staff and consumers can contribute to the information pool and drive improvement of understanding and service if the data is contextualised and used…. Quality & Outcomes What are our clients’ needs? How much of this and how much of that do we need?

Creating Outcomes Data Local CQUINs followed national directive to collect CROM, PROM & PREM and other quality indicators to support PbR development (CPA etc). Care review Discharge Initial community contact Initial community contact PROM Hope Agency Opportunity Working relationships Recovery Personalisation Outcomes Evidence Partnership Local choices of PROMs reflect an interest in understanding how Recovery orientated our service are (working aged adult MH) Holistic our interventions are (integrated older adult MH services).

Using Outcomes Data We aim to integrate data from different sources to pull together a narrative about clinical effectiveness and patient experience Validate with what the weather/health was like & move towards Predicting what it will be like? Role of interventions? The challenges are –to validate tools –build clinically relevant scenarios –Get clinicians asking questions rather than being recipients of deductive analysis

What can PROM and HoNOS show? They measure different things, but some scales are related and create an overall picture of well being. 5 EQ-5D scale for depression/anxiety relates to HoNOS65+ Mood and Other scales (7 & 8). HoNOS Scale 10 (Activities of Daily Living) and EQ-5D Self Care, Usual activities EQ-5D Visual analogue scale records change but EQ domains are unchanged. HoNOS 4-factor shows effect size change, when individual scales are aren’t significant alone. How does this help understand clinical effectiveness? Benchmarking to understand what change is possible/likely.. Reflective, constructive comparison between teams…. The sensitivity of scales may be appropriate to some, but not all service user groups/points in the care pathway.

Outcomes for service users in Cluster 4 Non-psychotic problems (moderate*/severe) of depression/anxiety/other NP. They may experience disruption to function in everyday life and increasing likelihood of significant risks. What can cluster assessment data show about what happens? AMH n=425 OPMH n=105 First, 50% of service users in AMH, but only 33% in OPMH have cluster 4 renewed. Service users are more likely to move to cluster 3 in OPMH. Do OPMH do a better job?

Number making transition to cluster 3 is n=93 AMH; n=53 OPMH. Average HoNOS/(65+) profiles from first cluster assessment in pair shown. *Starting points are significantly different for scales 5 & 9. Mean duration of these pairs was 200 days AMH, 230 days OPMH; not quite significantly different (p=0.057). However, over all cluster 4 reviews are done quicker by AMH than OPMH (182 days compared to 214; p=0.0013). Understanding Cluster 4:3 transition data All cluster 4 pairs Clinical change observed needs to be contextualised by time taken **

Cluster 4:3 Clinical outcomes The mean scores for HoNOS /(65+) recorded at cluster transition are shown. AMH pairs show sig difference* on scales 2, 7-10 (paired T test p<0.05). OPMH pairs show sig change on scales except 3 & 5. What change do service users report? What interventions are used? Mean changes recorded for OPMH service users are greater than AMH, noticeably on scales 1,7 & 8 which rate behaviour, mood & other problems. But the difference between AMH & OPMH is only significant on scale 7 (mood). * ****

Good & Bad News… No equivalent cluster 4 to cluster 3 transition pairs available for PROM, But we can investigate –Cluster 4 renewal pairs (and they show some improvement too…..) –PROM responses of those that have scale 7 & 8 ratings similar to those indicated for cluster 4:3 transitions. n =240n =35

What AMH service users in cluster 4 report… Trend data from Hope, Agency & Opp PROM suggests service users feel more hopeful etc. through the care pathway. Do cluster transitions follow? Our practice doesn’t yet follow clear temporal correlation between when PROM is offered and when cluster is reviewed. Mean duration of PROM pairs here 109 days compared to cluster review duration of 182 days. Limited pairs (n=20), all were renewals, mean duration 109 days Severe problem Improvement Deterioration Hopeless Hopeful Hopeless

What OPMH service users in cluster 4 report… Improvement Deterioration Improvement Deterioration Older aged adults, all cluster 4 pairs; n=32. Mean duration 6 months Trend data and paired data show improvement on the VAS (0-100 score) and individual domains (summarised as index score). Paired t test show statistical significance. Effect size? Mean duration 6 months is closer to cluster review (mean 7 months) than AMH PROM pairs. EQ indexVASSample size All pairsMedium (0.5) Medium (0.7) 32 New referral to review or discharge Medium (0.6) Large (1.1)13

PROMs for other clusters EQ-5D shows similar results for service users’ completed episodes in clusters 3 & 4, but cluster 18 data is different. The VAS & index profile recorded at initial assessment is sig diff for all three groups. Cluster 3Cluster 4Cluster 18 VAS EQ- index Starting point differs, as does “outcome”. Cluster 18 pairs show no change, cluster 4 show improvements.

HAO- Clusters 12 & 4 compared Comparison of trend scores for cluster 12 and cluster 4 show differences in the range of HAO scores recorded and variation in observed change. Paired scores suggest that there are clinically significant effects for service users in cluster 4, that are not apparent in cluster 12, although sample size is low.

Correlating CROM & PROM Service users rated as having mood or anxiety problems (HoNOS) have more HAO problems (any cluster). Improvement in mood / anxiety could trigger cluster change. Improved mood is likely to correlate with improved PROM. Improvement in anxiety is less well correlated. Improvement in hallucinations & delusions has no impact on HAO rating. n=1169 no, or minor problems not needing intervention n= 305 mod/severe hallucinations & delusions

Outcomes Information for whom? Cluster 12 results illustrate that outcomes for service user are wider than anything we measure, CROM changes but recovery PROM doesn’t. Service’s aspirations for outcomes information can be different to one another (AMH/OPMH) and individual clinicians’ and service users. But there is overlap… good outcomes for all; To improve quality of life Prevent relapse Help improve quality of services Measure at contact Supportive relationships Prevent relapse Physical well being Mental well being Work Self- management Home Our information represents sampling at point of contact

Summary what does the data show? HoNOS pairs give the clinicians’ perspective of change. Clustering has driven more regular collection. PROMS EQ-5D & HAO show the service users’ views, but cover different areas & don’t always correlate with cluster. They do help with engagement & evidence personalisation. When CROM & PROM coincide, they enrich our understanding of outcomes at clinically significant points. Different groups show different changes, e.g. cluster 18 and cluster 4 are different. Change varies depending on duration of pair and starting point. Clinician and service user report similar changes (or stabilisation). Suggesting we don’t engage with “gaming” (quality in PbR world).

(Some of) the Challenges ahead Validation (tools & models) More data! Quantity & quality please Information/ knowledge quality of contextualisation Using the data reflectively & more –Do cluster transitions help understand resource use? –Does benchmarking HoNOS/PROM show difference in practice? –help understand what changes we’d hope to see from interventions? –or encourage service improvement? 17

More questions? 18

Outcomes for service users We’ve tried to measure some of what everyone needs, but do clinical pathways now direct us towards more specific tools? HoNOS Hope, Agency, & Opportunity PROM

Section 2 Relating PROM to Clinician Reported Outcomes Service users who rated themselves as having a severe or disabling problem were compared to those who rated themselves as having no problem (2 domains chosen). Average HoNOS profiles for the two groups were compared. EQ-5D depression/anxiety maps to HoNOS mood and Other scale. Usual activities maps to more HoNOS scales.. Findings HoNOS65+ reports on similar problems to EQ-5D domains indicating clinician and patient reported outcomes should be comparable. The VAS is a more holistic measure, affected by a broader range of factors, some of which may not be captured by HoNOS65+. It represents a different way of expressing outcome, relating to overall well-being.

Appendix I …PROM

Appendix 1 Hope, Agency & Opportunity Questionnaire 22