1. Payment by Results a rational approach to outcomes Duncan Raistrick Leeds Addiction Unit Society for the Study of Addiction Conference 8-9 th November 2012

2. NIHR Nine CLAHRCs approx £10million each CLAHRC for Leeds/York/Bradford Addiction Research in Acute Settings (ARiAS) 1 of 5 themes ARiAS outcome measurement 1 of 6 strands CLAHRC research group: Duncan Raistrick, Gillian Tober, Christine Godfrey, Charlie Lloyd, Steve Parrot, Jude Watson, Veronica Dale Co-opted Expert Group convened March 2011: Owen Bowden-Jones, Alex Copello, Ed Day, Eilish Gilvarry, Don Lavoie, Damian Mitchell, Julia Sinclair, John Strang, and Alex Whincup

3. Summary o What is Payment by Results  Alcohol  Drugs  Stakeholders o Evidence for incentivised treatment  Types of incentives  Weaknesses of treatment outcome measurement o Measuring addiction outcomes  Types of measures  Substance use  Societal impact measures  Secondary outcome measures

4. Payment by Results Quality and Outcomes Indicators Report for Product Review Group Quality & Outcomes Sub Group. October 2011 PbR for mental illness based on tariffs for 21 clusters Outcomes: HoNOS, GAD7, PHQ9, Recovery Star, Carers’ and Users’ Expectations of Service (CUES), and possible 21 indicators/outcomes (incl. employment, % on CPA etc) HoNOS scoreCare ClusterPbR

5. Developed the Alcohol PbR Reflects an evidence based approach – fits with mental illness PbR Outcomes tbd but will include Q/F as key measure Developed the Drug PbR Reflects a political agenda – different to other PbRs Outcomes are all recorded on TOP (rater completed)

6. 100% PbR not required 5-10% more typical – if too high likely to lead to ‘gaming’ (fraud, manipulation, overestimation) 1.Free from drugs of dependence - abstinence TOP scores rater completed– not objective 2.Employment dropped 3.Offending if based on group values local patterns greater influence 4.Health and wellbeing – inj: NFA: HepB: wellbeing TOP scores rater completed– not objective Local Area Single Assessment and Referral System (LASARS) assess and refer clients and set their tariffs – expensive, not SU friendly...independent view to DH/NTA of Drug PbR...

7. Making Payment by Results work in 2012 Center for Policy Studies research fellow Kathy Gyngell suggests reforms to the Payment by Results for drug treatment “Offering the addict the real choice of a) a replacement prescription conditional on regular clean illicit drug use and alcohol tests, or b) a ‘breaking free of dependence’ option. Payment conditional on sustained drugs tested sobriety (abstinence from opiate substitute drugs, all illicit drugs and alcohol). Payments progressive - starting at one month sobriety (25%), a second at three months (25%) and the (50%) at six months”

8. Pre and Post Performance Based Contracting in Maine effect on public sector take up of most severe cases Take up of more severe problem drinkers Pre-PBCPost-PBCdiff Out-patient Public27%20%-7% Medicaid24%26%+2% In-patient Public44%64%+20% Medicaid47%58%+11% Source: Shen (2003) Health Services Research Incentives based on 3 efficiency, 13 effectiveness, 9 population standards

9. Payment by Results Quality and Outcomes Indicators Group set criteria..... The indicator demonstrably reflects quality or outcomes that are relevant to the service user, practitioner, provider or commissioner. Data is available at all levels and can be fed back to practitioner/teams Evidence / proof that the data required to demonstrate the indicators can be collected & evaluated They must support the high-level indicators which are likely to include employment, mortality, suicide and recovery That they have nationally agreed definitions They should fit with Commissioning for Quality and Innovation scheme (CQUIN). See Glossary for more detailed description. They are specific to currency group(s) A link can be made between needs (both physical & mental health), interventions and outcomes That other similar processes that may be trying to do achieve the same end should be evaluated, for example work on clinical indicator......but didn’t find any perfect measures

10. Recognise that different Payment by Results stakeholders want different things from the ‘Result’ o Service users and carers – abstinence o Politicians and commissioners – costs and benefits o Practitioners  Health workers – physical and mental health  Criminal justice workers – offending behaviour  Social workers – safeguarding children......so, there is no ideal measure, just the mix of measures that best suits the purpose. 1 st Key point

11. What do service users and carers think “being better” means? 1. Abstinence incl. no scrip drugs 2. Being with non- drinking/drug using people Source Thurgood et al. (in prep) CLAHRC Being better Relationships Social Situation Self awareness AbstinanceActivitiesHealth Friends and family

12. Understand different types of measure – choose the best mix Generic Measures (Health) Is treatment cost effective? How ill are people with addiction problems compared to other users of health care? How complex are the health problems? What is the illness profile of people with addictions? Dimension Measures (Addiction) How severe is the addiction? How difficult is treatment likely to be? How good is one addiction service compared to another? Do problems persist? Condition Specific Measures (Depression, Pregnancy) How severe is the specific condition? How do services targeting the condition compare? How effective is treatment for this specific problem? Societal Impact Measures (Service level) Source: Fitzpatrick et al. (1998) Health Technology Assessment 2nd Key point

13. Societal Impact Measures - SIMs  Treating the individual has benefits at the societal level – SIMs are service level  Political interest is in societal costs (a research exercise) – SIMs are a headline contribution  SIMs need to be: objective, easy to collect, capable of showing variability, reflect the impact of the service in the fewest possible measures............... Pregnancy and Parenting i) birth weight ii) child with mother at 12months Hospital In-reach i) A&E attendances ii) in-patient admissions in last 12months Detoxification i) % completing ii) % supervised disulfiram

14. A Quality Framework for Outcome Scales Self completion scales are the gold standard Scales evaluated by scoring for:  Evidence Base (including independent evaluations)  Psychometric Properties  Normative Data  Availability (free and supported by website)  Ease of Use  Universality (all substances and all socio-economic groups)  Service User Evaluations Source: NIMHE (2009)

15. Scales that best match quality criteria... Evidence (0-2) Number of validation publications Psychometric properties (0-6) Availability (0-2) Practicality (0-3) Universality (0-2) Population norms (0-2) EQ-5D health 2tbc62222 LDQ dependence 2352321 CORE-10 mental health 2252322 SSQ satisfaction 1152321 APQ problems 2262300 HoNOS mental health 2tbc4232 PHQ9 depression 2tbc 232 GAD7 anxiety 2tbc 232 IRS impulsivity 2052220 FMQ family coping tbc 2 20 PCQ parenting 1122320

16. Domains (or components) of Addiction psychological well being dependence social well being substance use

17. Justification for substance use as the primary outcome measure.......  Viewed by service users as most important  Viewed by careers as most important  Most convincing for general public and policy makers  Substance misuse is the condition  Correlation with societal costs

18. Correlations with Societal Costs example data for units of alcohol and health status weekly units of alcohol health as % of best possible health moderate drinking not associated with major health problems health may not recover once damage is done

19. Correlations with Substance Use.... BaselineFollow-up EQ5DLDQCORESSQEQ5DLDQCORESSQ Alcohol freq -0.32**0.48**0.28**-0.08*-0.30**0.62**0.39**-0.20** Alcohol units -0.38**0.47**0.25**-0.19**-0.30**0.62**0.38**-0.20** Heroin freq -0.110.39**0.09-0.10-0.210.40** -0.60** **p<.001 *p<.01 Source: unpublished CLAHRC clinical sample but....  substance use is difficult to measure – LDQ good proxy  other scales are needed to paint a picture of outcome.... There is a scientific and political case for the primacy of substance use as an outcome measure....

20. Factor analysis with #4 factor solution EQ5DLDQCORE-10SSQ F#1F#2F#3F#4F#1F#2F#3F#4F#1F#2F#3F#4F#1F#2F#3F#4.77.13.07-.09.10.80.25-.10.19.36.71-.12.11 -.04-.01.66.67-.02.15-.08.12.81.24-.15 -.01.15-.40 -.01 -.05-.04.70.57.28.31-.15.12.80.27-.10.02.26.44-.22 -.15 -.19-.03.57.68.22.08-.07.11.83.20-.11.04.19.60-.14 -.05 -.22-.06.59.25.31.69-.12.18.65.20-.11.28.25.75-.10 -.17 -.13-.30.56.15.75.18-.13.34.14.57-.20 -.11 -.07-.30.60.04.75.25-.08.26.30.55-.11 -.06 -.05-.20.64.04.66.22-.12.22.31.77-.22 -.02 -.11-.25.57.09.78.22-.12.20.30.73-.23.15.74.35-.13.27.22.69-.17 8% of variance23% of variance17% of variance9% of variance

21. Relationship between EQ5D LDQ CORE SSQ LDQ and CORE close to each other. Predicts if dependence is treated then mental health improves. SSQ is most separated. Predicts it will change most differently to other measures. EQ5D is generic but independent and between the other measures. Has less influence than LDQ on mental health more on social satisfaction.

22. Clinically Significant Change - ‘Gold Standard’ Reliable Change Score Well Functioning Population LDQ>= 4< 12 CORE-10>= 6< 14 SSQ>= 4> 10 Source: CLAHRC submitted Jacobson et al. (1999) proposed that in order to take account of baseline scores and measuring error, clinically significant change should a) be statistically reliable b) end scores be in a well functioning population range RC = reliable change 95% probability if RC >=1.28 S diff = standard error of difference between means of LDQ scores Se = standard error of measurement of LDQ S 1 = standard deviation of mean 1 r retest = test/retest reliabilty of LDQ

23. Clinically Significant Change - example reliable change >=4 if pushed into well functioning population range then Clinically Significant Change achieved well functioning population is 2standard deviation above general population mean LDQ<12

24. Reliable (RC) and clinically significant (CS) change (n-925).... n=925 LDQ %CORE %SSQ % RC improved61.046.036.7 RC worse4.65.712.0 Too small for RC14.36.15.2 CS improvement50.130.531.9 clinically significant change at 3mth (drinking n=396)..... CS change %YesNoYesNoYesNo Drinking22.031.113.439.616.736.4 Abstinent35.411.624.222.717.729.3 P<.001 n.s. 95% probability of real change social situation most difficult to recover abstinence associated with the most CSC. LDQ works across drinking outcomes CSC is a tough test of improvement The GOLD standard

25. Payment by Results as a package....  Payment by practitioner competence eg %competent on video recorded practice by process rating  Payment for effective management eg pay for participation in approved addiction service business improvement  Payment for quality/added value practice eg smoking cessation, health checks (as QoF in primary care) and SIMs  Give service users purchasing power eg vouchers, personal development budget  Payment for in-treatment performance eg abstinence, secondary measures, socital impacts Source: Humphreys & McLellan (2011) Addiction 3rd Key point