1. Standardised Mortality Ratios & their monitoring Paul Hawgood Everything that you wanted to know about SHMI but were too afraid to ask!

2. Housekeeping

3. Join the conversation.. #SaferNHS @AQUA_Inform

4. Agenda (1) Welcome, introductions, housekeeping Planned outcomes for the workshop Quiz An introduction to SMRs How SHMI is calculated Differences between SMRs What to look for in the data Myth busters

5. Agenda (Pre Coffee) ItemTime Welcome, introductions, housekeeping 9:30 Planned outcomes for the workshop9:45 Quiz9:50 An introduction to SMRs How SHMI is calculated Differences between SMRs What to look for in the data Myth busters 10:00 - 11:30 Break

6. Agenda (2) Q&A Session Quiz – yes, again! How do I engage with AQuA? How do I engage with my local trust / health economy/ social services?

7. Agenda (Post Coffee) ItemTime Q&A Session11:30 Quiz – yes, again!11:45 How do I engage with AQuA?12:00 How do I engage with my local trust / health economy/ social services? 12:15 Lunch 12:30

8. Outcomes Increase your understanding of SMRs in general and, specifically, SHMI and HSMR so that you can: – work with local acute Trusts on an informed basis –look to reduce mortality rates as a whole health economy Provide you with the knowledge to ask the right questions of providers. Place SMR in context/relation to other indicators of the quality of care.

9. It’s not about the data! “It’s true that cancer care has come on in leaps and bounds but we can’t look at our comparative survival rates and think anything other than we need to do more.” Professor Sir Bruce Keogh, National Medical Director 24/3/2014

11. Content My approach / house rules Crude rate Standardised Mortality Ratios [SMRs] Different methodologies SHMI in detail Interpretation Myth busters Questions

12. Crude [mortality] rate

13. Problem?

14. Standardised Mortality Ratios Standardising means “making adjustments” –age, sex, severity of condition –admission type, deprivation, palliative care Indirect and direct standardisation Calculates a new, standardised mortality rate ‘Rate’, ‘Ratio’ or ‘Index’?

15. Damned lies “Essentially, all models are wrong but some are useful.” George E.P. Box 1951

17. SHMI Oct 2011 to Oct 2012 (NW Trusts)

18. Methodologies 3 main methodologies in NHS SHMI, HSMR & RAMI –Describe –Similarities –Differences –Pros & Cons

19. Summary Hospital-Level Mortality Indicator SHMI Published by NHS [HSCIC] Published quarterly

20. Ja n

21. Summary Hospital-Level Mortality Indicator SHMI Published by NHS [HSCIC] Published quarterly Key Differences –137 Trusts –Out-of-hospital deaths (30 days post discharge) –Agnostic to Palliative Care

22. Hospital Standardised Mortality Ratio HSMR Published by Dr Foster Intelligence –Hospital Guide Fore-runner to SHMI Published annually / quarterly Key differences –All trusts –Explicit adjustment for deprivation –Palliative Care

23. Risk Adjusted Mortality Indicator RAMI Published by CHKS Published ?? Key differences –HRG not first episode –Excludes spells with Palliative Care

24. Key Differences AttributeSHMIHSMR Hospitals includedExcludes specialist, MH & independent All providers (SUS) Basket of conditionsNo exclusionsBasket of 56 diagnostic groups which relate to c. 80% of activity Death attributed to…Trust that patient died in or was discharged from (30 days) All trusts involved in the patient’s care during the ‘super-spell’ Out-of -hospitalIncludes deaths up to 30 days post discharge Not included Palliative CareNo adjustment madeRelative-risk weighting applied DeprivationProxy via co-morbidity [?]Relative-risk weighting applied

25. How SHMI is calculated SHMI Specification 1.14 Observed deaths Expected deaths Outliers

26. Observed deaths HES data [from CDS; from the Trust] Patients dying in hospital –Discharge method = 4 (therefore, not Stillbirths) –Excluding daycases, regular day attenders, regular night attenders [Classpat = 2,3 or 4] Patients that died within 30 days of discharge –ONS data; linked to HES data

27. How SHMI is calculated SHMI Specification 1.14 Observed deaths Expected deaths

28. Expected Deaths (1) All discharges (apart from noted exclusions) Split them up into all possible combinations of risk factors (i.e. stratify) –How many groups? GroupNumber Admission Method3 Age21 Charlson Index3 Gender3 Year3 CCS Group140 Total238,140

29. Expected deaths (2)

30. Bottom 10 and top 10 Intercept Values

31. Groups of Codes ICD10 Codes 16,000 CCS Categories 260 CCS Groups 140

32. Groups of Codes in practice Primary diagnosis of first episode [FFCE] Unless primary diagnosis is an ‘R’ code; in which case use primary diagnosis of 2 nd FCE Unless this is also an ‘R’ code; in which case revert to the primary diagnosis of the FFCE !!!!

33. Charlson Co-morbidity Index All Secondary diagnoses of first episode [FFCE] Unless primary diagnosis is an ‘R’ code; in which case use secondary diagnoses of 2 nd FCE Unless this is also an ‘R’ code; in which case revert to the secondary diagnoses of the FFCE !!!!

34. Outliers Data are put into a funnel plot

36. Outliers Data are put into a funnel plot Poisson distribution Over-dispersion [page 16] 3 bands

37. Interpretation (SHMI) 3 bandings (when compared to England average) –Higher than expected –As expected –Lower than expected

38. Interpretation (SHMI) 3 bandings (when compared to England average) –Higher than expected –As expected –Lower than expected Trends

42. Interpretation (SHMI) 3 bandings (when compared to England average) –Higher than expected –As expected –Lower than expected Trends What is driving a change? –Observed or Expected –CCS Groups, week-end and the rest…

43. Observed or Expected?

44. Interpretation (SHMI) 3 bandings (when compared to England average) –Higher than expected –As expected –Lower than expected Trends What is driving a change? –Observed or Expected –CCS Groups, week-end and the rest…

45. Myth-busters We’re in a highly deprived area We have some data recording issues, that’s all We provide high levels of Palliative Care We have audited every death and found no issues We have a high proportion of [SHMI] deaths that are out of hospital CCS Groups are useful The London effect

46. Deprivation (1) SHMI makes no adjustment for social deprivation –It might create the impression that a higher death rate for those who are more deprived is acceptable and has the potential to remove from the SHMI some of the differences that it is designed to measure

47. Deprivation (2)

48. Deprivation (3)

49. Deprivation (4)

50. Data/coding issues (1)

51. Data/coding issues (2) R codes

52. Palliative Care (1) SHMI: not adjusted for –Included in published “Contextual” Data HSMR: adjusted for RAMI: Spells discounted

53. Palliative Care (2)

54. Audit of deaths Why do it? –Learn lessons –Check coding Expected deaths based upon all discharges CCS Group analysis PRISM2 –Nick Black et al. –2,000 case-note reviews, nationally

55. Out of hospital deaths (1)

56. Out of hospital deaths (2) It is desirable to have fewer deaths –It is desirable to have fewer deaths in hospital –It is desirable to have fewer deaths <30 days of discharge Does the ratio of the 2 figures matter?

57. Out of hospital deaths (3) 25% 44%9%

58. Out of hospital deaths (4) Factors causing a higher rate –Patient discharged too early –Poor care in the community –Good hospice care / support to die at home Factors causing a lower rate –The opposite of the above –Good systems to prevent unnecessary admissions, especially in those likely to die Desirable things could cause a higher rate or a lower rate – so, it doesn’t matter.

59. Out of hospital deaths (5)

60. CCS Groups 140 Groups Alerts Relies on accurate coding Identifies clinical areas to look at… …or does it? Congruence with death certificate

61. The London effect

62. Any questions?

63. Pack Quiz – with answers SHMI Specification Latest quarterly mortality report PRISM2 review form Palliative Care coding – first 2 pages ccvol7issue4 Palliative Care Coding.pdf

64. Portal Avoidable mortality –Amenable to healthcare –Preventable deaths Lessons Learned + checklist

65. Next time… CUSUM alerts VLADs GP/CCG SHMI !!! Severe Winter!!!

66. Next Steps … What do you want from AQuA? CCG co-design session 16 th July (29 th July) ? other masterclasses Handouts & Links post session