1. Portsmouth Hospitals NHS Trust Dr David Prytherch and Dr Jim Briggs Health Care Computing Group University of Portsmouth Mr Paul Weaver and Dr Paul Schmidt, University of Portsmouth Professor Gary Smith, Portsmouth Hospitals NHS Trust, University of Bournemouth Measuring clinical performance using routinely collected clinical data.

2. Portsmouth Hospitals NHS Trust ………Hospitals and the NHS could tell you about throughput (number of patients treated), bed occupancy (the proportion of beds occupied in the hospital), and, latterly, the costs involved. But, generally speaking, quality of outcome was a closed book. Why look at Clinical Outcomes? At national level, the indicators of performance should be comprehensible to the public as well as to healthcare professionals. They should be fewer and of high quality, rather than numerous but of questionable or variable quality. Why Mortality? “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995. I Kennedy, HMSO 2001

3. Portsmouth Hospitals NHS Trust …… Variables such as case mix and where possible, in the case of surgery, operative risk must be allowed for, so that, wherever feasible, it is possible to compare like with like. “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995. I Kennedy, HMSO 2001 Why case-mix adjust? For the future the multiple methods and systems for collecting data must be reduced. Data must be collected as the by-product of clinical care. How to collect the data?

4. Portsmouth Hospitals NHS Trust Measuring Clinical Performance How do you measure clinical performance? You need to know what you expect – predicted outcomes. Compare predicted with reported. How do you predict outcomes? Case mix adjusted models Why case mix adjust? To gain clinician engagement To answer “my results are worse than … because my patients are sicker” Essentially it provides a ruler

5. Portsmouth Hospitals NHS Trust How do you case mix adjust?  Use clinical data that encapsulates the physiological state of the patient  Use this to predict a risk of “adverse outcome”  Trick is to collect necessary data in the clinical environment

6. Portsmouth Hospitals NHS Trust Aim of study was to see if data stored in core hospital systems could be used to predict (case mix adjust) clinical outcomes. Data from:PAS Biochemistry and Haematology modules of pathology system Data already collected / exists. No additional administrative or clinical burden. BHOM: Biochemistry and Haematology Outcome Modelling

7. Portsmouth Hospitals NHS Trust results adapted from: The use of routine laboratory data to predict in-hospital death in medical admissions D R Prytherch, J S Sirl, P Schmidt, P I Featherstone, P C Weaver, G B Smith. Resuscitation 2005; 66: 203-207 First demonstration of outcome prediction for General Medicine Data from PAS and Biochemistry and Haematology modules of pathology system 1st January 2001 - 31st December 2001 9497 discharges from GM with necessary data Model developed from Q1 and applied prospectively against Q2, Q3 and Q4 e.g., BHOM in General Medicine

8. Portsmouth Hospitals NHS Trust Urea Albumin Creatinine Na K Haemoglobin White Cell Count Age on admission Sex Mode of admission Mortality at discharge Data items used in models for General Medicine:

9. Portsmouth Hospitals NHS Trust Risk (%)Dis- charges Mean Risk (%) Predicted Deaths Reported Deaths 22  0 to  5 12792.272935 1.27 >5 to  7.5 2656.491720 0.48 >7.5 to  10 3218.862824 0.76 >10 to  12.5 16411.481920 0.08 >12.5 to  15 16914.002431 2.65 >15 to  20 15417.782728 0.02 >20 to  25 7222.901614 0.49 >25 to  33 7828.692220 0.36 >33 to  50 2840.25118 1.59 >50 to  100 1464.2198 0.31  0 to  100 25448.01204208 8.00  2 = 8.00 10 d.f P = 0.63 no evidence of lack of fit c-index=0.757 General Medicine Study Final 3 month period 1 st October – 31 st December 2001

10. Portsmouth Hospitals NHS Trust General Medicine Study Total Mortality through time – 1 st January 1998 to 31 st December 2001 (37283 discharges)

11. Portsmouth Hospitals NHS Trust Use to identify periods when performance deviates from norm

12. Portsmouth Hospitals NHS Trust Risk (%)Dis- charges Mean Risk (%) Predicted Deaths Reported Deaths 22  0 to  5 11031.401513 0.39 >5 to  7.5 1196.1676 0.26 >7.5 to  10 858.6676 0.28 >10 to  12.5 6811.29810 0.79 >12.5 to  15 3213.6746 0.70 >15 to  20 6617.341112 0.03 >20 to  25 5022.6411 0.01 >25 to  33 6129.201820 0.38 >33 to  50 9040.713736 0.02 >50 to  100 9374.586978 4.24  0 to  100 176710.68189198 7.09  2 = 7.09 10 d.f P = 0.72 no evidence of lack of fit c-index=0.92 General Medicine Study External Validation: T2 v T1

13. Portsmouth Hospitals NHS Trust  Clinical data obtained from a single venesection  Clinical data are used operationally in care of individuals  All data already stored on hospital core IT systems - no “extra” effort is required to collect data  Clinical data used are subject to extensive quality assurance Key points: 1

14. Portsmouth Hospitals NHS Trust  Case mix adjusted and uses high quality data trusted by clinicians (no coded data) – more likely to win clinical acceptance  Data immediately available to inform decisions  Cannot be “gamed”  Performance and surveillance tool Key points: 2

15. Portsmouth Hospitals NHS Trust National Application of BHOM Vascular Society of Great Britain and Ireland National Vascular Database (Risk adjusted predictive models of death after index arterial operations using a minimal data set. D R Prytherch, BMF Ridler, S Ashley on behalf of the Audit and Research Committee VSGBI, Br J Surg 2005; 92: 714-718) NCEPOD ( National Confidential Enquiry into Patient Outcome and Death ) study into Abdominal Aortic Aneurysm www.ncepod.org.uk

16. Portsmouth Hospitals NHS Trust Portsmouth NHS R&D Consortium Portsmouth Hospitals NHS Trust University of Portsmouth Sources of funding