1. Your life in their hands Mortality as a measure of clinical performance David Prytherch, Jeff Sirl, Paul Weaver, Paul Meredith, Michael Booth

2. ……… Hospitals and the NHS could tell you about throughput (number of patients treated), bed occupancy (the proportion of beds occupied in the hospital), and, latterly, the costs involved. But, generally speaking, quality of outcome was a closed book. Chapter 27 para 2 “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995 I Kennedy, HMSO 2001 Why look at Clinical Outcomes?

3. At national level, the indicators of performance should be comprehensible to the public as well as to healthcare professionals. They should be fewer and of high quality, rather than numerous but of questionable or variable quality. Recommendations para 153 “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995 I Kennedy, HMSO 2001 Why Mortality?

4. …… Variables such as case mix and where possible, in the case of surgery, operative risk must be allowed for, so that, wherever feasible, it is possible to compare like with like. Chapter 27 para 49 “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995 I Kennedy, HMSO 2001 Why Case-mix adjust?

5. For the future the multiple methods and systems for collecting data must be reduced. Data must be collected as the by- product of clinical care. Summary para 96 “Learning from Bristol”: The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984 to 1995 I Kennedy, HMSO 2001 How to collect the data?

6. What modelling provides: Stratified (=case mix adjusted model) enables:  Comparison of expected and observed outcomes  Comparison of outcomes / performance between hospitals and clinicians  Meaningful league tables based on clinical performance  Better understanding of the process of clinical care? Provides a ruler

7. Previous / existing models (General Surgery) require data beyond that routinely stored in “core” systems. Can useful models be constructed from the more limited data set stored in “core” systems? An interesting question

8. General Surgery

9. results adapted from: Towards a national clinical minimum dataset for general surgery. D. R. Prytherch, J. S. Sirl, P. C. Weaver, P. Schmidt, B. Higgins, G.L Sutton British Journal of Surgery, in print 5 year period, 1 st August 1997 to 31 st July 2002 28925 General Surgical in-patient episodes with necessary data Models constructed from years 1 and 2 and tested prospectively against years 3, 4 and 5

10. Data items used in BHOM models for General Surgery: Urea Na K Haemoglobin White Cell Count Age on admission Sex BUPA operative severity score Mode of admission – elective or emergency Mortality at discharge

12. Risk Bands (%) Dis- charges Mean Risk (%) Predicted Deaths Reported Deaths 22  0 to  5 22270.982224 0.23 > 5 to  10 2617.091921 0.36 > 10 to  15 11912.1314 0.01 > 15 to  25 9119.131720 0.48 > 25 to  50 5934.332019 0.12 > 50 to 1002464.171514 0.36 0  to 100 27513.91108112 1.56 General Surgery Study Final 6 month period 1 st February – 31 st July 2002  2 = 1.56, 6 d.f, P = 0.96, no evidence of lack of fit

13. General Medicine

14. General Medicine Study Data from HIS and Biochemistry and Haematology modules of pathology system 4 year period, 1 st January 1998 - 31 st December 2001 37283 discharges from GM with necessary data Models constructed from 3 months (Jan – Mar 01) and tested prospectively against the other 45 months of the study.

15. Data items used in BHOM models for General Medicine: Urea Albumin Creatinine Na K Haemoglobin White Cell Count Age on admission Sex Mortality at discharge

17. Risk (%)Dis- charges Mean Risk (%) Predicted Deaths Reported Deaths 22  0 to  5 12662.383034 0.50 >5 to  7.5 2816.521822 0.79 >7.5 to  10 3268.882924 0.93 >10 to  12.5 16011.471821 0.43 >12.5 to  15 17113.932429 1.31 >15 to  20 15817.722830 0.18 >20 to  25 7423.161713 1.30 >25 to  33 6728.5819 0.00 >33 to  50 2839.50119 0.63 >50 to  100 1364.4687 0.64  0 to  100 25447.99202208 6.71 General Medicine Study Final 3 month period 1 st October – 31 st December 2001  2 = 6.71 10 d.f P = 0.75 no evidence of lack of fit

18. How applicable are the models?

21. Conclusions Clinical data obtained from a single venesection Clinical data are used operationally in care of individuals No “extra” effort is required to collect data Clinical data used are subject to extensive quality assurance Available in all hospitals Candidate National Clinical Minimum dataset

22. Sources of funding Portsmouth NHS R&D Consortium Portsmouth Hospitals NHS Trust University of Portsmouth