1. Graphical methods for turning data into information Martin Utley Clinical Operational Research Unit (CORU) University College London www.ucl.ac.uk/operational-research

2. Monitoring outcomes to improve outcomes Care process Data Information system Analysis of data Feedback Need to get every step right

3. Care process Data Information system Analysis of data Feedback Steps discussed in this talk

4. Case study 1: monitoring outcomes of cardiac surgery Work done by: Steve Gallivan Chris Sherlaw-Johnson Jocelyn Lovegrove Tom Treasure Oswaldo Valencia CORU St Georges / Guy’s

5. Mortality data for cardiac surgery 0000000000000000100000000 0000000000000000000000000 0000000000000100000000000 0000000100000000000000000 0000000000000000000100000 0000000000000000000001010

6. Mortality data for cardiac surgery 0000000000000000100000000 0000000000000000000000000 0000000000000100000000000 0000000100000000000000000 0000000000000000000100000 0000000000000000000001010 6 perioperative deaths in 150 cases

7. 020406080100120140 Operation number 0 2 4 6 8 10 Cumulative deaths Graphical presentation of data First used in the context of surgery by DeLeval

8. 020406080100120140 Operation number 0 2 4 6 8 10 Cumulative deaths Graphical presentation of data Is this series of outcomes good or bad?

9. Risk of perioperative death Patient factors that contribute to risk of death To be fair, assessment of outcomes should account for case-mix

10. 020406080100120140 Operation number 0 2 4 6 8 10 Cumulative deaths Expected mortality (from risk model) Actual mortality Net life gain Par for the course Compare outcomes to expectations

11. Variable Life Adjusted Display (VLAD) 020406080100120140 Operation number 0 1 2 3 4 5 Net life gain VLAD plot for a single surgeon

12. 020406080100120140 Operation number 0 1 2 3 4 5 -2 -3 -4 Net life gain Vlad the impaler The venerable bleed Hawkeye Pierce Comparing three fictitious surgeons Unexpected death Survivor against the odds

13. Comparing surgeons within a unit Net life gain Operation number

14. -1% -5% -10% -25% +25% +10% +5% +1% tail Tools to assist interpretation Net life gain Operation number

15. Keys to success Surgeons say that visual display is intuitive Can be used to identify possible problems in real time Monitoring tool “rewards” good outcomes rather than just punish bad outcomes Clinical champion VLAD adopted worldwide

16. Case study 2: monitoring prescription errors Collaborators: Steve Gallivan Christos Paschalides Bryony Dean Franklin Ann Jacklin Kara O’Grady Nick Barber CORU Hammersmith London School of Pharmacy Funded by the Trustees of Hammersmith Hospitals NHS Trust

17. Monitoring the prescribing process Care process Data Information system Analysis of data Feedback Junior doctor writes prescription Ward pharmacist corrects any errors that he or she identifies

18. Care process Data Information system Analysis of data Feedback Prescription errors deemed sufficiently serious by pharmacist are logged as incidents Monitoring the prescribing process

19. Care process Data Information system Analysis of data Feedback Extensive research on nature and rates of reported prescription errors Monitoring the prescribing process

20. Care process Data Information system Analysis of data No systematic feedback to prescribers With no feedback, how can we expect prescribing practice to improve? The problem

21. Feasibility study Ward pharmacist checks new medication orders......& records consultant team, number of new orders and all errors identified Data entered onto spreadsheet Junior doctors write prescriptions Graphical summaries prepared Feedback sent to head of specialty

22. 0 200 400 600 800 1000 1200 Specialty C Specialty New medication orders with at least one error New medication orders without any errors Number of new orders Your specialty 93 (12%) of 773 new orders had at least one error Graphical summaries kept simple

23. How much statistics? 95% confidence interval Specialty C

24. Performance over time Specialty C

25. Prototype feedback pages 1 and 2

26. Prototype feedback page 3 Comments written by the Trust’s Principal Pharmacist highlighting any issues that arise from the data. Representative examples of prescribing errors recorded over the period.

27. ...does this process lead to improvement? Care process Data Information system Analysis of data Feedback Care process Data Information system Analysis of data Feedback Hang on...

28. Planned study Does monitoring and feedback reduce errors in the prescribing process? Time Error rate Monitor and feedback results

29. Summary Succinct graphical methods can be very useful in the analysis of clinical data and in feeding back information to clinical teams. Appropriate feedback cannot do any harm, can it? Evaluation of monitoring systems in terms of clinical improvement is desirable.