1. THE LIVERPOOL SYSTEM - CLASSIFICATION, LEARNING & PREVENTION Incident Reporting and Learning: Anthony Arnold Director Cancer Services, Illawarra Shoalhaven Local Health District Anthony.Arnold@sesiahs.health.nsw.gov.au

2. --- ROSIS Melbourne Australia 2012 --- 2

3. 3 Ref: IJROBP 2010 Volume 78, No 5, Pages 1548-1554

4. --- ROSIS Melbourne Australia 2012 ---  Complexity of radiation oncology  At the time no system of analysis was in place  Lack of clinical governance surrounding reporting  There was limited openness about reporting events  The culture was predominantly blame based  Standard reporting systems are ineffective for radiation oncology 4

5. --- ROSIS Melbourne Australia 2012 --- PrescriptionSimulation Computing / Dosimetry Pre-TmtTreatmentImagingBolusShieldingDocumentation 5

6. 6 Prescription Simulation Computing Pre- Treatment Treatment Bolus Shielding / MLC Imaging Documentation

7. --- ROSIS Melbourne Australia 2012 --- 7

8.  Event:  “event or circumstance which could have resulted, or did result in harm to a patient”  Actual Error:  “Error resulting in radiation exposure other than that intended or prescribed – correctable or otherwise”  Near Miss:  “Error or non-conformance detected before reaching the patient” 8

9. --- ROSIS Melbourne Australia 2012 --- PLAN Classification designed, database constructed, education System implementation, clinical leadership and support DO Staff encouraged to report all events irrespective of magnitude Supporting governance, openness, process based STUDY Summary reports analysed monthly across various forums Trend patterns analysed to highlight areas / systems in need ACT The data itself was used to focus QA and improvement activity Focussed education, workflow redesign, protocol changes 9

10. --- ROSIS Melbourne Australia 2012 --- Detect Staff detecting initiates report (narratives, tells story) Manage patient and situation, immediate actions Review Team review, contributing factors, further actions Agree on report as a team Share Reverse back through other staff and depts involved Learning, prevention, further analysis, additional factors Manage Review and classifiy, explore issues, system breaks Consider recommendations, initiate change / improvements 10

11. --- ROSIS Melbourne Australia 2012 --- Liverpool Macarthur CTC (2004-2007) Illawarra CCC (2006-2009) 4-5 linear accelerators Superficial / orthovoltage Brachytherapy Widespread conformal 3DCRT IMRT on horizon Large metropolitan centre 688 reports / 3925 courses 2 linear accelerators Superficial / orthovoltage No brachytherapy Widespread conformal 3DCRT IMRT widespread clinical use Small semi-regional centre 670 reports / 3645 courses 11

12. --- ROSIS Melbourne Australia 2012 ---  688 reports were logged during the study period  155 Actual errors (23%)  533 Near Miss(77%) 12

13. --- ROSIS Melbourne Australia 2012 ---  670 reports were logged during the study period  67 Actual errors (10%)  603 Near Miss(90%) 13

14. --- ROSIS Melbourne Australia 2012 --- Actual Error Near Miss Total Errors No. of Attendances % Actual Error p-value for Actual Error Rate Difference % Near Miss p-value for Near Miss Rate Difference % Total Events p-value for Total Errors Rate Difference Year 1 63184247217880.3%0.8%1.1% Year 2 58199257381340.2%p<0.010.5%p<0.00010.7%p<0.0001 Year 3 34150184550060.1%p<0.0010.3%p<0.00010.3%p<0.0001 Ref: Simple Interactive Statistical Analysis online statistical calculator. Available at: http://www.quantitativeskills.com/sisa/statistics/t-test.htm. Accessed 29 January 2008 14

15. --- ROSIS Melbourne Australia 2012 --- Actual Error Near Miss Total Errors No. of Attendances % Actual Error p-value for Actual Error Rate Difference % Near Miss p-value for Near Miss Rate Difference % Total Events p-value for Total Errors Rate Difference Year 1 1614516162210.26%-2.33%-2.59%- Year 2 12173185156870.08%0.0016*1.10%p<0.0001*1.18%p<0.0001* Year 3 27128155170280.16%0.1695*0.75%p<0.0001*0.91%p<0.0001* Year 4 12157169155820.08%0.0017*1.01%p<0.0001*1.08%p<0.0001* Ref: Simple Interactive Statistical Analysis online statistical calculator. Available at: http://www.quantitativeskills.com/sisa/statistics/t-test.htm. Accessed 29 January 2008 15

16. 16

17. 17

18. --- ROSIS Melbourne Australia 2012 --- StudyTime period Actual Error Rate per treatment episode Total Error Rate per treatment episode† Reporting scope Comments SimulationPrescriptionPlanning Treatment delivery Our study 2004-050.3%1.1% 2005-060.2%0.7% 2006-070.1%0.3% Macklis et al. [17] 1995 0.2% per treatment field NR x Block errors most common Fraass et al. [24] 96-970.4%NR xxx Treatment set- up and treatment accessory errors most common Huang et al. [22] 97-020.3%NR xxx Tight parameters for error. Treatment field errors of >0.5cm the most common. Calandrino et al. [19] 91-96 0.45% per treatment course 3.5% per treatment course xx x MU calculations only Barthelem- Brichant et al. [27] NR3.5%NR xxx Patton et al. [6] 99-000.2%NR xxx Swann- D’Emilia [25] 89-90 0.17% per treatment field NR xxx Most errors were due to errors in block placement 18

19. --- ROSIS Melbourne Australia 2012 --- Reduction in Errors Reduction in Error Rate Improved Patient Safety 19

20. --- ROSIS Melbourne Australia 2012 ---  REDUCTION IN REPORTED EVENTS as a function of attendances  Actual Error rate reduced from 0.26% to 0.08% (p=0.0017)  Near Miss rate reduced from 2.33% to 1.01% (p<0.0001)  IMPROVED RELATIVE PATIENT SAFETY RISK per treatment course  Actual error rate reduced from 1 in 19 courses to 1 in 75 courses; in other words from 5% down to 1.3% risk of detectable error (p=0.0003)  Near miss rate reduced from 1 in 2 courses to 1 in 6 courses; in other words from 50% down to 17% (p<0.0001) 20