1. Jeff Michalski, MD, MBA, FACR Professor and Vice-Chair Department of Radiation Oncology Washington University School of Medicine

2. Modern RT Recent sophistication – large fraction of modern treatment technology and practices developed in the past ten years High technical complexity Multiple systems (software and hardware) Limited to non-existent guidance/standards High pressure Increased potential for catastrophic failures

3. Current Processes Legacy processes and organization High reliance on manual checks and relatively poor use of electronic tools and automation Relatively Minimal Standardization Machine commissioning example – Reliance individual on facilities Almost Non-Existent Benchmarking Treatment planning example – Utilization of technology Poor quantification of processes

4. Error spectrum Publicized - One side of the spectrum, usually large dosimetric errors – NY Times Articles Semi-publicized – RPC data Approximately 30% of participating institutions fail to deliver IMRT dose indicated in their treatment plans to within 7% or 4mm to an anthropomorphic phantom (IJROBP. 2008;71(1 Suppl):S71-5). Unpublicized/unnoted – everyday occurrences “Small” dosimetric errors and geographic misses Suboptimal treatment plans (contouring and dose distributions) Care coordination issues Unnecessary treatment delays Jensen, Nellemann, and Overgaard, Tumor progression in waiting time for radiotherapy in head and neck cancer. Radiother Oncol, 2007, 84.

5. Errors in Radiation Oncology Staff and public exposures Suboptimal treatments Misadministrations Underdose Overdose Anatomical misses Magnitude From few percent to lethal doses From couple of millimeters to complete misses Regulatory Nuclear Regulatory Commission Errors that do not necessarily affect patients but have regulatory/legal consequences Sources Staff Software Hardware Random Affect one to few patients Systematic Affect hundreds of patients Potentially in a short period

6. Error Reporting in Radiation Oncology Use reported errors and near misses to develop safer software, hardware, tools, and processes

7. Background Global Problem “…it calls into question the integrity of hospital systems and their ability to pick up errors and the capability to make sustainable changes.” Sir Liam Donaldson, Chief Medical Officer, Department of Health Towards Safer Radiotherapy. London: The Royal College of Radiologists, 2008. Radiotherapy Risk Profile, Geneva: World Health Organization, 2009.

9. Systems Engineering in Health Care An outline for use of Systems Engineering for improvement of national health care system National Academy of Engineering and Institute of Medicine, 2005 “We often call this arrangement a “health care system” even though it was never created as a system and has never performed as a system.”

10. Error Reporting We are not airline industry nor nuclear power Perfection in complex systems across hundreds of diverse clinics is impossible Reporting systems for sake of reporting alone are a great way to squander resources and demoralize staff Error reporting as a part of broader process improvement efforts can be very valuable

11. Event Reporting Mandatory (statutory) Reporting required by law NRC in U.S. State requirements Mainly concentrated on well defined treatment delivery errors Guidelines for near-miss reporting typically not provided Voluntary Mainly at institutional level Some states in the U.S. have voluntary reporting systems – utility for rad onc not clear Errors and near misses tracked

12. Voluntary Reporting Dependent on Many Factors Culture Reporting guidelines Reporting system Competence to interpret reported data Willingness to implement, when necessary, significant changes based on collected data and subsequent analyses Ability to share the collected data and provide feedback

13. Reporting Culture Indemnity against disciplinarily proceedings and retribution Confidentiality To the extent practical, separation of those collecting the event data from those with the authority to impose disciplinary actions An efficient method for event submission A rapid, intelligent, and broadly available method for feedback to the reporting community

14. Taxonomy and Event Classification Event reporting should enable process improvement This requires efficient processing and analysis of data Submitted events must be classified and organized Enables efficient processing, analysis, and communication of data and trends

15. Web-Based Reporting

16. System Acceptance Paper

17. System Acceptance Voluntary Web-based Mutic et al, Submitted for publication, Med Phys, May 2010

18. System Acceptance Voluntary Web-based Mutic et al, Submitted for publication, Med Phys, May 2010

19. Therapy Nursing Physics Simulator Dosimetry

20. What are we doing? Organizational learning Error and near miss reporting Use reported events to improve processes and clinical tools Standardization QAIS QC – Based on reported errors and risk analysis Electronic Automated Intelligent Workflow tools - Based on reported errors and risk analysis

21. Electronic Chart Check Work Flow RT TPS Database OQA Database R&V Database Query Scripts dcm rtog ascii txt EcCk TPS & RV Prescription Field Parameters Consistency Reports etc Warning and Alerts Pending work Delivery problem indicators Independent checks Protocol Compliance Data Approvals, etc Statistics Benchmarking Future system improvements

22. Sample EcCk Report

23. Precheck scripts - workflow

24. Precheck: sample report

25. Plan Quality Benchmarking Moore et al, Submitted for publication, IJROBP, May 2010

26. Dynalog-based IMRT QA Report SummaryDelta FluenceMLC Graph Error Historical Performance of MLC

27. Tomotherapy: Sinogram based QA

28. The downside to the electronic world As implemented today Record and Verify (R&V) system was originally designed to operate as an independent system (Big Brother) Today these systems are integral part of the delivery process and the independent verification process is missing If data in the R&V system is wrong there is much less opportunity and chance that the error may be discovered

29. Possible Solution Electronic QA system (EQS) Independent system which compares TPS data with the data in the R&V system Greatly improves ability to compare initial data transfer and consistency of data in the R&V

30. Distributed Data Collection Each clinic with its own independent database Centralized Database ManufacturersRegulatory AgenciesProfessional Societies

31. Conclusion Error reporting in technologically advanced healthcare presents an opportunity to improve patient safety Independent oversight of established systems is desirable to monitor safety systems and implement immediate intervention Centralized reporting of errors and similar events (near misses, latent errors) is highly encouraged Use reported errors to improve commercial systems