1. Promise & limitations of surgical checklists: How can we effectively use them to improve the quality of surgical care? Shawn J. Rangel, MD, MSCE & J. William Sparks, MD Northeast Regional Patient Safety & Quality Improvement Conference February 5 th, 2011

4. WHO checklist

5. Reality check- IHI map

6. Even the stars are using it!

7. So, will the WHO checklist save mankind?

8. Reality check: one size does not fit all…

9. Review evidence supporting surgical safety checklists Attitudes toward the safety checklist at (CHB survey) Strategies for improving checklist utilization & relevance Next steps: IT, custom checklists & beyond… Outline of today’s discussion

10. WHO study

11. London, UK EUROEMRO WPRO I SEARO AFRO PAHO I Amman, Jordan Toronto, Canada New Delhi, India Manila, Philippines Ifakara, Tanzania WPRO II Auckland, NZ PAHO II Seattle, USA 8 Evaluation Sites

12. Methods 1 to 4 operating rooms targeted at each site 18-item checklist implemented (sign-in, time-out, sign-off) Pre-post intervention study design (general surgery cases) Primary outcome measure: aggregate 30-day major complication rate (NSQIP* defined) *National Surgical Quality Improvement Project

13. Results: impact on morbidity & mortality (3 month comparison periods) BaselineChecklistP value Cases3,7333,955- Death1.5%0.8%0.003 Any complication11.0%7.0%<0.001 Surgical site infection6.2%3.4%<0.001 Unplanned Reoperation2.4%1.8%0.047

14. Limitations of the study Unknown influence of the Hawthorne effect Unable to prove causality (non-randomized design) Effect size may be exaggerated (developing nations) Only one of the eight centers was in the U.S. Pediatric patients not included in analysis

15. SURPASS

16. Intervention The comprehensive “SURgical PAtient Safety System”:

17. Methods SURPASS implemented at 6 tertiary-care hospitals Pre-post intervention study design (3 month periods) 12 adverse event categories audited Outcomes compared with five “control” hospitals

18. Results: impact on morbidity & mortality (3 month comparison periods) Baseline (n=3,760) Checklist (n=3,820) P value Mortality1.5%0.8%- Any complication15.4%10.6%<0.001 Complications/100 cases27.3%16.7%<0.001 Use of the checklist was associated with a significant reduction in complication rates for 10 of the 12 (82%) event categories in the study

19. But, kids are not small adults!

20. Pediatric Safe Surgery Collaborative Children’s Hospital Boston Shawn J. Rangel (Study PI) Beth K. Norton (Co-study PI) Jessica Baxter Texas children’s Hospital Thomas Luerssen (site PI) Carrie Smith-Bruce Riley Children’s Hospital Fred Rescorla (Co-site PI) Charles Leys (Co-site PI) Margo Regas Denver Children’s Hospital Tammy Woolley (Site PI) Jenae Nieman Children’s Healthcare Atlanta Kurt Heiss (Site PI) Kawana Mitchell Children’s National Med Center Kurt Newman (Co-site PI) Rahul Shah (Co-site PI) Andrea Ewing-Thomas Children’s Hospital of Philadelphia Peter Mattei (Site PI) Lisa Czyzewski

21. Collaborative process Identification & recruitment of checklist champions Development of site-specific checklist Plan for piloting on small scale Obtaining buy-in from hospital leadership & peers Full implementation OR-wide Develop internal plan for auditing compliance

22. General study design Seven hospitals included as “checklist” implementers All inpatient procedures from 7 surgical services included Cardiac, General, Neuro, Ortho, Plastics, Oto & Urology Pre-post intervention comparison design (9 months) Primary endpoint: Aggregate 30-day adverse event rate PHIS database used to identify events Results compared against seven control hospitals

23. Patient characteristics (demographics) Checklist hospitals (n=7)Control Hospitals (n=7) Pre-intervention (n=19,867) Post-intervention (n=18,850) Pre-intervention (n=15,616) Post-intervention (n=15,298) Age (mean years) 6.97.17.27.4 Sex (% male) 55.354.256.556.6 Insurance status : Government (%) Private (%) Other (%) 62.5 24.8 12.7 63.2 24.6 12.2 50.2 39 10.8 53.9 39.7 7.3 Race/ethnicity: African American(%) Hispanic (%) 20.2 16.9 18.5 17.5 10.6 23.3 10.8 26 Case Mix Index 6.1 5.5 Acuity of procedure (% emergent) 15.618.918.819.6

24. Rate of any adverse event Incidence (%) P=0.064 P=0.364

25. MORTALITY RR of death with checklist utilization: 0.73 (95%CI: 0.57-0.93) Incidence (%) p=0.758 p=0.018

26. MORTALITY (High-risk specialties) Incidence (%) p=0.080 p=0.332 p=0.064

27. MORTALITY (emergent procedures) RR of death with checklist utilization: 0.58 (95%CI: 0.36-0.95) Incidence (%) p=0.724 p=0.029

28. MORTALITY (ICU admissions) RR of death with checklist utilization: 0.57 (95%CI: 0.40-0.82) Incidence (%) p=0.731 p=0.002

29. Limitations of the study Unknown influence of the Hawthorne effect Unable to prove causality (non-randomized design) Reliance on administrative data for outcomes analysis Variation/degree of checklist compliance unknown

30. What can be concluded from the available data? USE OF A SURGICAL SAFETY CHECKLIST CAN SAVE LIVES !!!!!!

31. So then, how do people feel about using a surgical safety checklist? --CHB Checklist Survey-- Multidisciplinary targeting (3-headed monster!) Assess attitudes towards the checklist Gain insight on CHB’s current safety culture Obtain feedback for improving checklist utility

32. Checklist survey: Responses by specialty (n=177)

33. Has the checklist improved safety? (response = yes) Proportion of responders (%) Chi 2, p=0.948

34. How has the checklist improved safety? Proportion of responders (%)

35. Have you witnessed an error or complication prevented by the checklist? (response=yes) Proportion of responders (%) Chi 2, p=0.048

36. Has the checklist improved efficiency? (response=yes) Proportion of responders (%) Chi 2, p=0.110

37. Would I want the checklist used for my child? (response=yes) Proportion of responders (%) Chi 2, p=0.122

38. Content adequacy of current checklist? Proportion of responders (%) Chi 2, p=0.987

39. So, is everyone in love with the checklist?? “This checklist is bullsh&! and just reinforces the Betty Crocker approach to medicine !!” “This is probably the most important surgical safety intervention we could ever implement !!”

40. “This is stupid- we do this all the time anyway” “This is stupid- the checklist does not apply to my cases” Blood products available? Imaging reviewed? Special equipment available? DVT prophylaxis considered? IV access adequate? Root causes of “checklist fatigue”

41. So then, how can we improve the effectiveness of our checklist? Implementation of forcing cues into work flow Incorporation of a more effective auditing system Transition to a “quality”-centered checklist paradigm Development of customized checklists

42. Change in the checklist paradigm: transitioning from “safety” to “quality” Surgical Quality Surgical Quality Value-based Efficient Effective Safe

43. Variation in the use of surgical antibiotic prophylaxis for common pediatric procedures

44. How can we accomplish these goals?

46. Take home lessons…. EFFECTIVE use of surgical checklists CAN SAVE LIVES! Checklist MUST be team-based and emphasize communication! Checklists HAVE to be developed with input from ALL stakeholders Leadership ABSOLUTELY has to be on board!

47. YOU ARE THE FUTURE OF SAFETY CULTURE!!! And finally….