1. Use of Simulation-based Surgical Education and Training within the Context of the Core Competencies, Milestones, Patient Safety, and the New ACGME Accreditation System: Daniel J. Scott, MD, FACS Frank H. Kidd, Jr., MD, Distinguished Professorship in Surgery Vice Chairman and Surgery Residency Program Director Director, Southwestern Center for Minimally Invasive Surgery Frank H. Kidd, Jr., MD, Distinguished Professorship in Surgery Vice Chairman and Surgery Residency Program Director Director, Southwestern Center for Minimally Invasive Surgery University of Texas Southwestern Medical Center at Dallas GENERAL SURGERY

2. Patient safety –Institute of Medicine: To Err Is Human. National Academy of Science, Sept. 1999 98,000 deaths per year due to medical error “Error in the performance of an operation” cited as one of many causes New procedures –Advanced laparoscopy –Endovascular –Endolumenal “Orphan” procedures –e.g. open CBDE, PUD Why is Skill Acquisition an Issue?

3. Cost –$53 million annually to train residents in the OR 1 OR time limitations Inadequate case volumes 80-hour work week constraints 16-hour R1 duty hour limit Why is Skill Acquisition an Issue? 1 Bridges, et al. Am J Surg 1999;177:28-32.

4. UT Southwestern 1998

5. Randomized, blinded, controlled trial 5 hours of videotrainer practice using “Southwestern Stations” Performance measured during laparoscopic cholecystectomy  Training improves operating room performance J Am Coll Surg 2000;191:272-283.

6. Skills Training: Gaining Momentum 2004 ACS/SAGES FLS Program Validated with high-stakes certification 2006 ACS Accredited Education Institutes Standards for Centers of Excellence 2007 ACS/APDS National Skills Curriculum High quality simulation exercises with free access 2008 RRC Mandate Mandatory skills labs in all residency programs 2010 ABS FLS Requirement FLS Certification required to sit for qualifying examination Surgery RRC

7. Perfect Storm

9. Immersion

10. Immersion Autonomy

11. Immersion Autonomy Teaching

12. Immersion Autonomy Teaching COMPETENCY

13. Are Graduating Residents Competent? 2012 American Board of Surgery (ABS) Pass Rates –Written Boards: 81% www.abs.org

14. Are Graduating Residents Competent? 2012 American Board of Surgery (ABS) Pass Rates –Written Boards: 81% –19% of graduates have knowledge deficiencies www.abs.org

15. Are Graduating Residents Competent? 2012 American Board of Surgery (ABS) Pass Rates –Written Boards: 81% –19% of graduates have knowledge deficiencies –Oral Boards: 72% www.abs.org

16. Are Graduating Residents Competent? 2012 American Board of Surgery (ABS) Pass Rates –Written Boards: 81% –19% of graduates have knowledge deficiencies –Oral Boards: 72% –28% of graduates have judgment deficiencies www.abs.org

17. Are Graduating Residents Competent? 2012 American Board of Surgery (ABS) Pass Rates –Written Boards: 81% –19% of graduates have knowledge deficiencies –Oral Boards: 72% –28% of graduates have judgment deficiencies 28% OF GRADUATES ARE NOT COMPETENT www.abs.org

18. Are Graduating Residents Competent? >80% of 1000 graduating residents enroll in fellowships –Lifestyle –Financial reward –Job opportunities in urban areas –Narrow field of expertise is appealing

19. Are Graduating Residents Competent? >80% of 1000 graduating residents enroll in fellowships –Lifestyle –Financial reward –Job opportunities in urban areas –Narrow field of expertise is appealing –Not comfortable with their level of competency

20. Are Graduating Residents Competent? >80% of 1000 graduating residents enroll in fellowships –Lifestyle –Financial reward –Job opportunities in urban areas –Narrow field of expertise is appealing –Not comfortable with their level of competency –DO THEY NEED MORE TRAINING?

21. Start Training during Medical School “The American Board of Surgery recommends and endorses that all incoming surgical residents beginning with the 2014 year complete a preparatory course PRIOR to beginning Surgical Training.”

22. More General Surgery Training after Residency ACS Transition to Practice Program –6 Programs starting in 2013 –“Help with your transition to independent practice in general surgery” –“Increase your competence and confidence in clinical practice” www.facs.org

23. www.absurgery.org/xfer/fyp2012summary.pdf UT Southwestern6095% xxxxxxxxxxxxxxxx2986% xxxxxxxxxxxxxxxx3184% xxxxxxxxxxxxxxxx2789% xxxxxxxxxxxxxxxx2886% xxxxxxxxxxxxxxxx3485% xxxxxxxxxxxxxxxx2588% xxxxxxxxxxxxxxxx4286% xxxxxxxxxxxxxxxx3281% xxxxxxxxxxxxxxxx4185% xxxxxxxxxxxxxxxx2588% xxxxxxxxxxxxxxxx2986% xxxxxxxxxxxxxxxx3566% xxxxxxxxxxxxxxxx2993% xxxxxxxxxxxxxxxx3884% xxxxxxxxxxxxxxxx4484% xxxxxxxxxxxxxxxx3394% xxxxxxxxxxxxxxxx3574% xxxxxxxxxxxxxxxx5178% xxxxxxxxxxxxxxxx3491% xxxxxxxxxxxxxxxx2588% xxxxxxxxxxxxxxxx2584% UT Southwestern Results 5-year ABS Pass Rates - First Attempt Both Components

24. General Surgery Residency Program Balance Immersive Experience Structured Education

25. Technical Skills Curricula General: –Basic Open Knot-tying and Suturing –Intermediate Open Knot-tying and Suturing Trauma/Critical care: –Surgical Airway –Basic Ventilator Management –Advanced Ventilator Management –Ultrasound for Central Line Placement –Ultrasound in the ICU –Ultrasound for Trauma (FAST) –Hemodynamic Monitoring –ATOM: Advanced Trauma Operative Management –Tracheostomy & PEG Tube Insertion Breast: –Percutaneous Breast Biopsy –Ultrasound Guided Breast Biopsy –Mammosite Module Vascular: –Vascular Patch Angioplasty –Vascular Anastomosis –Vascular Access Laparoscopic/Endoscopic: –Lap I Basic Laparoscopic Skills –Lap II Basic & Advanced Laparoscopic Skills (FLS) –Fundamentals of Laparoscopic Surgery (FLS) Ongoing Training –Laparoscopic Common Bile Duct Exploration (LCBDE) –Laparoscopic Cholecystectomy & Appendectomy –Laparoscopic Port Insertion –Laparoscopic Energy Sources –Laparoscopic Inguinal Hernia –Virtual Reality Flexible Endoscopy –Surgeon’s Training Endoscopic Proficiency (STEP) –SAGES Top 21 –SAGES Biliary Grand Rounds Robotic Training: –Comprehensive Console Surgeon Basic Training

26. Optimizing Skill Acquisition Deliberate practice –Separated from the real environment (OR) –Specifically designed plan for training –Goal oriented motivation for learning Structured training –Protected time allocation –Mandatory participation –Performance and attendance tracking Distributed training –Maximum 1-1.5 hours per session –Distributed over days/weeks/months –Superior to massed practice Alleviates fatigue Reinforces mapping/integration Ericsson KA. Acad Med 2004;79:S70-S81. Moulton, et al. Ann Surg 2006;244:400-409. Mackay, et al. Surg Endosc 2002;16:957-961.

27. Optimizing Skill Acquisition Proficiency-based training –Avoids the use of arbitrary training endpoints Such as predetermined duration or number of repetitions “One size does not fit all” –Tailors the educational experience to the needs of the learner Practice a little vs. a lot Optimizes efficiency and time requirements Maximizes effectiveness  uniform end product (skill acquisition) Seymour, et al. Ann Surg 2002;236:458-463. Brunner, et al. J Surg Research 2004;122:150-156. Korndorffer, et al. Arch Surgery 2005;140:80-84. Stefanidis, et al. Surgery 2006;140:252-262.

28. Open Skills Curriculum: 12 Tasks Palm Needle Driver Knot-tying, No Tension Knot-tying, Tension Suturing, InterruptedSuturing, Running

29. R1 (n=37) Proficiency achieved for 100% of the 12 tasks –12.7 hours (range 6.9-37.1) –141 repetitions (range 52-513) Cost: $776 Open Skills Curriculum p <0.001 Goova, et al. J Surg Educ 2008;65:309-15

30. Tie at Depth Suturing at Depth, Interrupted simple Tie on a Pass Suture Ligation (Stick tie) Ties in Continuity Atraumatic Tie Open Intermediate Skills Curriculum: 6 Tasks

31. PGY1 (n=39) Proficiency reached 3.4 ± 3.8 hours 29 ± 17 repetitions Cost $4 per trainee Open Intermediate Skills Curriculum Mashaud, et al. Surgery (in press)

32. Proficiency-based Curriculum Intracorporeal Extracorporeal Ligating Loop Pattern Cut Peg Transfer PGY1 (n=37) 2006-2007 Cognitive & Skills: Lap I + Lap II (FLS) FLS –7.4 ± 3.2 hours, 67 ± 28 repetitions 78% & 100% FLS exam pass rates after Lap I & II Passing Score = 270 P < 0.001 Lap I Lap II Goova MT, et al. J Surg Research 2008;144: 224-225.

33. PGY2-5 (n=54) 2006-2007 FLS Skills –6.2 ± 2.6 hours –103 ± 28 repetitions 63% passing at baseline 100% FLS exam pass rate Intracorporeal Extracorporeal Ligating Loop Pattern Cut Peg Transfer Goova MT, et al, J Am Coll Surg 2008; 207: S90. Passing Score = 270 Proficiency-based Curriculum

34. Ongoing FLS Training Protocol Task 4 & 5 Testing (1 rep) Re-training if indicated Ongoing FLS training –96% participation rate (all curricular components over 2-year period) 2006-2007 Initial Proficiency-based FLS Training PGY1-5 (n=91) 2007-2009 Ongoing Proficiency-based FLS Training PGY3-5 (n=33) 6 moRetention I 6 moRetention II 6 moRetention III 6 mo Retention IV PGY4-5 Certification (n=20) 2-Year Analysis Mashaud, et al. Surgery 2010;148:194-201.

35. Task 4: Performance Improvement and Retention (n=33) PGY 3 PGY 4 PGY 5 91% at 23.3 months 98% at 18 months 94% at 12.3 months 83% at 6.3 months Ongoing FLS Training Results Mashaud, et al. Surgery 2010;148:194-201.

36. Task 5: Performance Improvement and Retention (n=33) PGY 5 PGY 3 PGY 4 100% at 23.3 months 96% at 18 months 95% at 12.3 months 85% at 6.3 months Ongoing FLS Training Results Mashaud, et al. Surgery 2010;148:194-201.

37. Passing Score Certification Exam PGY 4-5 (n=20) Ongoing FLS Training 92% Retention at 23.3 months Mashaud, et al. Surgery 2010;148:194-201.

38. Comprehensive Proficiency-based Robotic Training Curriculum Curriculum Design –23 unique skills identified through task deconstruction –Residents, fellows, faculty –General Surgery, Gynecology, Urology Curriculum Components –Online tutorial (MCQ’s, certificate) –½ day interactive session (global ratings) –9 inanimate exercises (objective metrics) Validation (Content, Face, Construct, Reliability) Dulan G, et al. Surgery 2012;152:477-88. Dulan G, et al. Surg Endosc 2012;26:1516-1521. Dulan G, et al. Am J Surg 2012;203:535-539.

39. Task 1. Peg Transfer Task 2. Clutch/Camera MovementTask 3. Rubber Band TransferTask 4. Suture (Simple) Task 5. Clutch/Camera Peg Transfer Task 6. Stair Rubber Band TransferTask 7. Running/Cutting Rubber BandTask 8. Pattern Cut Task 9. Suture (Running)

40. Performance Improvement (n=53) *p<0.05 Arain et al. Surg Endosc 2012;26:2740-5.

41. Ensuring Competency: Is FLS Certification Necessary for Practicing Surgeons? Melanie L Hafford MD Kent R VanSickle MD Ross E Willis PhD Todd D Wilson MD Kristine Gugliuzza MD Kimberly M Brown MD Daniel J Scott MD University of Texas Healthcare Safety and Effectiveness Grant Program

42. Pretest: n=83 –26 (31%) failed manual skills Training –Skills: 2.5 ± 2.3 hrs. –Cognitive: 2.6 ± 2.1 hrs Certification Examination: n=76 –1 (1%) failed manual skills –9 (12%) failed cognitive exam 6 remediated, passed on re-take FLS Certification for Faculty Surgeons Hafford, et al. Surg Endosc Surg Endosc 2013;27:118-26. P= <0.001

43. Conclusions Numerous threats to developing competency exist Simulation plays a pivotal role –Technical skills –Judgment (i.e. decision-making)? Performance benchmarks may facilitate competency assessments in a longitudinal fashion (i.e. milestones)

44. Thank You!