Does the Rate of Achieving Competency on simulated Laparoscopy Training Programs Predict Actual Skill in Performing Laparoscopic Cholecystectomy? Warren D. Widmann, MD Mark A. Hardy, MD, Nancy Hogle, RN, MS, Akuezunkpa O. Ude, MD, and Dennis Fowler, MD New York Presbyterian Hospital, Columbia Campus Columbia University College of Physicians and Surgeons New York, New York

LapSim ® LapSim ® Basic Skills Camera Navigation Instrument Navigation Coordination Grasping Lifting and Grasping Cutting Clip Applying

Global Operative Assessment of Laparoscopic Skill GOALS depth perception bimanual dexterity efficiency tissue handling overall competence

Protocol 21 PGY1 residents All PGY1residents performed a laparoscopic cholecystectomy in a pig after minimal training; performance evaluated by skilled laparoscopic surgeons using GOALS 10 of the group trained to competency on the LapSim® All 21 PGY1 residents again performed laparoscopic cholecystectomies on pigs; performance evaluated by skilled laparoscopic surgeons using GOALS

Results Baseline scores, untrained, experimental group of residents2.18/5 Baseline scores, untrained, control group of residents2.05/5

Results Baseline scores, untrained, experimental group of residents2.18/5 Scores after training 2.77/5 Baseline scores, untrained, control group of residents2.05/5 Scores w/o training 2.24/5

Results Baseline scores, untrained, experimental group of residents2.18/52.77 –Time to complete training varied from 73 to 242 minutes Baseline scores, untrained, control group of residents2.05/52.24

Results Baseline scores, untrained, experimental group of residents2.18/52.77 Time to complete training varied from 73 to 242 minutes Baseline scores, untrained, control group of residents2.05/52.24 Time to complete later training varied from 60 to 239 minutes

Conclusion LapSim® training of novices improves the performance of a cholecystectomy in a pig

Conclusion LapSim® training improves the performance of a cholecystectomy in a pig by an untrained novice Not very surprising—LapSim® training has face validity, construct validity, and predictive validity Where do we go from here?

Do residents who learn quickly on a LapSim® program perform better than residents who learn at a slower rate?? We looked at the change in scores of the 4 fastest learners, baseline vs retesting after lapsim® competency Δ.75/5 in pigs The 4 slowest learners showed Δ.45/5 in pigs

Things Do Not Look So Good for the Slow Learners Have we truly tested for predictive validity in the clinical setting? Our model was the pig, our testing was done early in the PGY1 year Let’s take a look at actual performance when the now trained residents operated on patients during the next 12 months

Method Data was collected on 6 PGY1 categorical residents who performed 28 cholecystectomies during their PGY 1 year. Case loads varied from 2-10 per resident All were evaluated by skilled laparoscopic surgeons using the GOALS criteria

Results Fast Learners (68-79 minutes) 2.39/5 Slow Learners ( minutes) 2.53/5

Results Fast Learners (68-79 minutes) 2.39/5 Slow Learners ( minutes) 2.53/5 How come???

Characteristics of the Learners Prior Video Game Habit TimeEver> 3 hrs/wkCurrent user PigPerson Yes No Yes No Yes No Yes No

Conclusions Slow learners on the LapSim® were naïve gamers All residents learned to competency on LapSim®—some took appreciably longer than others The rate of learning and achieving competency on the LapSim® predicted performance on the pig early in the PGY1 year The rate of learning on the LapSim® failed to predict clinical performance throughout the rest of the PGY1 year

Characteristics of the Best Clinical Performer TimeEver> 3 hrs/wkCurrent user Average GOALS Yes No Yes No Yes No Yes No