Evaluating the Briefing, Intraoperative Teaching, Debriefing (BID) method for surgical skills teaching: a pilot study Daniel Hoppe MD MEd Oleg Safir MD FRCS(C) MEd Adam Dubrowski PhD

What is BID?

Intra-operative Teaching Journal of the American College of Surgeons (2009) Step Task Briefing Set learning objectives Intra-operative Teaching Teaching during the encounter Debriefing Reflection Rules Reinforce Correct

Why the BID?

WHAT LEARNERS WANT* WHAT ACTUALLY HAPPENS** Instructor “game-plan” Involvement in the learning process Ongoing and direct feedback WHAT ACTUALLY HAPPENS** “Opportunistic” teaching Passive transfer of information Limited feedback given *Vikis et al. (2008), Cox and Swanson (2001), Snyder et al (2012) **Scallon et al. (1992), Irby (1995), Roberts et al. (2008), Jefree and Clarke (2010)

Objective To evaluate whether the BID method is an effective framework for surgical skills teaching Learner reactions, satisfaction

Methods

BID Method Training Session First year orthopaedic residents (n=12) Allocated to control group (n=6) Allocated to BID group (n=6) PRE-TEST (OSATS) Learned Control Skill (Central Line) BID Method Training Session

BID Method Training Session First year orthopaedic residents (n=12) Allocated to control group (n=6) Allocated to BID group (n=6) PRE-TEST (OSATS) Learned Control Skill (Central Line) BID Method Training Session

BID Method Training Session 60 minute session Interactive - presentation, discussion, role- play Given to both instructor and BID group separately Learners trained to create synergistic learning effect* -evidence that suggests that both teacher and learner play a major role in the creation of an effective learning interaction *Aagaard et al. (2004), Manyon et al. (2003), Vikis et al. (2008)

BID Method Training Session First year residents (n=12) Allocated to control group (n=6) Allocated to BID group (n=6) PRE-TEST (OSATS) Learned Control Skill (Central Line) BID Method Training Session Learned Test Skill 1 (Bone Wedge) Learned Test Skill 2 (Bone Plating) POST-TEST (OSATS)

Change in individual OSATS scores - checklist and GRS Outcome Measures

Rizzolo (2006), National League for Nursing (2007) Outcome Measures Satisfaction Questionnaire, Educational Practices Questionnaire* Focus group data Rizzolo (2006), National League for Nursing (2007)

Standardization of Instructors Used a number of measures to try and ensure approximate instructor equivalency 1. Perception of Preparedness Questionnaire - no differences 2. “Assessing Teaching Skills” assignment (adapted from Ramsden et al. 1995) 3. “Control Skill” (Central Line Insertion) - taught to each group prior to BID training

Results

Assessing Instructor Equivalency Perception of Preparedness Questionnaires similar between both instructors also had similar teaching ideologies/experience -no formal educational training -no familiarity with educational theory -preferred small-group learning to lectures -feedback important (but neither knew framework for delivering feedback) -use questions to confirm understanding

Central Line Insertion OSATS Pre-test Data no significant differences in baseline pre-test scores between groups for checklist and GRS scores Skill GRS Checklist Central Line Insertion P=0.326 P=0.102 Bone Wedge P=0.478 P=0.536 Bone Plating P=0.927 P=0.263

Control Skill: Central Line Insertion Global Rating Scale Pre-Test Post-Test Control 46 96 BID 53 81 P=0.129

After BID training

Test Skill 1: Bone Wedge Cut Global Rating Scale P=0.049 Pre-Test Post-Test Control 59 69 BID 66 94

Test Skill 2: Bone Plating Global Rating Scale P=0.011 Pre-Test Post-Test Control 42 56 BID 43 82

Checklist Scores No significant difference in improvement in checklist scores for any skills

Secondary Outcome Measures

Questionnaire Data After first skill (both groups untrained) Control Group (%) BID Group (%) Satisfaction with Teaching Method 85 Objectives/Information 83 77 Active Learning/Faculty Interaction 79 Feedback 72

Questionnaire Data After last skill Control Group (%) BID Group (%) Satisfaction with Teaching Method 77 90 Objectives/Information 74 84 Active Learning/Faculty Interaction 72 88 Feedback 69 81

Focus Group Data Easy to use and remember For majority, first time setting objectives Formal time for feedback appreciated Less time wasted during sim sessions

Conclusions training both the learners and instructors in the BID method produced significant improvement in GRS scores for specific surgical skills trend towards higher learner satisfaction BID provides an easy-to-implement framework for surgical simulation teaching

Limitations Small sample size Choice of skills - limited Retention/transfer not tested In vivo vs. in vitro effects?

Future Directions these results are promising enough to continue with a larger-scale study more residents, more skills, more instructors simulation vs. OR role in Competency-Based Education?

“These sessions seemed a bit like a study group” Quotes from BID Group “These sessions seemed a bit like a study group” “Having an objective to work on focused me on a specific task” “I can see this working in the OR but it would need to have staff buy-in” “This may not work when I’m on call and it is very busy” “I think using this approach improved the flow of teaching” “I really liked the debriefing part, and I wish this would happen in real life!”

Thank you Acknowledgments: Thanks to Lisa Satterthwaite, Dr. Ranil Sonnandara, Carween Hui, Polina Mironova, Dr. Hesham Abdelbary, Dr. Michael Drexler, and the staff of the Mount Sinai Surgical Skills Centre

Intra-operative Teaching Step Task Script Briefing Set learning objectives “What would you like to focus on?” Intra-operative Teaching Teaching during the encounter Focus on stated objectives Debriefing Reflection Rules Reinforce Correct “How did you think you did?” “You did well at...” “Next time, do...”