Structuring M & M Conferences for Educational Effectiveness M.J. Kim, F.J. Fleming, J.H. Peters, R.S. Salloum, J.R.T. Monson, M. Eghbali Department of Surgery University of Rochester, Rochester, NY April 22, 2010 University of Rochester Medical Center

Background - Morbidity & Mortality Conferences: Required by ACGME for surgical residents “Practice Based Learning and Improvement” Analyzing patient complications Identifying causes Proposing potential solutions University of Rochester Medical Center

Background - Content/format discrepancies - Mixed lessons learned Aboumatar et al, American Journal of Medical Quality 2007 - Mixed lessons learned Bender et al, American Journal of Surgery 2009 - Structuring M&M presentations can improve Understanding of complications Learner satisfaction Risucci et al, Current Surgery 2003 Muryama et al, American Journal of Surgery 2002 University of Rochester Medical Center

Objectives 1) Evaluate effectiveness of existing M&M conferences Content/Delivery Resident learning Perceptions of effectiveness 2) Develop a standardized presentation format Missing content Streamline discussion 3) Determine effectiveness of intervention University of Rochester Medical Center

Definitions Educational Effectiveness Specificity of the cause of the complication “OR staff not notified that potentially unstable patient in ED” vs. “Patient had delay to OR” Specificity of the potential solution/change “Designate contact person in OR for potential emergency cases” vs. “Need better communication” University of Rochester Medical Center

Methods Study Design Pre/Post intervention evaluation Study Sample Conferences in five divisions All presentations by residents Audience of faculty, residents, students, and staff University of Rochester Medical Center

Methods - Conference Observation (TEACHING) Two independent observers Consensus regarding conference practices - Resident Questionnaires (LEARNING) 2 Questions + any comments - Surveys of Learners (PERCEPTIONS) Online, anonymous Last M&M conference attended University of Rochester Medical Center

Results: Observation %Yes (n = 40) Observation Item 1) Is the complication clearly stated? 80% 2) If so, is it stated by the resident presenter? 55% 3) Is a potential cause clearly established? 65% 4) Is the cause proposed by the presenter? 18% 5) Are strategies for practice change discussed during each case? 85% 6) Are the changes proposed by the presenter? 8%

Results: Questionnaires Specificity of cause of complication (n=26) Mean rating (±SD) = 3.11 (±1.48) 1 = Vague 5 = Specific Specificity of a practice change (n=26) Mean rating = 3.42 (±1.50) 1 = No practice change stated 3 = Vague practice change 5 = Specific practice change based on current discussion University of Rochester Medical Center

Results: Survey Question %Yes (n = 19) 1) Is the current format of M&M presentations effective for analyzing complications? 63% 2) Is the current format effective for learning? 53% 3) Are specific complications clearly stated? 47% 4) Are specific causes clearly stated? 21% 5) After presentation, is it clear how to avoid this complication in the future? 32%

Presentation Format Case Presentation (< 5 minutes) Introduction Clinical Information Analysis Supporting Information (< 5 mins) Literature/Background Review Take Home Points (Maximum 2!) Discussion (< 5 mins) = Total Time < 15 minutes per case University of Rochester Medical Center

Presentation Format Format based on Introduced to residents Suggestions from surveys Previous publications Introduced to residents Sample presentation demonstrated Template slides with instructions Introduced to faculty Format/process discussed at faculty meeting Template, sample, instructions sent to all residents, fellows, and faculty University of Rochester Medical Center

Results: Observation Observation Item Pre-Format %Yes (n = 40) Post-Format, %Yes (n = 35) 1) Is the complication clearly stated? 80% 100%* 2) If so, is it stated by the resident presenter? 55% 3) Is a potential cause clearly established? 65% 91%* 4) Is the cause proposed by the presenter? 18% 57%* 5) Are strategies for practice change discussed during each case? 85% 89% 6) Are the changes proposed by the presenter? 8% 54%* * p<0.01 University of Rochester Medical Center

Results: Questionnaires Pre-Format Mean (±SD) Post-Format Specificity of the cause of complication 3.11 (±1.48) 4.56 (±1.03)* Specificity of a future practice change 3.42 (±1.50) 4.31 (±1.40)* *p<0.05 University of Rochester Medical Center

Results: Survey Question Pre-Format, %Yes (n = 19) Post-Format, %Yes (n = 18) 1) Is the current format of M&M presentations effective for analyzing complications? 63% 89% 2) Is the current format effective for learning? 53% 3) Are specific complications clearly stated? 47% 4) Are specific causes clearly stated? 21% 72% 5) After presentation, is it clear how to avoid this complication in the future? 32% 78% University of Rochester Medical Center

Conclusions 1) Structuring presentations leads to more specific lessons taught during M&M 2) More specific lessons from resident presenters leads to better learning by peers 3) Structuring presentations is a practical intervention to improve M&M conferences University of Rochester Medical Center

University of Rochester Medical Center

University of Rochester Medical Center

University of Rochester Medical Center

Results: Survey Suggested areas for changes: Focused discussion period - 79% Established presentation format - 63% Time-limited presentations - 53% Resident-led discussion - 21% Other (specify improvements, streamline) - 21% University of Rochester Medical Center

Results: Observation Observation Item Pre-Format, %Yes (n = 40) Post-Format, %Yes (n = 35) 1) Is the complication clearly stated? 80% 100%* 1b) Within 5 minutes? 20% 2) If so, is it stated by the resident presenter? 55% 3) Is a potential cause clearly established? 65% 91%* 3b) Within 10 minutes? 10% 71%* 4) Is the cause proposed by the presenter? 18% 57%* 5) Are strategies for practice change discussed during each case? 85% 89% 6) Are the changes proposed by the presenter? 8% 54%* * p<0.01 University of Rochester Medical Center

Results: Extent of use Did residents use the new format? Mean = 3.57 ±1.40 (1 = No organization, 5 = Followed each step) Was there literature review? Mean = 2.71 ±1.71 (1 = No attempt at supporting info, 5 = clearly applicable studies cited) Did faculty wait to ask questions? Yes = 17% Was each case <15 minutes? Yes = 51% University of Rochester Medical Center

Future Directions 1) Evaluate long-term (6-8 months) adherence to format changes 2) Study when/why format is not used 3) Compare with other assessments to evaluate extent of benefit University of Rochester Medical Center