Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott.

Slides:

Advertisements

Similar presentations

Empowering tobacco-free coalitions to collect local data on worksite and restaurant smoking policies Mary Michaud, MPP University of Wisconsin-Cooperative.

Advertisements

Evaluating the Reliability and Validity of the Family Conference OSCE Across Multiple Training Sites Jeffrey G. Chipman MD, Constance C. Schmitz PhD, Travis.

Faculty Development for Ambulatory Care Education LuAnn Wilkerson, Ed.D. UCLA.

Paired Observation and Video Editing (POVE) Blending Continuous Peer Feedback and Reflection to Strengthen Communication Training Larry Mauksch, M.Ed,

Professional Development & the Evaluation Process MARIA BERNALBROWARD COLLEGE EDWARD CORNEJOJUNE 9, 2015 PEDRO OLIVEIRA.

Using Teen Actors to Teach How to Communicate with Adolescents Anisha Abraham, MD, MPH Associate Professor, Department of Pediatrics Chief, Section of.

Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.

Educating Medical Students about the Care of Patients with Disabilities Kira Zwygart, MD Laurie Woodard, MD University of South Florida College of Medicine.

Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,

Building Research Capacity for Community Organizations: Strategies from the Alliance for Research in Chicagoland Communities Ben Rucker MPH, 1,2 Sherida.

CBEI Essentials for Residents, Fellows, and Faculty A 10-minute primer on student performance assessment in required clerkships Stanford School of Medicine.

NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT Shannon M. Sexton, Julia M. Williams, & Timothy Chow Rose-Hulman.

Impact of the “Asthma Toolbox” for Improving Documentation of Pediatric Asthma Management in an Urban Community Health Center Presenter: Delaney Gracy,

Background: As students complete their clerkships throughout their M3 year they gain in clinical experience and confidence, which may translate into improved.

Disclosure of Financial Conflicts of Interest in Continuing Medical Education Michael D. Jibson, MD, PhD and Jennifer Seibert, MD University of Michigan.

In order to improve the Family Medicine obstetrics (OB) rotation orientation and to encourage resident- resident teaching, we developed an orientation.

Copyright restrictions may apply Randomized Trial of Teaching Brief Motivational Interviewing to Pediatric Trainees to Promote Healthy Behaviors in Families.

Introduction Aimed at educators from wide-ranging backgrounds and with varying levels of experience, the Teaching and Assessment for Medical Educators.

CBEI Essentials for residents, fellows, and faculty A 10-minute primer on the Criterion-Based Evaluation Initiative and student performance assessment.

BackgroundResults GME completed all design, start-up, reviews, assessment tool and inter-program comparisons, statistics PDs completed the ‘field work’

The Electronic Health Record Lab: A Comprehensive Educational Intervention for Outpatient Electronic Records Bruce Britton M.D. Cy Cedar MS4 Christine.

Teaching Medical Students Quality Improvement Projects Using Plan-Do-Study- Act Method Carrie Roseamelia, PhD Robert Ostrander, MD.

David M. Deci, M.D. Director Office of Medical Student Education Associate Professor, Department of Family Medicine University of Wisconsin School of Medicine.

Results of CERA Clerkship Director Survey: Practice Based Learning and Improvement in Family Medicine Clerkships Deanna R Willis, MD, MBA Betsy G Jones,

Qualitative Analysis of Student- Patient Interviews in Underserved Clinics William B. Shore, MD Jessica Muller, PhD George Saba, PhD UCSF-Family and Community.

Family Medicine End of Clerkship Assessment (EOCA) [Eee-O-Kah] Irmanie Eliacin MD Christine Dalton, MD John Delzell Jr, MD Suzanne Minor, MD Marquita Samuels,

APPROVED CLINICAL INSTRUCTOR WORKSHOP Evaluation.

From Program Theory to Systems Theory: Using Logic Analysis to Re- conceptualize an Evaluation Lori L. Bakken, PhD; Jonathan M. Ross, MD; Curtis A. Olson,

“STAR (Safe Transitions Across CaRe): A resident and faculty initiative to improve patient care across the healthcare continuum Nancy M. Denizard-Thompson,

Delivering the Milestones Evaluation: Structuring Feedback & Comments from the CCC Dr. Eric Beachy, MD, Dr. Manju Thothala, MD, Dr. Nicole McGuire, DHSc.

Scott Renshaw, MD Department of Family Medicine Third Year Clerkship.

Student Continuity of Practice Experience (SCOPE) – A Longitudinal Primary Care Medical Student Clerkship Kyu K. Jana, MD; Jennifer Raley, MD; Cassandra.

Ultrasound in the Family Medicine Clerkship Rachel S. Brown, MD Assistant Professor Brian D. Keisler, MD Assistant Professor Department of Family and Preventive.

Ultrasound in the Family Medicine Clerkship Rachel S. Brown, MD Assistant Professor Brian D. Keisler, MD Assistant Professor Department of Family and Preventive.

Defining Grades in the Surgery Clerkship Jeremy M. Lipman, MD MetroHealth Medical Center Case Western Reserve University School of Medicine.

Procedure Logging - What's old is new again Theodore Gaeta, DO, MPH Michael Cabezon, MD Annette Visconti, MD New York Methodist Hospital Introduction METHODS.

Teaching From Afar: Educational Innovations In a Distance-learning Environment Jeanne Cawse-Lucas, MD Misbah Keen, MD, MBI, MPH UWSOM Family Medicine Clerkship.

Comments: [NONE] The Art of Meaningful Feedback

Initial Outcomes of a Reflective Portfolio and Advising System for Clinical Medical Students Scott Moser, MD Alan Blakely, PhD Anne Walling MB, ChB.

An Inter-Professional Education Delirium Simulation Pilot Project Heather S. Anderson, MD 1, Daniel L. Swagerty, MD, MPH 1, Mary Meyer, RN, MSN, ARNP-BC.

FADYA EL RAYESS, MD, MPH GOWRI ANANDARAJAH, MD HELEN BRYAN, MA ROBERTA GOLDMAN, PHD DEPARTMENT OF FAMILY MEDICINE ALPERT MEDICAL SCHOOL OF BROWN UNIVERSITY.

An Effective Method for Teaching Patient-centered Communication Skills to Second-year Medical Students Rachel Bramson, M.D., M.S. Michelle Jeter, Angela.

Practice CSE-3: A Formative, Low Stakes Experience for Third Year Family Medicine Clerkship Students Jeri R. Reid, MD Donna M. Roberts, MD Mary B. Carter,

Introduction References Objectives Conclusions Results Faculty provision of performance feedback is critical for residents to improve their clinical skills.

Transitioning to a New Clerkship Director: Coordinator and Director Perspectives Laura Mayans, MD Mary Hursey.

Rural Physician Associate Program (RPAP) University of Minnesota STFM, Jan Therese Zink, MD, MPH David Power, MB, MPH Deb Finstad, BS Kathleen.

Family Medicine Online Penn State College of Medicine Shou Ling Leong, MD Department of Family and Community Medicine Penn State College of Medicine.

National University Of Rwanda Faculty Assessment: Focus Group Discussions Inis Jane Bardella, MD, FAAFP L. Miriam Dickinson, PhD Mark Hotchkiss Doug Fernald,

Health Literacy Summit Madison, WI

A Multidisciplinary Transitions in Care Workshop for Medical Students

THE OSTE AS A FACULTY DEVELOPMENT TOOL

Eugene Orientale, Jr. MD Program Director, UCONN/St. Francis

Communication Skills Self-assessment by Family Medicine Clerkship Students Undergoing an Integrated Standardized Patient Exam Presenting: Carin D. McAbee,

Michael Henehan, DO San Jose-O’Connor Hospital

Communicating Milestones Evaluation Data and Comments from the CCC to Residents Using an Innovative “Grade Card” Steven McDonald, MD – Program Director.

The One Minute Preceptor

Flip Recorders An Alternative to Direct Observation

Indiana University School of Medicine

Maureen Gecht-Silver OTR/L, MPH UIC Department of Family Medicine

Leigh E. Tenkku, PhD, MPH Department of Family and Community Medicine

Evaluating Learners Jennifer L. Middleton, MD, MPH, FAAFP

A Web-based Approach to Enhance Preventive Medicine Education Outcomes for Third Year Medical Students 35th Annual Predoctoral Education Conference 2009.

STFM Predoctoral Education Conference 2010

Wayne State University

in the Emergency Department CLICK TO GO BACK TO KIOSK MENU

Using an ‘Oral Board’ exam to assess for EPA 10 in

Experience Introducing Physician Assistant Students

Weighted as 40% of final clerkship grade.

Committee # 4: Educational Program For The MD

Improving Preceptor Narrative Feedback Comments on

Presentation transcript:

Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott Moser MD

Background 8 week required M3 clerkship using volunteer community faculty Clinical evaluation: hybrid normative/competency-based grading form Marked inter-rater variability Poor Spearman correlations (n=132) – Preceptor grade vs. OSCE =.133 – Preceptor grade vs. NBME shelf =.164

Purpose Improve the precision of community preceptors' grading of student performance.

Methods Subjects: 19 community volunteer preceptors Intervention: Preceptor development workshop at annual regional CME program

Workshop Agenda Feedback: + / Δ card Evaluation: clinical grading form SP Videos: – Student presentation to preceptor case 1 – Student / SP encounter case 1 – Student /SP encounter case 2 – Each video followed by + / Δ discussion, grading form vote and further dialogue Wrap-up

Demonstration VIDEO CLIP

Evaluation Comparison of preceptor grading form votes post video Participant post-workshop survey Pre and post intervention comparison of family medicine clerkship grades Precision correlation of preceptor grade to end-of-clerkship OSCE

Results Grading form – Technical difficulties resulted in voting by hand – Generally noted inter-rater variability with good discussion Survey – Average rating of the workshop = scale, highest = 4 – Reoccurring comments: Increased direct observation and use of + / Δ card

Conclusions Workshop well-received by participants Standardized, scripted video accurately depicted student performance at border grading margins Improved grading precision remains unknown

Limitations Workshop attendees represented only 11.4% of clerkship preceptors Some family medicine preceptors have limited student volume Potential time lapse between intervention and preceptor – student interaction

Next Steps Complete follow up on grade comparison and grade form/OSCE precision outcomes Reinforce material during routine academic detailing visit and through creation of a resource website Disseminate videos to other departments and schools for preceptor development

Reflections Workshop key elements – Accuracy: Importance of direct observation – Inter-preceptor precision: Value of SP video and discussion Current grading form needs revision to clearer competency-based criteria Need ways to reach more preceptors

Acknowledgements Funding - MAITF Grant Nancy Davis, PhD Trisha Melhado, MPH Terry Ast, research assistant Whitney Rowe, MD KUSM–W SP Lab Staff KUSM–W IT Staff

1. Alexander E, Osman NY, Walling JL, Mitchell VG. Variation and Imprecision of Clerkship Grading in US Medical Schools. Academic Medicine. 2012; 87(8): Ogden PE, Edward J, Howell M, Via R, Song J. The Effect of Two Different Faculty Development Interventions on Third-year Clerkship Performance Evaluations. Fam Med. 2008; 40(5): Christopher S, Weaver MD, Alysius J, et al. A more Explicit Grading Scale Decreases Grade Inflation in a Clinical Clerkship. Academic Emergency Medicine. 2007; 14: Plymale MA, French J, Donnelly MB, Iocono J, Pulito AR. Variation in Faculty Evaluation of Clerkship Students Attributable to Surgical Service. J Surgical Education. 2010; 67(3): Kogan JR,et al. Opening the black box of clinical skills assessment via observation: a conceptual model. Medical Education. 2011; 45: References

Contact Information Laura Mayans – Kari Clouse – Scott Moser –