Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,

Slides:

Advertisements

Similar presentations

Think of a good resident teacher you encountered when in medical school. What one or two things made the resident a good teacher?

Advertisements

M4 Interest Group Nabil Issa, MD- Director, Surgery Subinternship Northwestern University Feinberg School of Medicine.

Characterization and Needs Assessment Brenessa M. Lindeman, MD 1, Pamela Lipsett, MD 1, Adnan Alseidi, MD 2, Anne Lidor, MD 1 Johns Hopkins Hospital 1,

How Accurate is the ACGME Resident Survey? Comparison Between ACGME and In-House GME Survey Bridget N. Fahy 1, S. Rob Todd 1, Judy L. Paukert 2, Melanie.

Historically, teaching on the Consultation-Liaison Psychiatry (CLP) Service was case-based. As a result, second year residents (R2s) were not systematically.

Preceptor: Louise A. Mawn, M.D. May 30, Medical Documentation Medical record serves many functions For health care providers it facilitates: Communication.

Joan E. St. Onge, M.D. UMMSM At Holy Cross Hospital Internal Medicine Residency Faculty Development January 23, 2013 The Evaluation Toolkit.

UCD School of Medicine “Criterion Based” vs. “Norm-Based” Evaluation David L Gaspar MD October 18, 2008.

CBES Essentials for Residents, Fellows, and Faculty A 10-minute primer on student performance assessment in required clerkships Stanford School of Medicine.

American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants Who Will Pass the Examinations on the First Attempt? John L. Shellito,

Daniel Farkas, Kamal Nagpal, Ernesto Curras, Ajay Shah, John Cosgrove

Barbara J Pettitt, MD Emory University Dept. of Surgery Atlanta GA April 24, 2015.

Family Medicine Residents, Optometry Students, and Faculty Members Engaged in Health Professions Education and Collaborative Patient Care: An Example of.

Maximally-Invasive Curriculum: A Model Curriculum for Osteopathic Surgical Residencies (ACOS) India Broyles, EdD University of New England College of Osteopathic.

Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.

Johns Hopkins University Master of Education in the Health Professions MEHP Anne Belcher, PhD, RN, AOCN, ANEF, FAAN.

I really wanted to become a doctor since I was grade 1. There is a reason that I want to be a doctor because My father was also a doctor but I really.

What do you remember about your MSIII/IV Ophthalmology clerkship?

Assessment tool OSCE AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.

Assessing clinical judgment using the script concordance test: The importance of using specialty-specific experts to develop the scoring key Petrucci AM.

Is the Script-Concordance Test a Valid Instrument for Assessment of Intra-operative Decision-making Skills? Brent Zabolotny 1, Robert Gagnon 2, Bernard.

Background: As students complete their clerkships throughout their M3 year they gain in clinical experience and confidence, which may translate into improved.

Validation of a Toolkit to Assess the Communication Climate in Health Care Organizations Matthew Wynia, MD, MPH, 1 Megan Johnson, MA, 1 Thomas P McCoy,

Direct Observation of Clinical Skills During Patient Care NEW INSIGHTS – REYNOLDS MEETING 2012 Direct Observation Team: J. Kogan, L. Conforti, W. Iobst,

By: Lauren De Luca.  Specialize in the care of young people  Conduct physical exams and order tests to diagnose diseases and injuries  Recommend treatment,

Proficiency of surgical faculty and residents with ethical dilemmas: Is modeling enough? Kamela K. Scott, PhD David J. Chesire, PhD J. Bracken Burns, Jr,

TEACH ON THE RUN! SESSION 3: HELPING LEARNERS THINK! DIAGNOSTIC REASONING Created by Dr. Leslie Ann Sadownik.

Jonathan Fryer, Noreen Corcoran, Brian George, Ed Wang, Deb DaRosa Northwestern University Feinberg School of Medicine, Chicago Illinois.

Outpatient morning report  ALIREZA RAJAEI MD  ASSOCIATE PROFESSOR  INTERNIST, RHEUMATOLOGIST  LOGHMAN HOSPITAL  MEDICAL FACULTY EDO  SHAHID BEHSHTI.

NMC Online Conference December 2005 Assessing Learning in a MORPG Patricia Youngblood, PhD Director of Evaluation, SUMMIT (Stanford University Medical.

CBEI Essentials for residents, fellows, and faculty A 10-minute primer on the Criterion-Based Evaluation Initiative and student performance assessment.

Teacher Evaluation Systems 2.0: What Have We Learned? EdWeek Webinar March 14, 2013 Laura Goe, Ph.D. Research Scientist, ETS Sr. Research and Technical.

Neal E. Seymour, MD Baystate Medical Center Tufts University School of Medicine Baystate Simulation Center—Goldberg Surgical Skills Lab 2010 APDS Annual.

TMC Service Clinic Documentation Rounds APRIL-JUNE 2015 MA. CRISTINA TUAZON, MD TMC Department of PEDIATRICS.

Schizophrenia Issues of Reliability and Validity Explain the issues of reliability with classification of schizophrenia.

Using Simulated Encounters with Standardized Patients to Teach and Evaluate Difficult Discussions Rebecca Stetzer, MD Kathleen M Young, PhD, MPH Albany.

Essentials of Procedural Skills: Early Preclinical Introduction to Common Emergency Medicine Procedures Xiao C. Zhang †, MD, MS; Armon Ayandeh ‡, MSc,

The use of OSCE to assess Patient Care, Professionalism and Interpersonal Communication Milestones in EM residents Miriam Kulkarni, MD, Harsh Sule, MD,

TEMPLATE AND PRINTING BY: GRMERC Consortium Members: Grand Valley State University, Michigan State University, Saint Mary’s.

Simulation Training and Assessment in EMS Sean W Moore MD CM, FRCPC, emergency medicine University of Ottawa, Department of Emergency Medicine Associate.

Family Medicine End of Clerkship Assessment (EOCA) [Eee-O-Kah] Irmanie Eliacin MD Christine Dalton, MD John Delzell Jr, MD Suzanne Minor, MD Marquita Samuels,

Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott.

Scott Renshaw, MD Department of Family Medicine Third Year Clerkship.

Defining Grades in the Surgery Clerkship Jeremy M. Lipman, MD MetroHealth Medical Center Case Western Reserve University School of Medicine.

Background The Patient Centered Medical Home (PCMH) has become the framework for the future of primary care and the healthcare system in the United States.

Maria Gabriela Castro MD Archana Kudrimoti MBBS MPH David Sacks PhD

OBJECTIVE INTRODUCTION Emergency Medicine Milestones: Longitudinal Interrater Agreement EM milestones were developed by EM experts for the Accreditation.

Introduction References Objectives Conclusions Results Faculty provision of performance feedback is critical for residents to improve their clinical skills.

NEUROLOGY/NEUROSURGERY Joanne Wojcieszek, M.D., Course Director Rebecca Radford, Clerkship Coordinator Robert Pascuszzi,

VIRTUAL PATIENT - Computer based teaching CAMPUS SOFTWARE Srdjan Masic, MBI, MPH Dejan Bokonjic, MD, PhD.

Competency and Credentialing John Andazola, MD William Ellert, MD Phoenix, Arizona.

Comparison of graduates from the longitudinal integrated and rotation based clerkships Performance in family medicine residency: W. Woloschuk, D Myhre.

Todd Felix, MD, Sam Faber, MD, Dave Richard, MD

Evaluation of new medical school curricula: identifying and treating

Introduction Results Objectives Conclusions Methods Limitations

Introduction to Evaluation

Manual Musculoskeletal Medicine Elective for Allopathic Residents

Objective vs Subjective

Evaluating Learners Jennifer L. Middleton, MD, MPH, FAAFP

STFM Predoctoral Education Conference 2010

CLICK TO GO BACK TO KIOSK MENU Material & Methods (Click)

CLICK TO GO BACK TO KIOSK MENU Correlation Coefficient

Basics of Clinical Medicine Lecture Series:

Using an ‘Oral Board’ exam to assess for EPA 10 in

Joint Africa – EU Strategy Tuning Seminars

Maxine A. Papadakis, M.D. Professor of Medicine UCSF

Gender Bias in Nursing Assessments of Emergency Medicine Residents

Christopher S. Kiefer MD, Erica B. Shaver MD,

Evaluation Rating Forms

Marc R Moon, MD, Ralph J Damiano, MD, G

Presentation transcript:

Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks, MD, MEd 1 Johns Hopkins Hospital Departments of Surgery 1 and Pediatrics 2 Johns Hopkins School of Medicine Office of Medical Education Services 3

High Correlation Low Correlation Surgery Emergency Medicine Internal Medicine RadiologyPediatrics Objective and Subjective Knowledge Correlation

 Investigate convergent validity between subjective ratings of surgery student clinical knowledge and scores on the NBME subject examination  Inter-rater reliability of faculty and resident evaluations of global clinical knowledge

12345 UnacceptableNeeds Improvement At Expected Level Above Expectations Outstanding Unable to apply preclinical knowledge to understand basic medical problems. Inconsistent understanding of patient problems. Limited differential diagnosis. Knows basic differential diagnoses of major/active problems in patients. Understands team’s choice of therapy. Knows expanded differential diagnoses, including recognition of emergencies. Can discuss therapeutic options. Knows nuances of differential diagnosis, including disease prevalence and anticipated history & exam findings. Able to independently formulate a management plan. Able to assign prognoses. Subjective Student Evaluation Scale Retrospective, cross-sectional study from July 2009 – June 2011

Inter-rater Reliability Intraclass Coefficient Attending evaluations (n=216) Resident evaluations (n=207) Convergent Validity Correlation of: Average evaluation ratings NBME exam scores Quartile Comparison ANOVA Average resident and attending ratings

RatingAttendings % (n) Residents % (n) 1 N/A 2<1% (4)1% (6) 320% (178)28% (244) 457% (498)47% (402) 522% (193)24% (211)

ρ 2 : Attendings <0.01 Residents 0.01

Medical Student Ratings by NBME Percentile Group QuartileAttending n (SD) Resident n (SD) Bottom 25 th 3.96 (0.45) 3.81 (0.48) Middle 50 th 4.00 (0.35) 3.95 (0.43) Top 25 th 4.08 (0.40) 4.10 (0.49) Bottom 25th Middle 50th Top 25th

Other aspects of student performance are perhaps being considered as a proxy for knowledge Faculty members form a one-dimensional view of student performance when assigning grades 1 Faculty may conceptualize a generalized global assessment of student performance from which they base all of their ratings 1 Pulito AR, Donnelly MB, Plymale M, et al. What do faculty observe of medical students' clinical performance? Teach Learn Med 2006;18:

Contextual and discipline-specific trends may contribute to how faculty and residents perceive student performance The technical domain of surgeons may skew the ability to assess other areas of performance Faculty and residents would benefit from dedicated training in knowledge assessment