Evaluating Learners Jennifer L. Middleton, MD, MPH, FAAFP

Slides:

Advertisements

Similar presentations

T HE F LORIDA S TATE U NIVERSITY C OLLEGE OF M EDICINE Educating and developing exemplary physicians who practice patient-centered health care T HE F LORIDA.

Advertisements

Comprehensive Models of Formative Assessment. A theory of formative assessment.

Slide 1 Authentic Performance Assessment Ellen J. Lehning, Ph.D. Department of Anesthesiology.

Providing Effective Feedback Faculty Professional Development Series University of Pennsylvania School of Medicine October 2004 Jennifer R. Kogan, M.D.

Comparison: Traditional vs. Outcome Project Evaluative Processes Craig McClure, MD Educational Outcomes Service Group University of Arizona December 2004.

Improving Communication in the Emergency Department: The 5 Cs Model of Consultation Educational Soundbites CORD Academic Assembly 2011 San Diego, CA Chad.

The Emotional Intelligence Profile of Surgical Residents: A Descriptive Study AR Jensen, MD, AS Wright, MD, A Lance, MBA, K O’Brien, MA, C Pratt, MS, DJ.

Assessment of Communication Skills in Medical Education

Core Competency Assessment in Emergency Medicine from Design to Implementation Christian Arbelaez, MD, MPH Assistant Residency Director Harvard Affiliated.

ACGME OUTCOME PROJECT : THE PROGRAM COORDINATOR’S ROLE Jim Kerwin, MD University of Arizona.

Qualitative Evaluation Florida State University College of Medicine Rebecca Shiveler Office of Medical Education.

CBEI Essentials for Residents, Fellows, and Faculty A 10-minute primer on student performance assessment in required clerkships Stanford School of Medicine.

Classroom Assessments Checklists, Rating Scales, and Rubrics

© Copyright 2014 Milady, a part of Cengage Learning. All Rights Reserved. May not be scanned, copied, or duplicated, or posted to a publicly accessible.

Assessment of Emergency Medicine Residents’ Bedside Communication Skills: A Survey of Emergency Department Patients Amanda Keller York College of PA Biology.

Performance Assessment OSI Workshop June 25 – 27, 2003 Yerevan, Armenia Ara Tekian, PhD, MHPE University of Illinois at Chicago.

Disclosure of Financial Conflicts of Interest in Continuing Medical Education Michael D. Jibson, MD, PhD and Jennifer Seibert, MD University of Michigan.

Developing an Assessment System B. Joyce, PhD 2006.

Introducing Unit Specifications and Unit Assessment Support Packs Philosophy National 5.

Assessing Your Learner Lawrence R. Schiller, MD, FACG Digestive Health Associates of Texas Baylor University Medical Center, Dallas.

Comparison: Traditional vs. Outcome Project Educational Paradigms Craig McClure, MD Educational Outcomes Service Group University of Arizona December 2004.

What is a Planned Curriculum?

CBEI Essentials for residents, fellows, and faculty A 10-minute primer on the Criterion-Based Evaluation Initiative and student performance assessment.

Assessing Learners The Teaching Center Department of Pediatrics UNC School of Medicine The Teaching Center.

ACGME SIX CORE COMPETENCIES Minimum Program Requirements Language Approved by the ACGME, September 28, 1999 “The residency program must require its residents.

TEMPLATE AND PRINTING BY: GRMERC Consortium Members: Grand Valley State University, Michigan State University, Saint Mary’s.

Assessing Specialty Specific Milestones of ‘Off-Service’ Rotators During Emergency Medicine Rotation Lauren Walter, MD, FACEP, FAAEM and Andrew Edwards,

Abstract References Methods Introduction Results Conclusions Figures/Graphs Click headings to further view content Click Here to insert brief content.

IDENTIFICATION, DIAGNOSIS & REMEDIATION OF THE STRUGGLING LEARNER Jeannette Guerrasio, MD University of Colorado, SOM Chris Knight, MD University of Washington,

Defining Grades in the Surgery Clerkship Jeremy M. Lipman, MD MetroHealth Medical Center Case Western Reserve University School of Medicine.

Can Computers Teach Empathy? A Randomized Controlled Study Using Virtual Patients to Enhance Medical Students’ Empathic Communication Adriana Foster, MD.

Preparing for the ACGME's Next Accreditation System (NAS) A Prospective View for Family Medicine Residencies Joseph J. Brocato, PhD, University of Minnesota.

FADYA EL RAYESS, MD, MPH GOWRI ANANDARAJAH, MD HELEN BRYAN, MA ROBERTA GOLDMAN, PHD DEPARTMENT OF FAMILY MEDICINE ALPERT MEDICAL SCHOOL OF BROWN UNIVERSITY.

Maria Gabriela Castro MD Archana Kudrimoti MBBS MPH David Sacks PhD

Raj Woolever, MD Deborah Taylor, PhD Central Maine Medical Center Family Medicine Residency 2010 FMEC Northeast Region Meeting October 3o, 2010.

Curriculum Development for Outcome Based Learning

Designing Scoring Rubrics

EVALUATING EPP-CREATED ASSESSMENTS

AAC Assessment Glossary

Classroom Assessments Checklists, Rating Scales, and Rubrics

PORTFOLIO ASSESSMENT (Undergraduate Medical Education)

Evaluation Of and For Learning

Preparing for cbd wORKPLACE based assessment

Medical student evaluations:

CT & ACT National Conference

Introduction to Evaluation

8. Causality assessment:

FACILITATOR: Professor Pandeli Glavanis (PhD) Associate Director, CLT

Lawrence Family Medicine Residency

From Words to Action: Introduction of Core EPA Toolkits AAMC WGEA

Healthcare Matrix: Care of Patient(s) with….

Family Practice Residents’ Use of Clear Communication Skills

Classroom Assessments Checklists, Rating Scales, and Rubrics

Kuwait National Curriculum and Standards Development Project How to provide competence and standard-based assessment?

MPH Practicum overview February 13, 2018

Principles of Assessment & Criteria of good assessment

CBEI Essentials for Residents, Fellows, Advanced Practice Providers, and Faculty A 10-minute primer on student performance assessment in required clerkships.

Designing and Using Rubrics

Proposed End of Training EPAs

Standards-Based Grading

Competency, Milestones, and Credentialing: Evaluations and Simulation

Training Primary Care Residents In Integrated Settings Tailoring Behavioral Health to Residency Programs in Family Medicine and Beyond Patrick Hemming,

Evaluation Rating Forms

Changes in Training and Assessment

Gerald H. Sterling, Ph.D. Senior Associate Dean, Education

TESTING AND EVALUATION IN EDUCATION GA 3113 lecture 1

Access to HE Standardisation

An Introduction to the ACGME

Integrating Best Practices of Participant Evaluation Clinical Instructor Intensive Adrienne Small, DNP, FNP-C, CNE, CHSE Medical Instructor Duke University.

Principles of assessment in the clinical setting

Presentation transcript:

Evaluating Learners Jennifer L. Middleton, MD, MPH, FAAFP Academic Medicine Rotation August 2014 This is a very superficial overview of this topic – you can get master’s degrees in this! But it will give you the basics to help increase the quality of the evaluations you provide.

Objectives Describe possible goals of evaluating medical learners. List 3 challenges to accurate medical learner evaluation. Define “anchor point.”

What is evaluation? Why do we do it? Accreditation (ACGME, AAMC) Assess competence Gauge academic progress Predict performance Guide improvement Assign grades Judge program effectiveness http://familymed.uthscsa.edu/ACE/chapter6.htm

What’s the difference between feedback and evaluation?

Feedback Evaluation Provides information about performance YES Includes judgment about quality NO Example Your non-verbal skills showed your patient that you were interested in her story. Your doctor-patient communication skills are appropriate for your level of training.

Summative evaluation vs. Formative evaluation at the end during

How is formative evaluation different than feedback? provides assessment of quality How is formative evaluation different than feedback?

what an individual does Process performance vs. Outcome performance what an individual does the result Schmutz et al. Five Steps to Develop Checklists for Evaluating Clinical Performance: An Integrative Approach. Acad Med. 2014;89:996-1005

Subjective vs. objective measures Process performance: Subjective vs. objective measures based on predefined scoring categories using specific criteria based on clinical expertise of the rater subjective = problems with interrater reliability objective = done well should eliminate that problem Schmutz et al. Five Steps to Develop Checklists for Evaluating Clinical Performance: An Integrative Approach. Acad Med. 2014;89:996-1005

Objective criteria = checklist Is one of these better? (define “better”!)

Anchor points

Evaluation challenges Grade inflation Halo effect Vague descriptions How might objective evaluation measures help with these? (does it completely?)

One possible process for completing an evaluation: Complete yes/no and/or Likert scale questions first. Expand on any above average or below average scores. Offer specifics re: taking above average skills to the next level. Offer specifics re: improving below average marks. Provide your best estimate of the learner’s capability to improve below average areas.

Creating evaluation forms Summative or formative? Process or outcome? Assess objectives Legible – plenty of room!

Types of evaluation questions Yes/No Likert scale Qualitative What questions would work best to… Determine if your objectives were met? Rate skill at a procedure/behavior? Elicit ideas you might not have thought of?

The 6(7) ACGME (AOA) Competencies Medical knowledge Patient Care Practice Based Learning and Improvement Systems Based Practice Interpersonal Skills and Communication Professionalism Osteopathic Principles and Practices