1. Assigning Milestone Evaluations in Internal Medicine
Melvin Blanchard, MD, FACP
Program Director, Internal Medicine
Chief, Division of Medical Education
Department of Medicine

2. Outline Internal Medicine program overview
ACGME Charges re Competencies
Response by IM community
ACGME counter response
Our program’s approach to evaluation

3. Internal Medicine Overview
Discipline encompassing the study and practice of health promotion, disease prevention, diagnosis, care and treatment of adults
1 of 4 physicians in the US
IM residents
Our program:
151 trainees
2/3 inpatient; 1/3 outpatient
BJH, VA, community, international
~50 rotations
Category
Positions
Number of training programs
393
Positions in match
6177
First year fellows
4584

4. ACGME Charge - 1 1999 – ACGME launched the Outcomes Project
Required PDs to assess trainees in 6 competencies
Patient Care
Professionalism
Practice-Based Learning and Improvement
Interpersonal and Communication skills
Medical Knowledge
Systems-Based Practice
Competencies required of a physician to deliver competent medical care

5. ACGME Charge - 2
2009 – ACGME charged specialties with identifying milestones of competency development
Observable developmental steps
How do we know that PGY-2 resident will be competent at graduation?
How do we know that graduates from the 393 IM programs can deliver same quality of IM care?

6. IM Community Response 2009-12: Published 142 Milestones
Aka Curricular Milestones
Point in development that facilitates assessment of progression from beginner to expected proficiency at end of training
Published 16 Entrustable Professional Activities (EPAs)
KSAs critical to practice specialty
Milestones and EPAs categorized by 6 competencies

7. EPAs Manage patients with diseases across multiple care settings.
Provide age-appropriate screening and preventative care.
Resuscitate, stabilize, and care for unstable or critically ill patients.
Provide perioperative assessment and care. 
Manage transitions of care.
Facilitate family meetings.
Enhance patient safety.
Improve the quality of health care at individual and systems levels.
Demonstrate personal habits of lifelong learning.
Demonstrate professional behavior.

8. ACGME Counter Response
142 milestones is too numerous
Not optimal format for reporting to the ACGME
ACGME and ABIM combined forces
Developed milestones for reporting educational outcomes
Aka Reporting Milestones or Milestones
Grouped into 22 sub-competences
Grouped into 6 competencies
With each competency associated with 2 – 5 sub-competencies

9. Copyright © American College of Physicians.
From: The Internal Medicine Reporting Milestones and the Next Accreditation System
Ann Intern Med. 2013;158(7): doi: /
Figure Legend:
Copyright © 2012 The Accreditation Council for Graduate Medical Education and The American Board of Internal Medicine.
Date of download:
2/5/2014
Copyright © American College of Physicians.
All rights reserved.

10. Competencies
Curricular milestones
Reporting milestones
Milestones
Entrustable Professional Activities
Narratives
Sub-competencies

11. Competencies to MilestonesMK PC
PBLI
SBP
Works in teams
Cost conscious
Transition of care
Disregards communication
Inconsistent
Recognizes importance
Uses resources
Coordinates care
Competencies
Sub-
competencies
Milestones
Curricular Milestones
5 levels of milestones: Critical deficiency to Aspirational
EPAs
Entrustable Professional Activities

12. What did we do? Broke 22 sub-competencies into sub-sub competencies
Each sub-sub competency can be used as a question on a rotation evaluation
Rotations divided among 5 faculty
Each faculty work with core faculty/rotation directors to select evaluation questions appropriate to rotation
Tracked assignment to assure each sub-competency measured multiple times
Data from conference attendance, ITE, journal club participation, etc. also feed into evaluation system

13. 1. Gathers and synthesizes essential and accurate information to define each patient’s clinical problem(s). (PC1)
Critical Deficiencies
Ready for unsupervised practice
Aspirational
Does not collect accurate historical data.
Inconsistently able to acquire accurate historical information in an organized fashion.
Consistently acquires accurate and relevant histories from patients.
Acquires accurate histories from patients in an efficient, prioritized, and hypothesis-driven fashion.
Obtains relevant historical subtleties, including sensitive information that informs the differential diagnosis.
Does not use physical exam to confirm history.
Does not perform an appropriately thorough physical exam or misses key physical exam findings.
Consistently performs accurate and appropriately thorough physical exams.
Performs accurate physical exams that are targeted to the patient’s complaints.
Identifies subtle or unusual physical exam findings.
Relies exclusively on documentation of others to generate own database or differential diagnosis.
Does not seek or is overly reliant on secondary data.
Seeks and obtains data from secondary sources when needed.
Fails to recognize patient’s central clinical problems.
Fails to recognize potentially life threatening problems.
Inconsistently recognizes patients’ central clinical problem or develops limited differential diagnoses.
Uses collected data to define a patient’s central clinical problem(s).
Synthesizes data to generate a prioritized differential diagnosis and problem list.
Efficiently utilizes all sources of secondary data to inform differential diagnosis.
Effectively uses history and physical examination skills to minimize the need for further diagnostic testing.
Role models and teaches the effective use of history and physical examination skills to minimize the need for further diagnostic testing.

14. Milestone distribution across Rotations
Sub-competency 1 2 3 4 5 6 7 8
Competency
PC1
PC2
PC3
PC4
PC5
MK1
MK2
SBP1
Inpt Gen Med
Rheum
CAER
Neuro
GI consult NF