1. Competency Based Medical Education and Assessment
Maria Lucarelli, MD
Associate Program Director Internal Medicine Residency
January 13, 2015

2. Conflict of Interest
None

3. Describe the characteristics and benefits of competency assessments in medical education
Define types of competency assessments that may be applicable to your medical education area
Identify existing competency assessment tool in your medical education area
Objectives

4. Competency Based Education
Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner centeredness
J. R. Frank et al. 2010; 32: 631–637 Medical Teacher

5. Competency Based Education
Outcomes Based
Evaluation integrates knowledge, skills and attitudes
Time-independent
Learning is Individualized
Learning occurs in the workplace

6. Process versus Competency based Programs
Carraccio, C, Wolfsthal, SD, Englander, R, Ferentz, K, Martin C “Shifting Paradigms: From Flexner to Competencies” Acad Med (2002) 77:

7. Competency Based Education
Requires continuous, comprehensive assessment
Majority of assessment should happen in clinical environment
Requires ongoing, highly effective feedback

8. Milestones
A significant point in development and helps define the appropriate trajectory of a trainee
Identify discrete knowledge, skills and attitudes expected of learners as they progress through training

9. Milestones
Can demonstrate individual trajectory of competency acquisition
Provides clear path of progress
Can help focus assessment
Allows for richer feedback

11. Competency Observable Measurable
Integrating multiple components such as knowledge, skills, values, and attitudes
Can be assembled for progressive development
Descriptors of physicians

12. Competence
Abilities across multiple domains or aspects of performance in a certain context
Multi-dimensional and dynamic. It changes with time, experience, and setting.
Requires qualifiers
Relevant abilities
Context
Stage of training

13. Competent
Possessing the required abilities in all domains in a certain context at a defined stage of medical education or practice.

14. Dreyfus Model Graded supervision allows for… 1. Observing the activity
Expert
Proficient
Competent
Advanced Beginner
Novice
Graded supervision allows for…
1. Observing the activity
2. Acting with direct supervision present in the room
3. Acting with supervision available within minutes
4. Acting unsupervised (i.e., under clinical oversight)
5. Providing supervision to juniors

15. Statement of Awarded Responsibility
Expert
Proficient
Competent
Advanced Beginner
Novice
Providing supervision to others
Acting unsupervised
Acting with supervision available within minutes
Acting with direct supervision present in the room
Observing the Activity
Graded supervision allows for…
1. Observing the activity
2. Acting with direct supervision present in the room
3. Acting with supervision available within minutes
4. Acting unsupervised (i.e., under clinical oversight)
5. Providing supervision to juniors
A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research.
Jonker, Gersten; Hoff, Reinier; ten Cate, Olle
European Journal of Anaesthesiology. 32(2):71-76, February 2015.
DOI: /EJA

17. Entrustable Professional Activities (EPA)
Specific knowledge, skills and attitudes acquired over the course of training critical to performing as a physician
Ten Cate Academic Medicine, Vol. 82, No. 6 / June 2007

18. EPA
Competency
Subcompetencies
Milestones

19. Building Assessments for an EPA in Three Steps. AAIM Conect
End-of-Training EPA
Step 1
Description and Tasks
Step 2
Related Curricular Milestones
(Abbreviations on AAIM Website)
Step 3
Assessment Methods/Tools
Manage the care of patients with acute common diseases across multiple care settings
Internal medicine physicians entering into unsupervised practice are able to diagnose and manage common acute medical symptoms (e.g., joint pain, chest pain, and headache) and conditions (e.g., uncontrolled HTN, decompensated HF, and COPD exacerbation) in community, ambulatory, and hospital settings.
The tasks required:
Obtain accurate and complete information sufficient to develop differential diagnosis and inform care plan;
Knowledge of diseases common to internal medicine;
Communicate plans of care to patients, families and care givers
Adapt care plans to changing clinical information
Patient Care (PC)
A2, A3, B1, B2, C2, C3, D1, E1, F8
Multisource feedback
Chart stimulated recall
Chart audits
Direct observations
Standardized patient/OSCE
In-training examination
Medical Knowledge (MK)
A2, A3, B1, B3
Practice-Based Learning & Improvement (PBLI)
B1, B2, B3, D4, E1, E2
Interpersonal & Communication Skills (ICS)
A3, A4, A5, A7, B1, B3, D3, F1
Professionalism (P)
B1, B3, D2, E1, F1, F2, F3 G2, H1, I1, I2, K3
Systems-Based Practice (SBP)
A3, B2, D2, D4, E1, E3
Building Assessments for an EPA in Three Steps. AAIM Conect

20. Fig. 1
A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research.
Jonker, Gersten; Hoff, Reinier; ten Cate, Olle
European Journal of Anaesthesiology. 32(2):71-76, February 2015.
DOI: /EJA
Fig. 1 . Acquisition of competence. (a) Acquisition of competence, showing the competence threshold (corresponding to proficiency level IV) and continuing growth of expertise after delegation of a clinical activity. (b) Acquisition of competence for 5 different EPAs. At reaching the competence threshold, an informed and justified delegation decision can be made. Adapted with permission from 20.
© 2015 European Society of Anaesthesiology. Published by Lippincott Williams & Wilkins. 2

21. Role of Assessment Earlier identification Customized training
More rapid advancement
Deficiencies
Advanced
CBE requires more formative assessment and high quality feedback
Early identification of individuals with deficiencies
Facilitates identification of more advanced trainees allowing for more customization of their training to allow for more rapid advancement
Earlier identification
Customized remediation plans
Earlier and fair exit from training when necessary
Holmboe et al The role of assessment in competency based medical education. 2010; 32: Medical Teacher

22. Assessments Needs to be continuous and frequent
Formative > summative
Allows for deliberate practice
Needs to be coupled with effective feedback and ongoing mentoring/coaching

23. Assessments Criterion Based Take place in the clinical environment
Allows for appropriate expectations
Developmental (milestones)
Blueprint for assessment and informs appropriate methods/tools for assessment
Take place in the clinical environment
Does not eliminate the role of simulation
Heavily dependent upon faculty observations/assessment

24. Assessment Requires the use of high quality assessment tools
One single evaluation may not be perfect
Consider the use of multiple tools
Qualitative approaches to assessment can be valuable

25. Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States
Janice L. Hanson*, Adam A. Rosenberg and J. Lindsey Lane Front. Psychol., 21 November 2013

26. Knowledge assessments
360 evaluations
Patient surveys

27. Mapping Milestones
A Tool for Mapping the ACGME Milestones to Evaluation Sources
Linda Myerholtz Ph.D.

28. Resources PM&R Milestones Central
Society of Teachers of Family Medicine
Alliance For Academic Internal Medicine