Competency Based Medical Education and Assessment Maria Lucarelli, MD Associate Program Director Internal Medicine Residency January 13, 2015

Conflict of Interest None

Describe the characteristics and benefits of competency assessments in medical education Define types of competency assessments that may be applicable to your medical education area Identify existing competency assessment tool in your medical education area Objectives

Competency Based Education Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner centeredness J. R. Frank et al. 2010; 32: 631–637 Medical Teacher

Competency Based Education Outcomes Based Evaluation integrates knowledge, skills and attitudes Time-independent Learning is Individualized Learning occurs in the workplace

Process versus Competency based Programs Carraccio, C, Wolfsthal, SD, Englander, R, Ferentz, K, Martin C “Shifting Paradigms: From Flexner to Competencies” Acad Med (2002) 77:361-367.

Competency Based Education Requires continuous, comprehensive assessment Majority of assessment should happen in clinical environment Requires ongoing, highly effective feedback

Milestones A significant point in development and helps define the appropriate trajectory of a trainee Identify discrete knowledge, skills and attitudes expected of learners as they progress through training

Milestones Can demonstrate individual trajectory of competency acquisition Provides clear path of progress Can help focus assessment Allows for richer feedback

Competency Observable Measurable Integrating multiple components such as knowledge, skills, values, and attitudes Can be assembled for progressive development Descriptors of physicians

Competence Abilities across multiple domains or aspects of performance in a certain context Multi-dimensional and dynamic. It changes with time, experience, and setting. Requires qualifiers Relevant abilities Context Stage of training

Competent Possessing the required abilities in all domains in a certain context at a defined stage of medical education or practice.

Dreyfus Model Graded supervision allows for… 1. Observing the activity Expert Proficient Competent Advanced Beginner Novice Graded supervision allows for… 1. Observing the activity 2. Acting with direct supervision present in the room 3. Acting with supervision available within minutes 4. Acting unsupervised (i.e., under clinical oversight) 5. Providing supervision to juniors

Statement of Awarded Responsibility Expert Proficient Competent Advanced Beginner Novice Providing supervision to others Acting unsupervised Acting with supervision available within minutes Acting with direct supervision present in the room Observing the Activity Graded supervision allows for… 1. Observing the activity 2. Acting with direct supervision present in the room 3. Acting with supervision available within minutes 4. Acting unsupervised (i.e., under clinical oversight) 5. Providing supervision to juniors A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research. Jonker, Gersten; Hoff, Reinier; ten Cate, Olle European Journal of Anaesthesiology. 32(2):71-76, February 2015. DOI: 10.1097/EJA.0000000000000109

Entrustable Professional Activities (EPA) Specific knowledge, skills and attitudes acquired over the course of training critical to performing as a physician Ten Cate Academic Medicine, Vol. 82, No. 6 / June 2007

EPA Competency Subcompetencies Milestones

Building Assessments for an EPA in Three Steps. AAIM Conect   End-of-Training EPA Step 1 Description and Tasks Step 2 Related Curricular Milestones (Abbreviations on AAIM Website) Step 3 Assessment Methods/Tools Manage the care of patients with acute common diseases across multiple care settings Internal medicine physicians entering into unsupervised practice are able to diagnose and manage common acute medical symptoms (e.g., joint pain, chest pain, and headache) and conditions (e.g., uncontrolled HTN, decompensated HF, and COPD exacerbation) in community, ambulatory, and hospital settings. The tasks required: Obtain accurate and complete information sufficient to develop differential diagnosis and inform care plan; Knowledge of diseases common to internal medicine; Communicate plans of care to patients, families and care givers Adapt care plans to changing clinical information Patient Care (PC) A2, A3, B1, B2, C2, C3, D1, E1, F8 Multisource feedback Chart stimulated recall Chart audits Direct observations Standardized patient/OSCE In-training examination Medical Knowledge (MK) A2, A3, B1, B3 Practice-Based Learning & Improvement (PBLI) B1, B2, B3, D4, E1, E2 Interpersonal & Communication Skills (ICS) A3, A4, A5, A7, B1, B3, D3, F1 Professionalism (P) B1, B3, D2, E1, F1, F2, F3 G2, H1, I1, I2, K3 Systems-Based Practice (SBP) A3, B2, D2, D4, E1, E3 Building Assessments for an EPA in Three Steps. AAIM Conect

Fig. 1 A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research. Jonker, Gersten; Hoff, Reinier; ten Cate, Olle European Journal of Anaesthesiology. 32(2):71-76, February 2015. DOI: 10.1097/EJA.0000000000000109 Fig. 1 . Acquisition of competence. (a) Acquisition of competence, showing the competence threshold (corresponding to proficiency level IV) and continuing growth of expertise after delegation of a clinical activity. (b) Acquisition of competence for 5 different EPAs. At reaching the competence threshold, an informed and justified delegation decision can be made. Adapted with permission from 20. © 2015 European Society of Anaesthesiology. Published by Lippincott Williams & Wilkins. 2

Role of Assessment Earlier identification Customized training More rapid advancement Deficiencies Advanced CBE requires more formative assessment and high quality feedback Early identification of individuals with deficiencies Facilitates identification of more advanced trainees allowing for more customization of their training to allow for more rapid advancement Earlier identification Customized remediation plans Earlier and fair exit from training when necessary Holmboe et al The role of assessment in competency based medical education. 2010; 32:676-82. Medical Teacher

Assessments Needs to be continuous and frequent Formative > summative Allows for deliberate practice Needs to be coupled with effective feedback and ongoing mentoring/coaching

Assessments Criterion Based Take place in the clinical environment Allows for appropriate expectations Developmental (milestones) Blueprint for assessment and informs appropriate methods/tools for assessment Take place in the clinical environment Does not eliminate the role of simulation Heavily dependent upon faculty observations/assessment

Assessment Requires the use of high quality assessment tools One single evaluation may not be perfect Consider the use of multiple tools Qualitative approaches to assessment can be valuable

Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States Janice L. Hanson*, Adam A. Rosenberg and J. Lindsey Lane Front. Psychol., 21 November 2013

Knowledge assessments 360 evaluations Patient surveys

Mapping Milestones A Tool for Mapping the ACGME Milestones to Evaluation Sources Linda Myerholtz Ph.D.

Resources PM&R Milestones Central www.physiatry.org Society of Teachers of Family Medicine www.fmdrl.org Alliance For Academic Internal Medicine www.im.org