Accreditation Council for Graduate Medical Education © 2014 Accreditation Council for Graduate Medical Education Competency-based Medical Education (CBME) and Transformation Future Directions of Credentialing Research in Nursing: A Workshop

© 2014 Accreditation Council for Graduate Medical Education Disclosures  Eric Holmboe  Employed by ACGME  Formerly employed by ABIM  Receives royalties for textbook from Mosby-Elsevier

© 2014 Accreditation Council for Graduate Medical Education Why CBME: System Needs 3 Frenk J. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010

© 2014 Accreditation Council for Graduate Medical Education What Are The Outcomes? A competent (at a minimum) practitioner aligned with: CMS Triple Aim

© 2014 Accreditation Council for Graduate Medical Education The Assessment “System” Assessments within Program: Direct observations Audit and performance data Multi-source FB Simulation ITExam Qual/Quant “Data” Synthesis: Committee Residents Faculty, PDs and others Milestones and EPAs as Guiding Framework and Blueprint Accreditation Unit of Analysis: Program Certification and Credentialing Unit of Analysis: Individual JUDGEMENTJUDGEMENT D FB DD PUBLICPUBLIC

© 2014 Accreditation Council for Graduate Medical Education

Milestones By definition a milestone is a significant point in development. Milestones should enable residents, fellows and the training program to better determine an individual’s trajectory of competency acquisition.

© 2014 Accreditation Council for Graduate Medical Education PC1. History (Appropriate for age and impairment) Level 1Level 2Level 3Level 4Level 5 Acquires a general medical history Acquires a basic physiatric history including medical, functional, and psychosocial elements Acquires a comprehensive physiatric history integrating medical, functional, and psychosocial elements Seeks and obtains data from secondary sources when needed Efficiently acquires and presents a relevant history in a prioritized and hypothesis driven fashion across a wide spectrum of ages and impairments Elicits subtleties and information that may not be readily volunteered by the patient Gathers and synthesizes information in a highly efficient manner Rapidly focuses on presenting problem, and elicits key information in a prioritized fashion Models the gathering of subtle and difficult information from the patient Competency Developmental Progression or Set of Milestones Sub-competency Specific Milestone

Entrustable Professional Activities EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty The concept of “entrustable” means: ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to perform this activity [unsupervised].’’ 1 1 Ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82(6):542–547.

© 2014 Accreditation Council for Graduate Medical Education Competencies, Milestones and EPAs COMPETENCY Entrustable Professional Activity MILESTONES CharacteristicCompetenciesMilestonesEPAs GranularityLowModerate to HighLow to Moderate Synthetic/IntegratedModerateLow to ModerateHigh Practicality (application) LowModerateHigh ConceptualHighLowLow to Moderate COMPETENCY MILESTONES COMPETENCY MILESTONES COMPETENCY

© 2014 Accreditation Council for Graduate Medical Education Milestones and EPAs as Roadmap Observations: 1)Journey not a straight line 2)More than one path (but not infinite) 3)“If you don’t know where you are going, any road will get you there”

© 2014 Accreditation Council for Graduate Medical Education Dreyfus & Dreyfus Development Model Dreyfus SE and Dreyfus HL Carraccio CL et al. Acad Med 2008;83:761-7 Time, Practice, Experience Novice Advanced Beginner Competent Proficient Expert/ Master

© 2014 Accreditation Council for Graduate Medical Education Dreyfus & Dreyfus Development Model Dreyfus SE and Dreyfus HL Carraccio CL et al. Acad Med 2008;83:761-7 Time, Practice, Experience Novice Advanced Beginner Competent Proficient Expert/ Master MILESTONES Curriculum Assessment Curriculum Assessment Curriculum Assessment Curriculum Assessment Curriculum Assessment

Effective Assessment System Processes  Most important component of curriculum is the clinical care clnicians provide and experience  Clarity on right outcomes linked to curriculum  Integration of the educational and clinical systems  Right combination and synthesis of assessment methods  Critical importance of shared understanding & mental models of competence  Competencies, milestones, entrustable professional activities (EPAs)

© 2014 Accreditation Council for Graduate Medical Education Improvements We Can Implement Now  Observe, observe, observe  There is currently no substitute for ongoing observation and feedback from an expert clinician.  Stop seeking the grail of assessment forms.  Assessment forms are only as good as the individual using them.  Align assessment forms with purpose & construct.  Invest in assessor development.  Refine use of work-based assessment methods.  Treat local assessment practices as a continuous quality improvement activity.

© 2014 Accreditation Council for Graduate Medical Education Next Steps for Assessment  Need to advance development of work- based assessment (WBA) methods  Interprofessional team care  Effective use of clinical decision support  Quality, systems-based practice, safety  Sophisticated communication skills  Embed WBA into routine clinical work  EMRs will need to evolve  Robust and longitudinal feedback loops