Milestones, EPAs, NAS…and Other ACGME Jargon Committee on Graduate Medical Education September 24, 2012 Sara LP Ross, MD

Objectives To discuss the Next Accreditation System and what is known about how that will look in 2019 To discuss the system of trainee evaluation – Milestones and EPAs

NAS: Next Accreditation System Goals Complete realization of the Outcomes promise Free up “good programs” to innovate Assist poor programs in improving Reduce burden of accreditation Establish and implement milestones to better track program and institutional performance Provide accountability to the public July 2012: Seven initial core specialties/RRCs begin NAS training July 2013: NAS officially begins; seven specialties “go live”; remaining specialties begin training July 2014: All specialties/RRCs using NAS

NAS: Next Accreditation System 10-year self-study visit model: next visit 2019 for all Peds department programs Structure, Resources, Core processes, Detailed processes, Outcomes “What is your plan for the next 10 years to improve” Residents will submit a confidential consensus list of five strengths and “opportunities for improvement” (OFIs) the residents wish to discuss Site visitor will share strengths, but will only share OFIs if residents give permission (makes residents feel more connected to the site visit) Annual program surveillance Performance indicators for each specialty developed by a “community of educators” within the specialty Annual Resident Survey Core elements of the competencies Levels of performance Core methods of assessment Annual Faculty Survey Case Log Data Acgme will define the components of the self-study in preparation for the site visits over the next few years

NAS: Next Accreditation System Institutional 18 month review Ongoing creation of Milestones Programs will get a letter annually stating that they meet all “performance indicator thresholds” May be placed on “accreditation with warning” at any point during the 10-year cycle May warrant an “immediate visit” If problems not fixed during a given window of time, program may be placed on probation

Site Visits Structure of the visit: 2 site visitors Brief meeting with PD Resident and faculty interviews Meeting with DIO Meeting with PD PIF Elimination – YES (most likely) More focus on strengths of programs

Competencies Competence: the ability to do something successfully (Oxford Dictionary of English) Competencies: broad, general attributes of a good doctor With attempt at evaluation they get widdled down to detailed skills/activities In the end don’t really reflect the original meaning of the general competency Competence = Attribute Activity= Element of professional work Both competence and competency should be thought of as a “quality or state of being” - For example if the trainee should have the competencies to handle the care of a well newborn, then the next question should be “and what are these competencies?” i.e. checking the red reflex on exam, verifying administration of Vitamin K, assessing maternal risk factors Competent professional is clearly more than the sum of many detailed operational competencies Most activities reflect several competencies and most competencies are applied in several activities

Entrustable Professional Activities (EPAs) Units of work that may be awarded a more or less formal qualification at the moment when supervisors confirm the trainee is ready to assume responsibility for such activities Not ceremonial; can happen at any moment STAR: implies that a trainee has demonstrated enough competence to carry out the activity in question independently from now on - Requires supervisors to make deliberate decisions about trainees competence to perform all critical activities at some point in time

Entrustable Professional Activities (EPAs) Which critical professional activities cover the relevant competencies of the profession? How can supervisors learn when to entrust such activities to the trainee? “Trust reflects a dimension of competence that reaches further than observed ability. It includes the real outcome of training – the quality of care” Part of essential professional work Require specific knowledge, skill and attitude Lead to recognized output of professional labor Confined to qualified personnel Be independently executable within a timeframe Be observable and measurable in its process and outcome (well done or not well done) Reflect one or more competencies to be acquired Authors estimate 50-100 EPAs should cover the objectives of a full postgraduate training program of 5-6 years

Domains of Competency Competencies Milestones EPAs

EPA/Competency Matrix Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice? ten Cate, Olle; Scheele, Fedde Academic Medicine. 82(6):542-547, June 2007. DOI: 10.1097/ACM.0b013e31805559c7

EPAs May be acknowledged formally as a “statement of awarded responsibility” (STAR) Five levels of proficiency Has knowledge May act under full supervision May act under moderate supervision May act independently May act as a supervisor and instructor

Expected Levels of Confidence Time to achieve STAR in a specific EPA dependent on: The EPA The working environment The trainee The clinical teacher Columns are pgy2, 4 and 6

EPA Mapped to Competencies/ Subcompetencies

Milestones Developmental roadmap for the competencies and subcompetencies Observable developmental steps moving residents from novices to experts/masters Means of restructuring competencies into a measurable rubric of six domains of clinical competency

Domains of Competency Competencies Milestones EPAs

What milestone levels equate to different levels of proficiency? Who determines you can practice the EPA independently going forward? Minimum standards for advancement/graduation?

Where Are We in Pediatrics? January 2012 The Pediatrics Milestone Project (51) September 2012 Selection of 21 Pediatric Milestones to be reported on semi-annually - Requests for additional sites to study/ develop each of these milestones Fall/Winter 2012 Program development of evaluation tool July 2013 Implementation of Milestone reporting 51 milestones across 7 acgme competency domains; the additional domain being personal and professional development; not utilized in the ultimate 21 selected for evaluation

Where Are We in Pediatrics? January 2012 The Pediatrics Milestone Project (51) September 2012 Selection of 21 Pediatric Milestones to be reported on semi-annually - Requests for additional sites to study/ develop each of these milestones Fall/Winter 2012 Program development of evaluation tool July 2013 Implementation of Milestone reporting Evaluation algorithm: envisioned by ACGME that in the initial phase of implementation/evaluation, programs will use typical evaluation tools and a small committee of faculty will utilize those evaluations to completed the milestone grids; estimated to take about an hour/resident the first go-round and faster thereafter

References Carraccio C, Burke A. Beyond competencies and milestones: Adding meaning through context. J of Grad Med Ed. 2010;2(3):419-422. ten Cate O. Trust, competence, and the supervisor’s role in postgraduate training. British Medical Journal. 2006;333:748-751. ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2001;82:542-547.