1. Getting Students to Entrustment on the Graduation EPAs Robert Englander, MD, MPH UCSF February 25 th, 2015

2. Disclosures I have no financial Conflicts of Interest to disclose

3. E PA Entrustment refers to the ability to effectively perform a professional activity unsupervised (GME) or without direct supervision (UME) Brings trust and supervision into assessment which are intuitive for faculty working with trainees Entrustment decisions allow inference about a learner’s competence

4. Premise: A framework based on Entrustment decisions requires a foundational quality in all learners

5. Trustworthiness is Based On: Ability or level of KSA Conscientiousness (Reliability? Accountability?) Telling the truth – absence of deception (truthfulness) Knowing one’s limits (discernment) Kennedy et al., Acad Med. 2008;83(10 Suppl):S89–92

6. Team Activity #1: Table Discussions How do you currently determine if your learners are trustworthy: Implicitly? Explicitly What have been effective strategies at other places you have been/worked? What is one thing UCSF can/should do going forward to make this determination more explicit?

7. EPAEPA Professional is a modifier of activities that refers specifically to: Area of practice (e.g. specialty) Scope of practice (e.g. profession) Point in an education-training-practice continuum (e.g. UME-to-GME or GME-to-practice transitions)

8. EP A The Activities: Represent the essential work that defines a discipline (in aggregate) Lead to a recognized outcome Should be independently executable within a given time frame Are observable and measurable units of work in both process and outcome Require integration of critical competencies and milestones

9. EPAs as the framework for assessment in the UME space: The Core Entrustable Professional Activities for Entering Residency

10. Timothy Flynn, Chair Stephanie Call Carol Carraccio Lynn Cleary Tracy Fulton Maureen Garrity Steve Lieberman Brenessa Lindeman Monica Lypson Rebecca Minter Jay Rosenfield Joe Thomas Mark Wilson Drafting Panel AAMC Staff: Carol Aschenbrener Bob Englander

11. Rationale Graduate Medical Education competencies well established Gaps identified between: Expectations of Program Directors and the skills of entering residents What residents do without supervision and what they have been documented as competent doing without supervision International focus on transitions

12. Project Charge Develop a clear, concise list of what graduating medical students should be entrusted to do without direct supervision on DAY ONE of residency

13. Conceptual Framework Two frameworks considered: Competencies Entrustable Professional Activities

14. EPAs  Make sense to faculty, trainees, and the public  Represent the day-to-day work of the professional  Situate competencies and milestones in the clinical context in which we live  Make assessment more practical by clustering the milestones into meaningful activities  Explicitly add the notions of trust and supervision into the assessment equation Competencies  Have been the basis for assessment in the GME space for a decade  In the aggregate define the “good physician”  Already have a good literature base on assessment in the “traditional” domains (medical knowledge and patient care)  Have established or developing milestones of performance for at least the GME years Conceptual Frameworks: Benefits

15. EPAs  Relatively recent introduction in the literature  Little operationalization of the concept worldwide  Original concept designed for the residency-to-practice transition Competencies  Are abstract  Are granular: not the way we think about or observe learners Conceptual Frameworks: Disadvantages

16. Core EPAs For Entering Residency EPAs For any Practicing Physician Expectations for the Medical School Graduate EPAs For Specialties

17. The Relationships of EPAs, Competencies and Milestones Each EPA “mapped” to its critical competencies using a sorting technique to prioritize Milestones established for the pre- entrustable and entrustable learner for each competency Expected behaviors for the pre-entrustable and entrustable learner delineated based on the milestones Vignettes created to illustrate the expected behaviors for the pre-entrustable and entrustable learner

18. EPA DOC M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx EPA: Entrustable Professional Activity DOC: Domain of Competence C: Competency M: Milestone C2C2 C3C3 C1C1 C4C4 C2C2 C5C5

19. M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx M1M1 MxMx Narrative description of an early (novice) learner Narrative description of a learner at “x” level

20. Core EPAs for Entering Residency 1) Gather a history and perform a physical examination 2) Prioritize a differential diagnosis following a clinical encounter 3) Recommend and interpret common diagnostic and screening tests 4) Enter and discuss orders/prescriptions 5) Document a clinical encounter in the patient record 6) Provide an oral presentation of a clinical encounter 7) Form Clinical Questions and retrieve evidence to advance patient care

21. Core EPAs for Entering Residency 8) Give or receive a patient handover to transition care responsibility 9) Collaborate as a member of an interprofessional team 10) Recognize a patient requiring urgent or emergent care, and initiate evaluation and management 11) Obtain informed consent for tests and/or procedures 12) Perform general procedures of a physician 13) Identify system failures and contribute to a culture of safety and improvement

22. Contents of the Complete Final Document Online (for Curriculum Developers) Full details for each EPA (description, critical competencies, milestones, expected behaviors and vignettes) Appendix A: Bulleted list of behaviors for the pre-entrustable and entrustable learner for all EPAs Appendix B: List of competencies used in the mapping process Appendix C: Grid of EPAs mapped to their critical Competencies

23. Contents of the Faculty Manual For each EPA: Title Description List of expected behaviors, narrative of expected behaviors, and vignette describing the pre-entrustable learner List of expected behaviors, narrative of expected behaviors, and vignette describing the entrustable learner

24. Team Activity 2: CEPAER Q Sort Exercise

25. Sample text here for a transition slide GQ 4 o 5 Program Directors All or most 1. Gather a history and perform a physical examination 94.9%88.4% 2. Prioritize a differential diagnosis following a clinical encounter 88.3%66.2% 3. Recommend and interpret common diagnostic and screening tests 83.9%66.0% 4. Enter and discuss orders/prescriptions54.9%59.7% 5. Document a clinical encounter in the patient record 89.4%80.2% 6. Provide an oral presentation of a clinical encounter 88.6%81.2% 7. Form clinical questions and retrieve evidence to advance patient care 87.9%63.8%

26. Sample text here for a transition slide GQ 4 or 5 Program Directors All or most 8. Give or receive a patient handover to transition care responsibility 77.3%54.8% 9. Collaborate as a member of an inter- professional team 91.9%78.3% 10. Recognize a patient requiring urgent or emergent care, and initiate evaluation and management 82.9%55.5% 11. Obtain informed consent for tests and/or procedures 79.9%49.6% 13. Identify system failures and contribute to a culture of safety and improvement 67.1%30.7%

27. Sample text here for a transition slide GQ 4 or 5 Program Director’s All or most 12. Perform general procedures of a physician (IV line insertion, Phlebotomy, BVM ventilation, CPR) 24.2% IV line insertion39.8% Phlebotomy43.6% Bag-valve-make ventilation (BVM)67.8% Cardiopulmonary resuscitation (CPR)62.9%

28. Team Activity 3: CEPAER Q Sort Exercise

29. Next Steps: Pilot Goal to study the implementation of EPAs in four areas: Curriculum Assessment Entrustment (including the UME to GME handoff) Faculty Development Recruited 10 pilot schools To begin design phase with face-to- face meeting of core teams in DC October 2014

30. Next Steps: Beta Testing Much of what we learn will come from Beta-testing in the field Partnering with institutions outside the pilot to create a learning community Core EPAs listserve coming soon…stay tuned

31. Questions/Comments