1. Are They Practicing What You are Preaching? Entrustable Professional Activities (EPA) for the Geriatrician Kelly Caverzagie, MD Department of Internal Medicine University of Nebraska Medical Center 1

2. Disclosures A portion of my salary at the University of Nebraska is reimbursed by the ABIM: ◦ Practice Improvement Module (PIM) Development ◦ Academic Affairs 2

3. Learning Objectives Identify the basic principles of Competency- Based Medical Education (CBME) Understand the concept of an Entrustable Professional Activity (EPA) as it relates to trainee assessment Apply lessons learned to advance work of community in developing EPA’s 3

4. Outline Background review of CBME ◦ Need for meaningful assessment Assessment of Competence ◦ Entrustable Professional Activities (EPA) Small group activities ◦ Reflect upon draft set of Geriatric EPA’s and recommend revisions 4

5. Competency-Based Medical Education CBME is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies. The International CBME Collaborators, 2009

6. ACGME – Outcome Project Introduced 1999 Implemented 2001 6 General Competencies Medical Knowledge Patient Care and Procedural Skills Professionalism Interpersonal & Communication Skills Practice-based Learning & Improvement Systems-based Practice 6

7. Struggles Moving Forward Programs have had trouble moving from traditional framework (structure/process) to CBME framework (competence/outcomes) ◦ Unclear and complex ACGME general competencies ◦ Difficulty in assessing and evaluating resident competence

8. Critical Care Medicine Ambulatory Clinic General Medicine Wards Rheumatology Cardiology Consult Research Elective Geriatrics Average 4.32 / 5 for Systems- Based Practice Old Assessment System

9. Traditional vs. CBME: Start with System Needs 9 Frenk Lancet, 2010

10. ACGME – Outcome Project Introduced 1999 Implemented 2001 6 General Competencies Medical Knowledge Patient Care and Procedural Skills Professionalism Interpersonal & Communication Skills Practice-based Learning & Improvement Systems-based Practice 10

11. IOM High Quality Care Timely – reducing waits and harmful delay Efficient – avoiding waste Equitable – providing care that does not vary in quality because of personal characteristics Safe – avoiding injuries rom care Effective – providing services based on scientific knowledge to all who could benefit Patient-centered – providing care that is respectful of and responsive to patient preferences, needs and values 11

12. Medicare-Determined Outcomes Work effectively in multiple settings Coordinate care within and across settings Understand cost and value of diagnostic and treatment options Work in inter-professional teams and multi- disciplinary team-based models Identify systematic errors and in implement systematic solutions in case of errors 12

13. Traditional vs. CBME 13 Educational Program VariableStructure/ProcessCompetency-based Driving force: CurriculumContentOutcome Goal of educ. encounterKnowledge acquisitionKnowledge application Typical assessment toolSingle subjective measureMultiple objective measures Assessment toolProxyAuthentic (mimic real tasks of profession) SettingRemoved (gestalt)Direct observation Focus of AssessmentNorm-referencedCriterion-referenced Timing of assessmentEmphasis on summativeEmphasis on formative Adapted from Carracchio, et al. 2002

14. Critical Care Medicine Ambulatory Clinic General Medicine Wards Rheumatology Cardiology Consult Research Elective Geriatrics Average 4.32 / 5 for Systems- Based Practice Old Assessment System

15. Resuscitate critically ill patient Manage multiple meds Manage transitions of care Perform musculoskeletal exam Consult to non-medicine teams Critically appraise literature Goals of care discussion Example activities of unsupervised internist OR Inform steps along the way to becoming an unsupervised internist New Assessment System

16. How does Brady get to drive a car?

17. Competency An observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes. The International CBME Collaborators, 2009

18. Competent Possessing the required abilities in all domains in a certain context at a defined stage of medical education or practice. The International CBME Collaborators, 2009

19. Competence Competence entails more than the possession of knowledge, skills and attitudes; it requires you … to apply these [abilities] in the clinical environment to achieve optimal results. ten Cate, Med Teach, 2010

20. Competenglish Competency – the thing(s) they need to do Competent – can do all of the things Competence – does all of the things consistently, adapting to contextual and situational needs

21. How Does Brady get to Drive a Car? Competency ◦ Can accelerate and brake smoothly ◦ Can approach an intersection and can turn left Competent ◦ Passes driver’s education classes ◦ Passes driver’s exam to get the license Competence ◦ Drives safely on interstate or during bad weather, avoids accidents, no traffic tickets ◦ Dad gives him the keys and walks away

22. When do your fellows get the keys to the car? Lots of good evaluations … Absence of bad evaluations … Survived a year of fellowship… Didn’t commit any crimes … Your program director followed the rules…

23. CBME is a new paradigm We must know the trainee has demonstrated competence and is ready to progress to the next stage of their career: ◦ Requires clear definition of expected competencies (i.e. thing they need to do) ◦ Requires assessment to determine whether these things are done consistently and within the contextual needs of the clinical environment

24. Geriatric Curricular Competencies 24

25. Potential Criticisms There are too many! ◦ Programs cannot assess all of them Reductionistic? ◦ Potential to deconstruct the learners’ performance into discreet tasks or checklists ◦ Checking off competencies does not equate to competence

26. CBME Challenge Ensure that the outcomes of the assessment actually demonstrate competence in the activities that define the profession Do they equate to the things that the public trusts that geriatricians are doing? Entrustable Professional Activities (EPA’s) 26

27. Entrustable Professional Activities “… identify the critical activities that constitute a specialty … the activities of which we would all agree should be only carried out by a trained specialist.” 27 ten Cate et al. Acad Med 2007; 82: 542-47

28. An Entrustable Professional Activity Part of essential work for a qualified professional Requires specific knowledge, skill, attitude Acquired through training Leads to recognized output Observable and measureable, leading to a conclusion Reflects the competencies expected EPA’s together constitute the core of the profession ten Cate et al. Acad Med 2007; 82: 542-47 28

29. Entrustable Professional Activities “Patients’ and instructors’ … entrustment of responsibility to a trainee is an essential concept in this approach…” 29 ten Cate et al. Acad Med 2007; 82: 542-47

30. Entrustment in Residency Training Attending physicians assess a multi- dimensional construct of “trustworthiness” when deciding a level of supervision Entrustment implies a level of competence Kennedy, et. al. Acad Med 2008; 83(10 Suppl): S89-92 30

31. COMPETENCE 31

32. 32 “Windows to Competence” Caverzagie and Iobst

33. 33 EPA - Lead and work within interprofessional teams

34. 34 Accept feedback EPA - Lead and work within interprofessional teams

35. 35 Manage diverse opinions with goal optimizing patient care EPA - Lead and work within interprofessional teams

36. 36 Maintain climate of mutual respect and shared values EPA - Lead and work within interprofessional teams

37. 37 Engage in collaborative communication EPA - Lead and work within interprofessional teams

38. 38 Identify and understand roles of team members EPA - Lead and work within interprofessional teams

39. 39 EPA - Lead and work within interprofessional teams Identify and understand roles of team members Engage in collaborative communication Maintain climate of mutual respect and shared values Manage diverse opinions with goal optimizing patient care Accept feedback

40. DRAFT Geriatricians EPA’s (v13) 1. Prevent, diagnose and treat conditions in older adults with attention to multimorbidity, age-related changes in physiology, function, treatment efficacy and response, medication management and psychosocial issues. 2. Prevent, diagnose and treat geriatric syndromes. 3. Provide and coordinate health care for older adults in all settings, during transitions of care, and at end of life. 4. Provide geriatric consultation. 5. Conduct meetings with patients, families and caregivers to clarify goals of care and make complex decisions in the context of prognosis, net benefit and burden, preferences and values. 6. Collaborate effectively as a member or leader of a health care team. 7. Teach the principles of geriatric care and aging-related health care issues to professionals and nonprofessionals. 8. Participate in quality improvement and other systems-based initiatives to assure patient safety and improve outcomes for older adults. 40

41. Group Activity #1 - Task Using your assigned Geriatric EPA … ◦ Describe the activity ◦ Identify the tasks required for you to entrust the fellow to do this activity at the end of training Goals ◦ Develop ‘shared mental model’ of the EPA in order to inform meaningful assessment ◦ Write down recommendations and provide to ________________. 41

42. Considerations Compare / contrast to Internal Medicine EPA’s Use Geriatric Curricular Competencies as a guide Balance between specific vs. vague / standardized vs. flexible Ideally, the description and tasks will be evidence-based and informed by the knowledge, skills and attitudes required of physicians to meet the outcomes expected of the future health care system 42

43. Group Activity #2 - Task Consider the entire list of Geriatric EPAs ◦ Is anything missing? Is anything redundant? ◦ Do they reflect the expected outcomes?  Profession  Patient / Public  Policy makers Goals ◦ Hand in recommend revisions ◦ Use your expertise to guide community 43

44. Building Assessments for an EPA Step 1 – Describe the activity. What tasks are required for you to entrust the fellow to do this activity at the end of training? Step 2 – Identify the Curricular Competencies that will help you assess a fellow performing this activity Step 3 – Identify specific assessment methods / tools 44

45. Clinical Competency Committee Educational Milestones Assessment System Next Accreditation System ADS UpdateSelf Study Resident SurveyFaculty Survey Board Pass RateCLER Visit Clinical Experience Log Core Faculty Scholarly Activity

46. Learning Objectives Identify the basic principles of Competency- Based Medical Education (CBME) Understand the concept of an Entrustable Professional Activity (EPA) as it relates to trainee assessment Apply lessons learned to advance work of community in developing EPA’s 46

47. Thank You! kelly.caverzagie@unmc.edu 47