University Medical Center Utrecht, the Netherlands* Med Ed Grand Rounds UCSF, October 14, 2015 EPAs in the undergraduate medical curriculum Lessons learned Experiences from across the pond Olle ten Cate, PhD University Medical Center Utrecht, the Netherlands* *Thanks to Lisanne Welink MD, Margot Weggemans MD, Suzanne van der Velden MD, Sanne van den Munckhof MD, Marijke van Dijk MD PhD - all UMC Utrecht

No conflict on interest to be reported Financial support received from EU’s 7th Framework Programme, under grant agreement 619349 (WATCHME).

Content – lessons from UMC Utrecht For consideration when developing EPAs for UME: EPAs were not conceived for UME Background: Utrecht UME curriculum development Identifying the EPAs What are adequate levels of supervision  Entrustment before entrustment decisions Nesting specific EPAs within general EPAs Translation into examination rules Faculty buy-in  When are EPAs EPAs and when not?

EPAs were not conceived for UME Essence: bringing trainees to an adequate level for unsupervised practice Serving competency-based medical practice through competency-based medical education CEPAER initiative has given the impression that EPAs should be mastered before residency; cf McGaghie et al, AcadMed Nov. 2015:

EPA’s may be used in UME but.. They are not end stage of summative entrustment for unsupervised practice They require careful consideration regarding a number of critical issues Role of EPAs and entrustment in years before UME graduation still unclear Not all important objectives for UME may be captured in (only) EPAs

Utrecht UME curriculum development Involved are a Curriculum Committee CRU+ (including junior MD staff of Education Center) and groups of faculty, established to develop new courses and clinical clerkships

Features of current UME curriculum Two closely connected phases (3yr Bachelor -3yr Master) Integration of basic sciences in 5-week almost full time ‘blocks’ in first two years (B1+B2) Much small group work; limited lectures; constructivist philosophy, not fully PBL Early clinical rotations in B3 year (6 weeks internal medicine; 6 weeks surgery), intermittent blocks PGY 6: transition to residency: longer clerkships, more responsibility

New features in CRU+ (from 2015) Bachelor phase Qualitative entrance selection; no lottery Focus on knowledge retention. Repeated testing of knowledge. Students must pass block tests and four “CRUX” tests (each including the content of one semester) B3 will include an integrated clerkship of 12 weeks (internal medicine, surgery and family medicine) Every student adopts a panel of 4 patients or families for 3 years from one family medicine practice

New features in CRU+ (from 2016) Master phase M1 & M2: 4 units of 6 block weeks preparing for 12 consecutive weeks of longitudinally integrated clerkship (LINKs), each including 2-4 disciplines Every student has own clinical preceptor for each LINK EPAs form central feature in the structure and assessment in LINKs M3 will not change much, but core EPAs for entering residency will feature too.

Non-clinical blocks and elective Ma 3 START: Super-vised Training in Attitude Research Teaching Sub-internship Elective Research term Elective Ma 2 Block LINK YELLOW Family medicine, ENT, Opthalmology, Public health, Dermatology LINK PURPLE Internal medicine Surgery EM, Anesthesiology Elective Ma 1 Block LINK RED Pediatrics ObGyn Clinical genetics LINK BLUE Neurology Psychiatry Geriatrics Elective Block LINK GREEN Family medicine Internal medicine Surgery Ba 3 Non-clinical blocks and elective Ba 2 Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Ba 1 Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block Non- clinical block

Identifying the EPAs Resources Dutch national Framework of Objectives for UME (legally binding) AAMC Core EPAs for entering residency (13) A proposed German list of EPAs (Charité U Berlin) Iterative construction of EPA framework July 2014: 10 EPAs September 2014: 9 EPAs + 2 integrative Ma3 EPAs June 2015: 2 Ba-EPAs, 8 Ma1/Ma2 EPAs, 2 Ma3 EPAs August 2015: 2 Ba3 EPAs; 5 graduation core EPAs

UME EPAs first iteration 1 Checking basic vital functions of a stable adult patient 2 Gathering and reporting basic general patient information that does not require investigations 3 Requesting and collecting basic bodily materials 4 Conducting simple therapeutic acts 5 Gathering and reporting basic specialty-specific patient information that does not require investigations 6 Requesting, interpreting and sharing basic diagnostic investigations 7 Design and initiating a treatment plan for common disorders 8 Breaking bad news to patients and family about non-terminal conditions 9 Conducting basic specialty-specific procedures 10 Acting as primary-responsible caregiver for a small ward

UME EPAs - second iteration 1 Checking and reporting basic vital functions of a stable adult patient 2 Gathering and reporting basic general patient information of a stable adult patient 3 Requesting and collecting basic bodily materials of stable adults 4 Conducting simple therapeutic acts on a stable patient 5 Requesting, interpreting and sharing results of basic diagnostic investigations 6 Designing and initiating a treatment plan for common disorders 7 Breaking bad news to patients and family about non-terminal, non chronic conditions 8 Recognizing and acting on an emergency situation in the hospital 9 Caring for a patient around end-of-life decisions 10 Managing an inpatient ward (integrates 1-9) 11 Managing an outpatient clinic (integrates 1-9)

UME EPAs - third iteration 1 BA3 Vital parameters 2 Basic medical procedures 3 MA1/2 History and general physical examination 4 Common procedures of the physician 5 Conducting simple therapeutic acts on a stable patient 6 Requesting, interpreting and sharing results of basic diagnostic investigations 7 Designing and initiating a treatment plan for common disorders 8 Breaking bad news to patients and family about non-terminal, non-chronic conditions 9 Recognizing and acting on an emergency situation in the hospital 10 Caring for a patient around end-of-life decisions 11 MA3 Managing an inpatient ward 12 AM3 Managing an outpatient clinic

UME EPAs - fourth iteration 1 The Clinical Consultation 2 General Medical Procedures 3 Informing and Advising Patients and their Families 4 Intercollegial Communication 5 Care Under Unsusual Conditions

UME EPAs - fourth iteration 1 The Clinical Consultation Taking a medical history Performing physical examination Prioritizing a differential diagnosis Requesting common diagnostic tests Interpreting diagnostic tests Designing a treatment plan 2 General Medical Procedures Capillary blood taking Venous blood withdrawal and taking a blood culture Swabs: oral, nasal, ears, skin, anal or wounds Giving infusions Ankle brachial index Administering a simple bandage and scarf bandage Urethral catheterization Suturing and injection of local anesthetic to skin Perform an ECG Give intracutaneous, subcutaneous or intramuscular injections Arterial blood gas

UME EPAs - fourth iteration 3 Informing and Advising Patients and their Families About diagnostic options (incl informed consent) About prognosis (incl breaking bad news) About therapeutic options (incl compliance and obtaining informed consent) 4 Intercollegial Communication Discharge letter Oral handover Consulting other care providers Refer to other care providers Report on medical errors Give oral patient or research presentation 5 Care Under Unsusual Conditions Establishing patient death Basic and advanced life support

UMCU EPAs versus AAMC Core EPAs CEPAER EPA 1: Gather a history and perform a physical examination X EPA 2: Prioritize a differential diagnosis EPA 3: Recommend and interpret common diagnostic and screening tests EPA 4: Enter and discuss orders and prescriptions EPA 5: Document a clinical encounter in the patient record EPA 6: Give an oral presentation of a clinical encounter EPA 7: Form clinical questions and retrieve evidence EPA 8: Give or receive a patiënt handover EPA 9: Collaborate as a member of an interprofessional team EPA 10: Give urgent or emergent care EPA 11: Obtain informed consent EPA 12: Perform general procedures of a physician EPA 13: Identify system failures and contribute to a culture of safety and improvement

What are adequate levels of supervision? Issue: Existing entrustment and supervision scale not satisfactory Too little gradation in first levels of supervision Levels 4 and 5 will not be reached during UME Existing PGME entrustment and supervision scale 1 Not allowed to practice EPA 2 Allowed to practice EPA only under proactive, full supervision 3 Allowed to practice EPA only under reactive/on-demand supervision 4 Allowed to practice EPA unsupervised 5 Allowed to supervise others in practice of EPA

But, early 2015, there was Carrie Chen et al. PGME entrustment & supervision scale UME entrustment & supervision scale (Chen et al 2015, Academic Medicine) 1. Not allowed to practice EPA 1a: Not allowed to observe EPA 1b: Allowed to observe EPA 2. Allowed to practice EPA only under proactive, full supervision (direct) 2a: As coactivity with supervisor 2b: With supervisor in room ready to step in as needed 3. Allowed to practice EPA only under reactive/on demand supervision (indirect) 3a: With supervisor immediately available, all findings double checked 3b: With supervisor immediately available, key findings double checked 3c: With supervisor distantly available (e.g. by phone), findings reviewed 4. Allowed to practice EPA unsupervised 5. Allowed to supervise others in practice of EPA

Entrustment before entrustment decisions Issue No early full (‘summative’) entrustment possible yet, but Students need to practice with limited supervision Student cannot always be directly supervised How to justify that students already perform tasks with limited supervision

Entrustment before entrustment decisions Approach Ad hoc entrustment occasional permission to practice with limited supervision for educational purposes (to be confirmed every time) Summative entrustment formalised, default permission to act with limited supervision

Nesting specific EPAs within general EPAs Issue ‘Perform a physical examination’ ‘Prioritize a differential diagnosis’ ‘Recommend common diagnostic tests’ Many general skills of a physician require discipline-specific skills and knowledge

Nesting specific EPAs within general EPAs Approach Training of discipline-specific history, physical examination and specific procedures in designated blocks and LINKs Integration in to full EPA in final (Ma3) year

Nesting specific EPAs within general EPAs EPA1: The Clinical Consultation Specifications medical history physical examination differential diagnosis common diagnostic tests treatment plan Discipine-specific skills and knowledge Medical Surgical Pediatric Gynaecological Neurological Psychiatric Dermatological ENT Ophthalmologic Issue ‘Perform a physical examination’ ‘Prioritize a differential diagnosis’ ‘Recommend common diagnostic tests’ General skills of a physician require discipline-specific skills and knowledge

Nesting specific EPAs within general EPAs Procedures Capillary blood taking Venous blood withdrawal and taking a blood culture Swabs: oral, nasal, ears, skin, anal or wounds Giving infusions Ankle brachial index Administering a simple bandage and scarf bandage Urethral catheterization Suturing and injection of local anesthetic to skin Performing an ECG Giving intracutaneous, subcuta-neous and intramuscular injections Measuring arterial blood gas EPA 2: general medical procedures  5th year students (Ma2) EPA 2a: basic medical procedures  3rd year students (Ba3)

Nesting specific EPAs within general EPAs Procedures Capillary blood taking Venous blood withdrawal and taking a blood culture Swabs: oral, nasal, ears, skin, anal or wounds Giving infusions Ankle brachial index Administering a simple bandage and scarf bandage Urethral catheterization Suturing and injection of local anesthetic to skin Performing an ECG Giving intracutaneous, subcuta-neous and intramuscular injections Measuring arterial blood gas EPA 2: general medical procedures  5th year students (Ma2) EPA 2a: basic medical procedures  3rd year students (Ba3)

Translation to assesment rules & procedures No numerical final clerkship scores End-of LINK qualifications [+expected percentages]: “Fail” [5% or less] “Requires attention” [10 -15%] “Good” [75 to 80%] “Excellent” [10% or less] Dominant WBA tools: Short practice observations Case-based discussions Patient presentations Written reports on selected patients Multi source professional behavior observation

Translation to assesment rules & procedures EPA summative entrustment decisions in final year, fed by ‘nested’ small specific ‘EPAs’ Decision by local director of subinternship + advice by longitudinal family medicine mentor Required for graduation: All EPAs must be trusted on Level 3a (indirect supervision, all findings checked); Level 3b (key findings checked) or 3c (review only) exceeds standard expectation E-portfolio support for feedback and entrustment decision making is being prepared

Faculty buy-in While EPAs usually make ‘intuitive sense’, nailing down the EPAs and translation to teaching and assessment, requires explanation and adequate understanding A recommendable procedure appears monitoring the active role of clinical teachers in development of procedures in the workplace Continuous returning to the definition of EPA Focus of discussion: when to leave students alone

When are EPAs EPAs and when not? “Identify system failures and contribute to a culture of safety and improvement” “scientifically active” “Intercollegial communication” Titles are insufficient to determine this, operationalization of the (set of) acivities is necessary. Entrustable: Acts requiring trust – by colleagues, patients, society. Prohibited for unqualified persons. Professional: Confined to occupations with extra-ordinary legal qualification. Activities: Tasks that must be done. May be scheduled, may be listed in work descriptions.

Thank you