1. University Medical Center Utrecht, the Netherlands*
Med Ed Grand Rounds UCSF, October 14, 2015
EPAs in the undergraduate medical curriculum Lessons learned Experiences from across the pond
Olle ten Cate, PhD
University Medical Center Utrecht, the Netherlands*
*Thanks to Lisanne Welink MD, Margot Weggemans MD, Suzanne van der Velden MD, Sanne van den Munckhof MD, Marijke van Dijk MD PhD - all UMC Utrecht

2. No conflict on interest to be reported
Financial support received from EU’s 7th Framework Programme, under grant agreement (WATCHME).

3. Content – lessons from UMC Utrecht
For consideration when developing EPAs for UME:
EPAs were not conceived for UME
Background: Utrecht UME curriculum development
Identifying the EPAs
What are adequate levels of supervision 
Entrustment before entrustment decisions
Nesting specific EPAs within general EPAs
Translation into examination rules
Faculty buy-in 
When are EPAs EPAs and when not?

4. EPAs were not conceived for UME
Essence: bringing trainees to an adequate level for unsupervised practice
Serving competency-based medical practice through competency-based medical education
CEPAER initiative has given the impression that EPAs should be mastered before residency;
cf McGaghie et al, AcadMed Nov. 2015:

5. EPA’s may be used in UME but..
They are not end stage of summative entrustment for unsupervised practice
They require careful consideration regarding a number of critical issues
Role of EPAs and entrustment in years before UME graduation still unclear
Not all important objectives for UME may be captured in (only) EPAs

6. Utrecht UME curriculum development
Involved are a Curriculum Committee CRU+ (including junior MD staff of Education Center) and groups of faculty, established to develop new courses and clinical clerkships

7. Features of current UME curriculum
Two closely connected phases (3yr Bachelor -3yr Master)
Integration of basic sciences in 5-week almost full time ‘blocks’ in first two years (B1+B2)
Much small group work; limited lectures; constructivist philosophy, not fully PBL
Early clinical rotations in B3 year (6 weeks internal medicine; 6 weeks surgery), intermittent blocks
PGY 6: transition to residency: longer clerkships, more responsibility

8. New features in CRU+ (from 2015)
Bachelor phase
Qualitative entrance selection; no lottery
Focus on knowledge retention. Repeated testing of knowledge. Students must pass block tests and four “CRUX” tests (each including the content of one semester)
B3 will include an integrated clerkship of 12 weeks (internal medicine, surgery and family medicine)
Every student adopts a panel of 4 patients or families for 3 years from one family medicine practice

9. New features in CRU+ (from 2016)
Master phase
M1 & M2: 4 units of 6 block weeks preparing for 12 consecutive weeks of longitudinally integrated clerkship (LINKs), each including 2-4 disciplines
Every student has own clinical preceptor for each LINK
EPAs form central feature in the structure and assessment in LINKs
M3 will not change much, but core EPAs for entering residency will feature too.

10. Non-clinical blocks and elective
Ma 3
START: Super-vised Training in
Attitude
Research
Teaching
Sub-internship
Elective
Research term
Elective
Ma 2
Block
LINK YELLOW
Family medicine, ENT, Opthalmology, Public health, Dermatology
LINK PURPLE
Internal medicine Surgery EM, Anesthesiology
Elective
Ma 1
Block
LINK RED
Pediatrics ObGyn Clinical genetics
LINK BLUE
Neurology Psychiatry Geriatrics
Elective
Block
LINK GREEN Family medicine Internal medicine Surgery
Ba 3
Non-clinical blocks and elective
Ba 2
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Ba 1
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block
Non-
clinical
block

11. Identifying the EPAs Resources
Dutch national Framework of Objectives for UME (legally binding)
AAMC Core EPAs for entering residency (13)
A proposed German list of EPAs (Charité U Berlin)
Iterative construction of EPA framework
July 2014: 10 EPAs
September 2014: 9 EPAs + 2 integrative Ma3 EPAs
June 2015: 2 Ba-EPAs, 8 Ma1/Ma2 EPAs, 2 Ma3 EPAs
August 2015: 2 Ba3 EPAs; 5 graduation core EPAs

12. UME EPAs first iteration1
Checking basic vital functions of a stable adult patient 2
Gathering and reporting basic general patient information that does not require investigations 3
Requesting and collecting basic bodily materials 4
Conducting simple therapeutic acts 5
Gathering and reporting basic specialty-specific patient information that does not require investigations 6
Requesting, interpreting and sharing basic diagnostic investigations 7
Design and initiating a treatment plan for common disorders 8
Breaking bad news to patients and family about non-terminal conditions 9
Conducting basic specialty-specific procedures 10
Acting as primary-responsible caregiver for a small ward

13. UME EPAs - second iteration1
Checking and reporting basic vital functions of a stable adult patient 2
Gathering and reporting basic general patient information of a stable adult patient 3
Requesting and collecting basic bodily materials of stable adults 4
Conducting simple therapeutic acts on a stable patient 5
Requesting, interpreting and sharing results of basic diagnostic investigations 6
Designing and initiating a treatment plan for common disorders 7
Breaking bad news to patients and family about non-terminal, non chronic conditions 8
Recognizing and acting on an emergency situation in the hospital 9
Caring for a patient around end-of-life decisions 10
Managing an inpatient ward (integrates 1-9) 11
Managing an outpatient clinic (integrates 1-9)

14. UME EPAs - third iteration1
BA3
Vital parameters 2
Basic medical procedures 3
MA1/2
History and general physical examination 4
Common procedures of the physician 5
Conducting simple therapeutic acts on a stable patient 6
Requesting, interpreting and sharing results of basic diagnostic investigations 7
Designing and initiating a treatment plan for common disorders 8
Breaking bad news to patients and family about non-terminal, non-chronic conditions 9
Recognizing and acting on an emergency situation in the hospital 10
Caring for a patient around end-of-life decisions 11
MA3
Managing an inpatient ward 12
AM3
Managing an outpatient clinic

15. UME EPAs - fourth iteration1
The Clinical Consultation 2
General Medical Procedures 3
Informing and Advising Patients and their Families 4
Intercollegial Communication 5
Care Under Unsusual Conditions

16. UME EPAs - fourth iteration1
The Clinical Consultation
Taking a medical history
Performing physical examination
Prioritizing a differential diagnosis
Requesting common diagnostic tests
Interpreting diagnostic tests
Designing a treatment plan 2
General Medical Procedures
Capillary blood taking
Venous blood withdrawal and taking a blood culture
Swabs: oral, nasal, ears, skin, anal or wounds
Giving infusions
Ankle brachial index
Administering a simple bandage and scarf bandage
Urethral catheterization
Suturing and injection of local anesthetic to skin
Perform an ECG
Give intracutaneous, subcutaneous
or intramuscular injections
Arterial blood gas

17. UME EPAs - fourth iteration3
Informing and Advising Patients and their Families
About diagnostic options (incl informed consent)
About prognosis (incl breaking bad news)
About therapeutic options (incl compliance and obtaining informed consent) 4
Intercollegial Communication
Discharge letter
Oral handover
Consulting other care providers
Refer to other care providers
Report on medical errors
Give oral patient or research presentation 5
Care Under Unsusual Conditions
Establishing patient death
Basic and advanced life support

18. UMCU EPAs versus AAMC Core EPAs
CEPAER
EPA 1: Gather a history and perform a physical examination X
EPA 2: Prioritize a differential diagnosis
EPA 3: Recommend and interpret common diagnostic and screening tests
EPA 4: Enter and discuss orders and prescriptions
EPA 5: Document a clinical encounter in the patient record
EPA 6: Give an oral presentation of a clinical encounter
EPA 7: Form clinical questions and retrieve evidence
EPA 8: Give or receive a patiënt handover
EPA 9: Collaborate as a member of an interprofessional team
EPA 10: Give urgent or emergent care
EPA 11: Obtain informed consent
EPA 12: Perform general procedures of a physician
EPA 13: Identify system failures and contribute to a culture of safety and improvement

19. What are adequate levels of supervision?
Issue: Existing entrustment and supervision scale not satisfactory
Too little gradation in first levels of supervision
Levels 4 and 5 will not be reached during UME
Existing PGME entrustment and supervision scale 1
Not allowed to practice EPA 2
Allowed to practice EPA only under proactive, full supervision 3
Allowed to practice EPA only under reactive/on-demand supervision 4
Allowed to practice EPA unsupervised 5
Allowed to supervise others in practice of EPA

20. But, early 2015, there was Carrie Chen et al.
PGME entrustment & supervision scale
UME entrustment & supervision scale (Chen et al 2015, Academic Medicine)
1. Not allowed to practice EPA
1a: Not allowed to observe EPA
1b: Allowed to observe EPA
2. Allowed to practice EPA only under proactive, full supervision (direct)
2a: As coactivity with supervisor
2b: With supervisor in room ready to step in as needed
3. Allowed to practice EPA only under reactive/on demand supervision (indirect)
3a: With supervisor immediately available, all findings double checked
3b: With supervisor immediately available, key findings double checked
3c: With supervisor distantly available (e.g. by phone), findings reviewed
4. Allowed to practice EPA unsupervised
5. Allowed to supervise others in practice of EPA

21. Entrustment before entrustment decisions
Issue
No early full (‘summative’) entrustment possible yet, but
Students need to practice with limited supervision
Student cannot always be directly supervised
How to justify that students already perform tasks with limited supervision

22. Entrustment before entrustment decisions
Approach
Ad hoc entrustment
occasional permission to practice with limited supervision for educational purposes (to be confirmed every time)
Summative entrustment
formalised, default permission to act with limited supervision

23. Nesting specific EPAs within general EPAs
Issue
‘Perform a physical examination’
‘Prioritize a differential diagnosis’
‘Recommend common diagnostic tests’
Many general skills of a physician require discipline-specific skills and knowledge

24. Nesting specific EPAs within general EPAs
Approach
Training of discipline-specific history, physical examination and specific procedures in designated blocks and LINKs
Integration in to full EPA in final (Ma3) year

25. Nesting specific EPAs within general EPAs
EPA1: The Clinical Consultation
Specifications
medical history
physical examination
differential diagnosis
common diagnostic tests
treatment plan
Discipine-specific skills and knowledge
Medical
Surgical
Pediatric
Gynaecological
Neurological
Psychiatric
Dermatological
ENT
Ophthalmologic
Issue
‘Perform a physical examination’
‘Prioritize a differential diagnosis’
‘Recommend common diagnostic tests’
General skills of a physician require discipline-specific skills and knowledge

26. Nesting specific EPAs within general EPAs
Procedures
Capillary blood taking
Venous blood withdrawal and taking a blood culture
Swabs: oral, nasal, ears, skin, anal or wounds
Giving infusions
Ankle brachial index
Administering a simple bandage and scarf bandage
Urethral catheterization
Suturing and injection of local anesthetic to skin
Performing an ECG
Giving intracutaneous, subcuta-neous and intramuscular injections
Measuring arterial blood gas
EPA 2: general medical procedures
 5th year students (Ma2)
EPA 2a: basic medical procedures
 3rd year students (Ba3)

27. Nesting specific EPAs within general EPAs
Procedures
Capillary blood taking
Venous blood withdrawal and taking a blood culture
Swabs: oral, nasal, ears, skin, anal or wounds
Giving infusions
Ankle brachial index
Administering a simple bandage and scarf bandage
Urethral catheterization
Suturing and injection of local anesthetic to skin
Performing an ECG
Giving intracutaneous, subcuta-neous and intramuscular injections
Measuring arterial blood gas
EPA 2: general medical procedures
 5th year students (Ma2)
EPA 2a: basic medical procedures
 3rd year students (Ba3)

28. Translation to assesment rules & procedures
No numerical final clerkship scores
End-of LINK qualifications [+expected percentages]:
“Fail” [5% or less]
“Requires attention” [10 -15%]
“Good” [75 to 80%]
“Excellent” [10% or less]
Dominant WBA tools:
Short practice observations
Case-based discussions
Patient presentations
Written reports on selected patients
Multi source professional behavior observation

29. Translation to assesment rules & procedures
EPA summative entrustment decisions in final year, fed by ‘nested’ small specific ‘EPAs’
Decision by local director of subinternship + advice by longitudinal family medicine mentor
Required for graduation:
All EPAs must be trusted on Level 3a (indirect supervision, all findings checked); Level 3b (key findings checked) or 3c (review only) exceeds standard expectation
E-portfolio support for feedback and entrustment decision making is being prepared

30. Faculty buy-in
While EPAs usually make ‘intuitive sense’, nailing down the EPAs and translation to teaching and assessment, requires explanation and adequate understanding
A recommendable procedure appears monitoring the active role of clinical teachers in development of procedures in the workplace
Continuous returning to the definition of EPA
Focus of discussion: when to leave students alone

31. When are EPAs EPAs and when not?
“Identify system failures and contribute to a culture of safety and improvement”
“scientifically active”
“Intercollegial communication”
Titles are insufficient to determine this, operationalization of the (set of) acivities is necessary.
Entrustable: Acts requiring trust – by colleagues, patients, society. Prohibited for unqualified persons.
Professional: Confined to occupations with extra-ordinary legal qualification.
Activities: Tasks that must be done. May be scheduled, may be listed in work descriptions.

32. Thank you