1. Not Just “MK-1” How learning the skills of EBM relates to the pediatric milestones
Martha S Wright, MD, MEd
Rainbow Babies and Children’s Hospital

2. I have documented no financial relationships to disclose or Conflicts of Interest (COIs) to resolve.

3. I am…
An EBM Teacher
Committed to helping learners develop these skills both for
Direct patient care
Life-long learning
A Program Director
Responsible for adhering to requirements
Mindful of competing demands and time limitations
Who is an EBM teacher?
Who here is responsible for the EBM curriculum development at their program?
Any residency program directors?

4. At the end of this session, you will be able to…
Objectives
At the end of this session, you will be able to…
Describe how and where EBM skill development fits into the current paradigm of residency training requirements and performance assessment
Develop innovative curricula that integrate EBM training into current and emerging educational models

5. A brief history of “Competency-based Medical Education (CBME)”
The ACGME
Outcomes Project
Pre-1999
1999
Relatively few
actual
requirements
No duty
hour restrictions
Organize talk using a timeline showing how the approach to resident assessment have evolved over the last 16 years and what that has meant for those of us who teach residents (in general) and specifically teach about EBM
Before 1999 Hard to remember these days, I know…), Guyatt had coined the term “EBM” in 1992 and convened the EBM working group, David Sackett had begun to describe an approach to using the literature to answer patient care questions
But in the late 90s, there was growing concern about whether GME programs were producing competent physicians so in 1999, the ACGME began the Outcomes project with the intent to introduce processes that would document the outcome of GME –namely competent physicians, rather than just the process of the education (this many months of this and that…)
It was from this project tthat the Competencies were born…
They were introduced as a way to better define what physicians should know and be able to do. The Core Competencies were part of the movement to better measure resident outcomes. The desired outcome is to ensure that programs educate physicians who can practice independently.
Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education
The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure.
Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level.
ACGME/ABMS
Adopt 6 Core
Competencies
And Sub-competencies

6. A brief history of “Competency-based Medical Education (CBME)”
The ACGME
Outcomes Project
Pre-1999
1999
No duty
hour restrictions
Relatively few
actual requirements
Residencies
required to integrate
The Competencies
into their curricula
ACGME/ABMS
Adopt 6 Core
Competencies
And Sub-competencies
Ah the good old days, but I digress
They were introduced as a way to better define what physicians should know and be able to do. The Core Competencies were part of the movement to better measure resident outcomes. The desired outcome is to ensure that programs educate physicians who can practice independently. The Outcome Project had difficulty in measuring outcomes: resident performance and competency
Programs had difficulty defining or measuring the outcomes, and assessment tools were never fully developed to help programs to measure the outcomes. Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education
The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure.
Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level.

7. Core Competencies
Attempted to define what physicians should know and be able to do
Skills and tasks organized into 6 domains
Required us to find ways to teach, assess AND DOCUMENT resident performance in these domains

8. 1999-2008: Job Security for EBM teachers
EBM Skill development part of the PBLI competency
From the ACGME Core Program Requirements:
“Residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning.
Residents are expected to develop skills and habits to be able to meet the following goals:
locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems…”
Period of job security for EBM teachers!
(ACGME 2003, 2007)

9. 1999-2008: Job Security for EBM teachers
Sackett, et al. publish Evidence-based Medicine: How to Practice and Teach EBM
Explosion of GME Curriculum development
Journal Clubs (traditional, guided)
Stand-alone workshops, seminars
On-line educational modules
“On-the-job” training
Morning report sessions
Dedicated rotation blocks .

10. Challenges for The Competencies
Did not result in assessments that documented resident ability “to practice independently”
Context independent
Assessment tools never fully developed
Assessments still process-focused
“Describe (a) how and by whom residents are taught to access, appraise, and apply knowledge and (b) how and by whom they are evaluated in so doing.
Describe (a) the evidence-based medicine exercise that residents complete during training and attach the tool used for assessment as Appendix B; and (b) if residents are given the option of choosing from more that one type of activity to fulfill this requirement, describe each one briefly. If no options exist, enter “NO” for (b).”
Did not live up to their promised potential. The Outcome Project had difficulty in measuring outcomes: resident performance and competency
Programs had difficulty defining or measuring the outcomes, and assessment tools were never fully developed to help programs to measure the outcomes.

11. A brief history of “Competency-based Medical Education (CBME)”
The ACGME
Outcomes Project
The ACGME
Milestones Project
Pre-1999
1999
2012
2014
No duty
hour restrictions
Relatively few
actual requirements
Residencies
required to integrate
The Competencies
into their curricula
Residencies
report data on
21 Milestones
to ACGME
ACGME/ABMS
Adopt 6 Core
Competencies
And Sub-competencies
Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education
The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure.
Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level.
ABP publishes
48 Pediatric
Milestones

12. 2012: What the milestones added
Descriptions of level-specific learner behaviors along a developmental continuum
Prompted for a longitudinal approach to teaching and assessment of skill from knowledge to bedside application
Required observation-based assessments

13. Pediatricians Understand Milestones

14. MK-1: “The EBM milestone”
Evidence-based practice, self-regulated learning,

15. Teaching and Learning EBM Skills
Acquire knowledge
and skills
Longitudinal experiences
Use and practice
at the bedside
Road map, not only for observation of performance but for curricula-with specific attention to helping learners get from one level to the next through on-going experiences

16. Teaching and Learning EBM Skills
Acquire knowledge and skills
Didactics
E-learning modules
Small group activities
Computer labs
Longitudinal experiences
Journal Clubs
CATs
On-going didactics
Use and practice at the bedside
Role modeling
Educational prescriptions
Medical librarian on rounds
Blogs, wikis
Mobile devices, Apps

17. Pediatric Milestones PC1: Should I get a CXR on this asthmatic?
PC4: Which antibiotic should I
choose for this soft
tissue infection?
Should I get a CXR on this asthmatic?
Which antibiotic should choose for this soft tissue infection?
What would be the best source to read about community acquired pneumonia?
One shouldn’t be able to progress in these milestones without the acquisition of EBM skills….
PBLI 2: What would be the best
source to read about CAP?

18. Challenges of the Milestones
Do not assess learners holistically
Milestones can help us assess where the learner is along the continuum of individual skill development:
“gather accurate information about a patient”
“locate evidence related to their patient’s health problems”
They don’t tell us whether someone can put it all together to take care of a patient

19. A brief history of “Competency-based Medical Education (CBME)”
The ACGME
Outcomes Project
The ACGME
Milestones Project
Pre-1999
1999
2012
2014
2015
No duty
hour restrictions
Relatively few
actual requirements
Residencies
required to integrate
The Competencies
into their curricula
Residencies
report data on
21 Milestones
to ACGME
ACGME/ABMS
Adopt 6 Core
Competencies
And Sub-competencies
48 Pediatric
Milestones
published by ABP
ABP “finalizes”
Pediatric
EPAs

20. Entrustable Professional Activities (EPAs)
Describe routine activities of a pediatrician in specific contexts
Require “workplace-based” application and assessment of skills (e.g. “OPAs” Observed Practice Activities)
EPAs are executable within a given time frame; observable and measurable; and
suitable for focused entrustment decisions.1 EPAs are units of work (e.g., anesthetic care of an uncomplicated patient), while competencies describe
people’s abilities (e.g., knowledge, professional attitude, communication skill). Units of work and abilities of persons can be viewed as two dimensions
of a grid. Competencies remain theoretical if not grounded in practice.

21. 1. Provide consultation to other health care providers caring for children
2. Provide recommended pediatric health screening
3. Care for the well newborn
4. Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting
5. Provide a medical home for well children of all ages.
6. Provide a medical home for patients with complex, chronic, or special health care needs.
7. Recognize, provide initial management and refer patients presenting with surgical problems
8. Facilitate the transition from pediatric to adult health care
9. Assess and manage patients with common behavior/mental health problems
10. Resuscitate, initiate stabilization of the patient and then triage to align care with severity of illness
11. Manage information from a variety of sources for both learning and application to patient care
12. Refer patients who require consultation
13. Contribute to the fiscally sound and ethical management of a practice (e.g. through billing, scheduling, coding, and record keeping practices)
14. Apply public health principles and quality improvement methods to improve care and safety for populations, communities, and systems
15. Lead an interprofessional health care team
16. Facilitate handovers to another healthcare provider either within or across settings
17. Demonstrate competence in performing the common procedures of the general pediatrician
Pediatric EPAs

22. 1. Provide consultation to other health care providers caring for children
2. Provide recommended pediatric health screening
3. Care for the well newborn
4. Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting
5. Provide a medical home for well children of all ages.
6. Provide a medical home for patients with complex, chronic, or special health care needs.
7. Recognize, provide initial management and refer patients presenting with surgical problems
8. Facilitate the transition from pediatric to adult health care
9. Assess and manage patients with common behavior/mental health problems
10. Resuscitate, initiate stabilization of the patient and then triage to align care with severity of illness
11. Manage information from a variety of sources for both learning and application to patient care
12. Refer patients who require consultation
13. Contribute to the fiscally sound and ethical management of a practice (e.g. through billing, scheduling, coding, and record keeping practices)
14. Apply public health principles and quality improvement methods to improve care and safety for populations, communities, and systems
15. Lead an interprofessional health care team
16. Facilitate handovers to another healthcare provider either within or across settings
17. Demonstrate competence in performing the common procedures of the general pediatrician
Pediatric EPAs

23. Care for the normal well newborn
EPA Title
Care for the normal well newborn
The functions of this activity include:
Performing a physical examination to look for congenital anomalies
Identifying and applying key evidence based guidelines for care of the newborn
Caring for common problems that develop primarily in the nursery
Using judgment to know when common problems can be handled at home and arrange for discharge and follow-up
Assessing maternal/family readiness to care for the infant post discharge
Transitioning care to the community practitioner
Demonstrating confidence that puts new parents at ease
Link with Domains of Competence
_X_ Patient Care
___ Medical Knowledge
_X_ Practice-based Learning and Improvement
_X_ Interpersonal & Communication Skills
___ Professionalism
___ Systems-based Practice
_X_ Personal & Professional Development
Critical Competencies
PC 3: Transfer of care
PC 5: Physical exam
MK 2/PBLI 6: EBM
ICS 1: Communicate with patients
PPD 7: Self-confidence
(updated ABP 3/11/14)

24. Integration of EBM learning, practice and assessment into the Clinical Environment
Demonstration of skills with direct observation in the “workplace”
Using educational experiences longitudinally to help learners advance to higher levels
1998
2015

25. Opportunities
Assessment of EBM skills by direct observation in the clinical environment
Observation tool development
Educator development
Use of clinical decision tools in bedside decision making
Life-long learning curricula
The Art of “Keeping Up”
High Value, Cost-Conscious Care
Requires understanding of guidelines, systematic reviews, diagnostic testing, therapy/harm
The next EBM frontier

26. Summary
In 2015 there is still a place in residency training for teaching and learning EBM skills
EBM curricula should focus on the continuum from knowledge acquisition to application in the clinical environment
Educators will need to:
integrate educational activities into the existing/evolving clinical workflow
create authentic opportunities for learners to demonstrate and teachers to assess EBM skill application
To be most effective at teaching these skills (and most likely to get PD buy in for inclusion), educators will need to integrate ed opps into

27. Questions????