Not Just “MK-1” How learning the skills of EBM relates to the pediatric milestones Martha S Wright, MD, MEd Rainbow Babies and Children’s Hospital

I have documented no financial relationships to disclose or Conflicts of Interest (COIs) to resolve.

I am… An EBM Teacher Committed to helping learners develop these skills both for Direct patient care Life-long learning A Program Director Responsible for adhering to requirements Mindful of competing demands and time limitations Who is an EBM teacher? Who here is responsible for the EBM curriculum development at their program? Any residency program directors?

At the end of this session, you will be able to… Objectives At the end of this session, you will be able to… Describe how and where EBM skill development fits into the current paradigm of residency training requirements and performance assessment Develop innovative curricula that integrate EBM training into current and emerging educational models

A brief history of “Competency-based Medical Education (CBME)” The ACGME Outcomes Project Pre-1999 1999 Relatively few actual requirements No duty hour restrictions Organize talk using a timeline showing how the approach to resident assessment have evolved over the last 16 years and what that has meant for those of us who teach residents (in general) and specifically teach about EBM Before 1999 Hard to remember these days, I know…), Guyatt had coined the term “EBM” in 1992 and convened the EBM working group, David Sackett had begun to describe an approach to using the literature to answer patient care questions But in the late 90s, there was growing concern about whether GME programs were producing competent physicians so in 1999, the ACGME began the Outcomes project with the intent to introduce processes that would document the outcome of GME –namely competent physicians, rather than just the process of the education (this many months of this and that…) It was from this project tthat the Competencies were born… They were introduced as a way to better define what physicians should know and be able to do. The Core Competencies were part of the movement to better measure resident outcomes. The desired outcome is to ensure that programs educate physicians who can practice independently. Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure. Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level. ACGME/ABMS Adopt 6 Core Competencies And Sub-competencies

A brief history of “Competency-based Medical Education (CBME)” The ACGME Outcomes Project Pre-1999 1999 2002-2008 No duty hour restrictions Relatively few actual requirements Residencies required to integrate The Competencies into their curricula ACGME/ABMS Adopt 6 Core Competencies And Sub-competencies Ah the good old days, but I digress They were introduced as a way to better define what physicians should know and be able to do. The Core Competencies were part of the movement to better measure resident outcomes. The desired outcome is to ensure that programs educate physicians who can practice independently. The Outcome Project had difficulty in measuring outcomes: resident performance and competency Programs had difficulty defining or measuring the outcomes, and assessment tools were never fully developed to help programs to measure the outcomes. Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure. Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level.

Core Competencies Attempted to define what physicians should know and be able to do Skills and tasks organized into 6 domains Required us to find ways to teach, assess AND DOCUMENT resident performance in these domains

1999-2008: Job Security for EBM teachers EBM Skill development part of the PBLI competency From the ACGME Core Program Requirements: “Residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning. Residents are expected to develop skills and habits to be able to meet the following goals: locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems…” Period of job security for EBM teachers! (ACGME 2003, 2007)

1999-2008: Job Security for EBM teachers Sackett, et al. publish Evidence-based Medicine: How to Practice and Teach EBM Explosion of GME Curriculum development Journal Clubs (traditional, guided) Stand-alone workshops, seminars On-line educational modules “On-the-job” training Morning report sessions Dedicated rotation blocks .

Challenges for The Competencies Did not result in assessments that documented resident ability “to practice independently” Context independent Assessment tools never fully developed Assessments still process-focused “Describe (a) how and by whom residents are taught to access, appraise, and apply knowledge and (b) how and by whom they are evaluated in so doing. Describe (a) the evidence-based medicine exercise that residents complete during training and attach the tool used for assessment as Appendix B; and (b) if residents are given the option of choosing from more that one type of activity to fulfill this requirement, describe each one briefly. If no options exist, enter “NO” for (b).” Did not live up to their promised potential. The Outcome Project had difficulty in measuring outcomes: resident performance and competency Programs had difficulty defining or measuring the outcomes, and assessment tools were never fully developed to help programs to measure the outcomes.

A brief history of “Competency-based Medical Education (CBME)” The ACGME Outcomes Project The ACGME Milestones Project Pre-1999 1999 2002-2008 2012 2014 No duty hour restrictions Relatively few actual requirements Residencies required to integrate The Competencies into their curricula Residencies report data on 21 Milestones to ACGME ACGME/ABMS Adopt 6 Core Competencies And Sub-competencies Milestones were created with the goal to allow programs to more accurately evaluate the outcomes of residency education The Milestones were developed to address those shortcomings, by making assessment more explicit, more concrete, and easier to understand and measure. Milestone evaluations provide a process to set expectations on what more there is to learn or what a resident has to accomplish to move to the next level. ABP publishes 48 Pediatric Milestones

2012: What the milestones added Descriptions of level-specific learner behaviors along a developmental continuum Prompted for a longitudinal approach to teaching and assessment of skill from knowledge to bedside application Required observation-based assessments

Pediatricians Understand Milestones

MK-1: “The EBM milestone” Evidence-based practice, self-regulated learning,

Teaching and Learning EBM Skills Acquire knowledge and skills Longitudinal experiences Use and practice at the bedside Road map, not only for observation of performance but for curricula-with specific attention to helping learners get from one level to the next through on-going experiences

Teaching and Learning EBM Skills Acquire knowledge and skills Didactics E-learning modules Small group activities Computer labs Longitudinal experiences Journal Clubs CATs On-going didactics Use and practice at the bedside Role modeling Educational prescriptions Medical librarian on rounds Blogs, wikis Mobile devices, Apps

Pediatric Milestones PC1: Should I get a CXR on this asthmatic? PC4: Which antibiotic should I choose for this soft tissue infection? Should I get a CXR on this asthmatic? Which antibiotic should choose for this soft tissue infection? What would be the best source to read about community acquired pneumonia? One shouldn’t be able to progress in these milestones without the acquisition of EBM skills…. PBLI 2: What would be the best source to read about CAP?

Challenges of the Milestones Do not assess learners holistically Milestones can help us assess where the learner is along the continuum of individual skill development: “gather accurate information about a patient” “locate evidence related to their patient’s health problems” They don’t tell us whether someone can put it all together to take care of a patient

A brief history of “Competency-based Medical Education (CBME)” The ACGME Outcomes Project The ACGME Milestones Project Pre-1999 1999 2002-2008 2012 2014 2015 No duty hour restrictions Relatively few actual requirements Residencies required to integrate The Competencies into their curricula Residencies report data on 21 Milestones to ACGME ACGME/ABMS Adopt 6 Core Competencies And Sub-competencies 48 Pediatric Milestones published by ABP ABP “finalizes” Pediatric EPAs

Entrustable Professional Activities (EPAs) Describe routine activities of a pediatrician in specific contexts Require “workplace-based” application and assessment of skills (e.g. “OPAs” Observed Practice Activities) EPAs are executable within a given time frame; observable and measurable; and suitable for focused entrustment decisions.1 EPAs are units of work (e.g., anesthetic care of an uncomplicated patient), while competencies describe people’s abilities (e.g., knowledge, professional attitude, communication skill). Units of work and abilities of persons can be viewed as two dimensions of a grid. Competencies remain theoretical if not grounded in practice.

1. Provide consultation to other health care providers caring for children 2. Provide recommended pediatric health screening 3. Care for the well newborn 4. Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting 5. Provide a medical home for well children of all ages. 6. Provide a medical home for patients with complex, chronic, or special health care needs. 7. Recognize, provide initial management and refer patients presenting with surgical problems 8. Facilitate the transition from pediatric to adult health care 9. Assess and manage patients with common behavior/mental health problems 10. Resuscitate, initiate stabilization of the patient and then triage to align care with severity of illness 11. Manage information from a variety of sources for both learning and application to patient care 12. Refer patients who require consultation 13. Contribute to the fiscally sound and ethical management of a practice (e.g. through billing, scheduling, coding, and record keeping practices) 14. Apply public health principles and quality improvement methods to improve care and safety for populations, communities, and systems 15. Lead an interprofessional health care team 16. Facilitate handovers to another healthcare provider either within or across settings 17. Demonstrate competence in performing the common procedures of the general pediatrician Pediatric EPAs

1. Provide consultation to other health care providers caring for children 2. Provide recommended pediatric health screening 3. Care for the well newborn 4. Manage patients with acute, common diagnoses in an ambulatory, emergency, or inpatient setting 5. Provide a medical home for well children of all ages. 6. Provide a medical home for patients with complex, chronic, or special health care needs. 7. Recognize, provide initial management and refer patients presenting with surgical problems 8. Facilitate the transition from pediatric to adult health care 9. Assess and manage patients with common behavior/mental health problems 10. Resuscitate, initiate stabilization of the patient and then triage to align care with severity of illness 11. Manage information from a variety of sources for both learning and application to patient care 12. Refer patients who require consultation 13. Contribute to the fiscally sound and ethical management of a practice (e.g. through billing, scheduling, coding, and record keeping practices) 14. Apply public health principles and quality improvement methods to improve care and safety for populations, communities, and systems 15. Lead an interprofessional health care team 16. Facilitate handovers to another healthcare provider either within or across settings 17. Demonstrate competence in performing the common procedures of the general pediatrician Pediatric EPAs

Care for the normal well newborn EPA Title Care for the normal well newborn The functions of this activity include: Performing a physical examination to look for congenital anomalies Identifying and applying key evidence based guidelines for care of the newborn Caring for common problems that develop primarily in the nursery Using judgment to know when common problems can be handled at home and arrange for discharge and follow-up Assessing maternal/family readiness to care for the infant post discharge Transitioning care to the community practitioner Demonstrating confidence that puts new parents at ease Link with Domains of Competence _X_ Patient Care   ___ Medical Knowledge _X_ Practice-based Learning and Improvement _X_ Interpersonal & Communication Skills ___ Professionalism ___ Systems-based Practice _X_ Personal & Professional Development Critical Competencies PC 3: Transfer of care PC 5: Physical exam MK 2/PBLI 6: EBM ICS 1: Communicate with patients PPD 7: Self-confidence (updated ABP 3/11/14)

Integration of EBM learning, practice and assessment into the Clinical Environment Demonstration of skills with direct observation in the “workplace” Using educational experiences longitudinally to help learners advance to higher levels 1998 2015

Opportunities Assessment of EBM skills by direct observation in the clinical environment Observation tool development Educator development Use of clinical decision tools in bedside decision making Life-long learning curricula The Art of “Keeping Up” High Value, Cost-Conscious Care Requires understanding of guidelines, systematic reviews, diagnostic testing, therapy/harm The next EBM frontier

Summary In 2015 there is still a place in residency training for teaching and learning EBM skills EBM curricula should focus on the continuum from knowledge acquisition to application in the clinical environment Educators will need to: integrate educational activities into the existing/evolving clinical workflow create authentic opportunities for learners to demonstrate and teachers to assess EBM skill application To be most effective at teaching these skills (and most likely to get PD buy in for inclusion), educators will need to integrate ed opps into

Questions????