Introduction Results Objectives Conclusions Methods Limitations Critical Care Simulation Curriculum to Assess Pediatric Residents on ACGME Developmental Milestones Joshua D. Patterson, MD; Matthew A. Borgman, MD; Heather M. Delaney, MD; Renée I. Matos, MD, MPH Department of Pediatrics, San Antonio Military Medical Center, Ft Sam Houston, TX 78234 Introduction Results In 1999, the Accreditation Council for Graduate Medical Education (ACGME) introduced the six domains of clinical competency to better assess resident progression In 2009, ACGME began a multiyear process to restructure its system to be based on educational outcomes, which resulted in the development of a consensus set of milestones in the six competencies in each residency training program1 Milestones were organized with a 5-level model beginning with goals for entering training in a new specialty, progressing to graduation goals, and ending at an advanced level2 ACGME put forth 21 milestones in 2012 to better assess pediatric residents; however, subjective assessment of can be difficult and may vary based on patient acuity and volume3 Jan 2013 – Nov 2015: 39 PGY2 residents trained Milestones addressed: Decision Making (PC4), Management Plans (PC5), Identifying one’s knowledge (PBLI1), Emotional response (ICS2), Respect for others (Prof1), and Coordinating Care (SBP1) Simulation scores contributed to milestone scores End-of-year milestones for all residents were assessed between Level 2 to 3.5 based on all PGY2 rotations The Asthma and Sedation simulations were more likely to correlate with the average milestone score than the PEA or TBI scenarios 20 of 158 (12.7%) scenarios were repeated for failure Table 1. Simulation Scores Sim # Mean Score SD Failures R2 Milestone Correlation S1 84.48% 8.21% 3 0.24 S2 69.97% 14.23% 8 0.00 S3 75.52% 8.07% 2 0.37 S4 80.86% 10.61% 7 0.05 Objectives Our 6-bed PICU (Average daily census = 2.7) has a rotating PGY2 who previously would receive variable lectures and ad hoc simulation scenarios To standardize learning and assessment, in January 2013 we initiated a standardized curriculum of lectures and high fidelity simulated pediatric scenarios, including objectives and quantifiable assessment tools This curriculum synchronized with the pediatric resident ICU rotation and aided with milestone calculations and evaluations Conclusions Hospital-based simulations may enhance multidisciplinary patient care through accessible, repetitive team training4 An integrated lecture and crisis-based simulation curriculum aids in assessing pediatric residency ACGME milestones Simulation creates opportunities for objective evaluation during a 4-week rotation in a low-volume pediatric ICU Pediatric residents demonstrated difficulty with basic PALS skills & knowledge, especially pulseless simulations, resulting in added simulation exposure during ward block Less clinically challenging simulation scenarios were more likely to correlate with global milestone assessments Quantifiable simulation assessments may be more sensitive in detecting weaknesses in clinical competency than the more qualitative ACGME milestones assessment Methods 4 high-fidelity simulations (SimJunior & SimBaby, Laerdal): S1: Procedural Sedation with associated Complications S2: Pulseless Electrical Activity due to hyperkalemia S3: Asthmatic respiratory failure with ventricular tachycardia S4: Traumatic brain injury and hyponatremic seizures Objective assessment forms created for each scenario Same four attending physicians gave standardized lectures and graded simulation (inter-rater reliability, Kappa = 1.0) After the first 10 residents, the scenarios were considered failures when they scored 2 SD below the mean, and then were repeated. Only initial scores were utilized for results All residents were certified in PALS prior to training Limitations Curriculum developed before milestones Retrospective analysis Milestone scores are based on a collective summary of all rotations and evaluations Limited evaluation of milestones (6 of 21) Future Direction Prospectively create scenarios or curriculum to specifically target ACGME milestone assessments Reference Swing SR, Clyman SG, Holmboe ES, William RG. Advancing resident assessment in graduate medical education. J Grad Med Educ 2009; 1:278-86 Swing SR, Beeson MS, Carraccio C, Coburn M, et al. Educational milstones development in the first 7 specialties to enter the next accreditation system. J Grad Med Educ 2013; 5:98-106 ACGME, ABP. The Pediatric Milestones Project, July 2015. Weinstock PH, Kappus LJ, Kleinman ME, et al. Toward a new paradigm in hospital-based pediatric education: The development of an onsite simulator program. Pediatr Crit Care Med 2005; 6:635-641. Figure 2. During the simulation scenarios, both SimJunior© and SimBaby©, pictured here, were used by pediatric residents The views expressed herein are those of the authors and do not reflect the official policy or position of San Antonio Military Medical Center, the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Department of the Army, Department of the Air Force, or U.S. Government. Texas Pediatric Society Electronic Poster Contest