Emergent/Critical Care Imaging Simulation Resident Assessment and Training Tool Linda Lanier, MD Associate Professor and Associate Chairman, Department of Radiology Chris Sistrom, MD, MPH, PhD Anthony A. Mancuso, MD Professor and Chairman, Department of Radiology

Emergent/Critical Care Imaging Simulation

What are we trying to Simulate?

What are we trying to Simulate? ALL of the skills necessary for service as a radiologist performing Emergent and Critical Care Imaging

Observational skills Full DICOM image set ( 1-3000 images) Image manipulation, optimal viewing All Modalities: CR, CT, CTA, US, MRI, MRA, NucMed, Fluoro All Systems: Body (Chest/Abdomen/Pelvis/GI/GU), Neuro/ENT, Pediatric, MSK, Vascular

Confident Determination of Normal Observational skills Confident Determination of Normal Downside of Overcall Comprehensive Search Pattern

Confident Determination of Normal Identification of Key Findings Observational skills Confident Determination of Normal Identification of Key Findings Critical/Important/Incidental Findings Clinical Context (WITH or WITHOUT History) Prioritize Findings

Observational skills Confident Determination of Normal Identification of Key Findings Recognition of Acuity

Confident Determination of Normal Identification of Key Findings Observational skills Confident Determination of Normal Identification of Key Findings Recognition of Acuity Communication of Results - Actual Practice Phone/In person Report Clarity

Confident Determination of Normal Identification of Key Findings Observational skills Confident Determination of Normal Identification of Key Findings Recognition of Acuity Communication of Results - Simulation Response is Free Text Entry No multiple choice No cueing or ability to “game” responses No True/False

Observational skills Confident Determination of Normal Identification of Key Findings Recognition of Acuity Communication of Results Initiate Appropriate action

Observational skills Confident Determination of Normal Identification of Key Findings Recognition of Acuity Communication of Results Initiate Appropriate action Stamina

Building the Case Library Core Curriculum ( 600-700 Topics) Collect and vet cases (10-20 cases per topic) Enter Case data into RCMS Grading key Reports

Building Simulations

The HOT SEAT

Pick a case from the workflow …

Free text response ….

Select acuity level ….

Submit case ….

Oooops … need addendum ?

Select a new case from workflow list …..

The “Front End” PACS Workstation 8 Hour Shift 65 Cases Typical Work Flow Style

The “Front End” PACS Workstation 8 Hour Shift 65 Cases Typical Work Flow Style (Minus the phone calls, interruptions …..)

The “Back End”

The “Back End” Tracking Capability

Tracking Capability

The “Back End” Tracking Capability Grading

Grading Tool

Results to Program Director

Results to Program Director

Results to Individual Residents

Video Review of Cases Residents

Emergent/Critical Care Imaging Simulation 2011-2014 YEAR SIMULATION Total # Residents R LEVELS INSTITUTIONS 2011 SIM 1 22 R1-12 R2-10 UF 2012 36 R1-20 R2-16 UF, UF-J, Mt. Sinai 2013 SIM1 66 R1-25 R2-24 R3-11 R4 -6 U Mississippi 2014 SIM 2 103 R1-63 R2-31 R3-6 R4-3 UF, UF-J, Mt Sinai SUNY, Creighton, Baylor-Dallas, Geisinger, U Kentucky

Emergent/Critical Care Imaging Simulation 2015 YEAR SIMULATION Total # Residents R LEVELS INSTITUTIONS 2015 SIM 3 Taken 127 R1 94 R2 33 Albert Einstein Dartmouth Geisinger Hahnemann Mount Sinai Miami Stony Brook SUNY- Brooklyn University of Florida University of Florida-Jax University of Kansas University of Kentucky University of Louisville University of Mississippi University of Pennsylvania University of Vermont University of West Virginia Registered (not yet taken) 19 R1 19 Army Air Force SIM Monmouth 2 R1 1 R2 1 Monmouth Medical Center

Averages and Ranges YEAR SIMULATION R LEVEL (N) AVG SCORE RANGE 2011-2013 SIM 1 R1 (57) R2 (50) R3 (11) R4 (6) 61% 74% 73% 78% 42%-74% 65%-85% 60%-85% 73%-81% 2014 SIM 2 R1 (63) R2 (31) R3 (6) R4 (3) 58% 70% 66% 68% 39%-78% 48%-81% 54%-72% 59%-79% 2015 SIM 3 R1 (94) R2 (33) 59% 67% 47%-68% 63%-74%

Simulation – A Self Discovery Process Identifies the Educational Tasks Identifies the Knowledge Gaps Individuals Programs Across institutions Identifies Areas of Uncertainty Trains Toward Competency

Future SIM - Tiered Options SIM - Scoring by key Findings SIM - Natural Language Processing (NLP) SIM – Adaptive testing? Curriculum – “Emergent/Critical Care Imaging” Interactive Non synchronous Case based Educational Video Clips/Flicks

Other uses of Imaging Simulation Platform Training for Other specified Groups ED Stroke Team Med students Night Hawk Practice Hires Hospital safety

Pathway to Competency

Emergent/Critical Care Imaging Simulation

Content SIM 1

Content SIM 2

Content SIM 3