Simulation Training and Assessment in EMS Sean W Moore MD CM, FRCPC, emergency medicine University of Ottawa, Department of Emergency Medicine Associate Simulation Director, Medical Director Ornge Ottawa Bases May 16, 2013

Disclosures I do not have any affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.

My Background

Objectives Review the history of the use of simulation in EMS and emergency medicine Discuss ways in which simulation is best used for education and assessment Present Literature and controversy

History of Simulation Simulation in science broad topic Medical Simulation relatively recent High Fidelity Simulation developed over the past 50 years Bradley, P. (2006), The history of simulation in medical education and possible future directions. Medical Education, 40: 254–262.

Asmund Laerdal – 50s

Denson & Abrahamson 60s

Modern Simulators (uOSSC)

Mobile Simulators

Why Teach with Sim? Control for the area of interest Protect critically ill patients from novice/rookie Activation of participants to increase learning Teach skills which are not easily taught when patients are dying or not easily observed Specialized skills and uncommon serious situations The Utility of Simulation in Medical Education: What Is the Evidence? Yasuhaai Okuda, MD,^'^ Ethan O. Bryson, MD,^ Samuel DeMaria Jr, MD,^ Lisa Jacobson, MD,^ Joshua Quiñones, MD,^ Bing Shen, MD,^ and Adam I. Levine,

EMS Driving Skills

Field Amputation

Debriefing and Feedback This is where the money is... to find the frames that explain why learners do the things that they do.

Why Test with Sim? Ensure knowledge (written vs sim) Ensure clinical skill Ensure communication, flow, CRM Situations where difficulty to assess performance Boulet, John R., PhD. Summative Assessment in Medicine: The Promise of Simulation for High-stakes Evaluation. Academic Emergency Medicine 2008: 15:

Checklists Need to decide on checklist items: teams or consensus for determining? Multiple testing prior to seeing the success rate of measurement Guideline based? Experts? Subjective? Critical errors vs approach? Teaching the approach / rattle off multiple

Holistic / Global Holistic or Global Scoring looks at performance as a whole. Subjective but may be adequate Hawks vs Doves (Barker et al. The role of teamwork in the professional education of physicians: Current status and assessment recommendations. Jt Comm J Qual Patient Saf. 2005; 31: ) (Van Zanten et al. Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States medical license exam step 2 clinical skills exam. Acad Med. 2007;82(10 Suppl 1):S65-8.)

Validity Experts vs Novice examinees Performance Variability Do they correlate with clinical performance and scores on other examinations?

Test Scoring Issues Reliability - a bad day Multiple testing scenarios is the most important aspect Rater Training Multiple Raters - Blinded Raters Multiple tasks included in rating

Example of Performance

Is there Support in the Literature? About 1000 comparison studies to date using simulation Simulation is effective teaching tool, excellent examination tool

Evidence of Value Number of hours of simulation correlate strongly with educational objectives Can test and teach subjects in the arena familiar to the practitioner Provide feedback, repetitive practice, curriculum integration, controlled setting Effect of practice on standardised learning outcomes in simulation-based medical education William C McGaghie, S Barry Issenberg, Emil R Petrusa & Ross J Scalese

Early EMS Simulation Lit Bond, Kostenbader, & McCarthy, 2001 (prehospital professionals accepting of sim) Hall et al., 2005 (good to teach intubation) LeBlanc, MacDonald, McArthur, King, & Lepine, 2005 (dosage calculations after simulation stressors increased errors)

What about costs? Equipment expensive Operators Teach 1-4 at a time

What about Geography? Mobile Simulators? Go to Central Locations? Distributive Learning vs Train Trainers Locally?

Distance Simulation Computer based distance learning with virtual reality scenarios at infancy Getting better with technology

Efficiency of Throughput 51 Paramedic services…approximately 5200 paramedics 70 full time and 350 part time educators Using downtime vs dedicated days Testing and Teaching costs Final.pdf

High vs Low Fidelity Procedures and tasks taught using task trainers High Fidelity better for Crisis Resource Management Didactic and written exams also useful for knowledge base Putting it all together or prep for complex cases … sim is the way to go

Large Scale Simulations

Questions?