1. Simulation Training and Assessment in EMS Sean W Moore MD CM, FRCPC, emergency medicine University of Ottawa, Department of Emergency Medicine Associate Simulation Director, Medical Director Ornge Ottawa Bases May 16, 2013

2. Disclosures I do not have any affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.

3. My Background

4. Objectives Review the history of the use of simulation in EMS and emergency medicine Discuss ways in which simulation is best used for education and assessment Present Literature and controversy

5. History of Simulation Simulation in science broad topic Medical Simulation relatively recent High Fidelity Simulation developed over the past 50 years Bradley, P. (2006), The history of simulation in medical education and possible future directions. Medical Education, 40: 254–262.

6. Asmund Laerdal – 50s

7. Denson & Abrahamson 60s

8. Modern Simulators (uOSSC)

9. Mobile Simulators

10. Why Teach with Sim? Control for the area of interest Protect critically ill patients from novice/rookie Activation of participants to increase learning Teach skills which are not easily taught when patients are dying or not easily observed Specialized skills and uncommon serious situations The Utility of Simulation in Medical Education: What Is the Evidence? Yasuhaai Okuda, MD,^'^ Ethan O. Bryson, MD,^ Samuel DeMaria Jr, MD,^ Lisa Jacobson, MD,^ Joshua Quiñones, MD,^ Bing Shen, MD,^ and Adam I. Levine,

11. EMS Driving Skills

12. Field Amputation

14. Debriefing and Feedback This is where the money is... to find the frames that explain why learners do the things that they do.

15. Why Test with Sim? Ensure knowledge (written vs sim) Ensure clinical skill Ensure communication, flow, CRM Situations where difficulty to assess performance Boulet, John R., PhD. Summative Assessment in Medicine: The Promise of Simulation for High-stakes Evaluation. Academic Emergency Medicine 2008: 15:1017-1024.

16. Checklists Need to decide on checklist items: teams or consensus for determining? Multiple testing prior to seeing the success rate of measurement Guideline based? Experts? Subjective? Critical errors vs approach? Teaching the approach / rattle off multiple

17. Holistic / Global Holistic or Global Scoring looks at performance as a whole. Subjective but may be adequate Hawks vs Doves (Barker et al. The role of teamwork in the professional education of physicians: Current status and assessment recommendations. Jt Comm J Qual Patient Saf. 2005; 31:185-202) (Van Zanten et al. Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States medical license exam step 2 clinical skills exam. Acad Med. 2007;82(10 Suppl 1):S65-8.)

18. Validity Experts vs Novice examinees Performance Variability Do they correlate with clinical performance and scores on other examinations?

19. Test Scoring Issues Reliability - a bad day Multiple testing scenarios is the most important aspect Rater Training Multiple Raters - Blinded Raters Multiple tasks included in rating

20. Example of Performance

21. Is there Support in the Literature? About 1000 comparison studies to date using simulation Simulation is effective teaching tool, excellent examination tool

22. Evidence of Value Number of hours of simulation correlate strongly with educational objectives Can test and teach subjects in the arena familiar to the practitioner Provide feedback, repetitive practice, curriculum integration, controlled setting Effect of practice on standardised learning outcomes in simulation-based medical education William C McGaghie, S Barry Issenberg, Emil R Petrusa & Ross J Scalese

23. Early EMS Simulation Lit Bond, Kostenbader, & McCarthy, 2001 (prehospital professionals accepting of sim) Hall et al., 2005 (good to teach intubation) LeBlanc, MacDonald, McArthur, King, & Lepine, 2005 (dosage calculations after simulation stressors increased errors)

24. What about costs? Equipment expensive Operators Teach 1-4 at a time

25. What about Geography? Mobile Simulators? Go to Central Locations? Distributive Learning vs Train Trainers Locally?

26. Distance Simulation Computer based distance learning with virtual reality scenarios at infancy Getting better with technology

27. Efficiency of Throughput 51 Paramedic services…approximately 5200 paramedics 70 full time and 350 part time educators Using downtime vs dedicated days Testing and Teaching costs http://www.ottawaparamedics.ca/misc/OPA%20Survey%20Report%20 Final.pdf

28. High vs Low Fidelity Procedures and tasks taught using task trainers High Fidelity better for Crisis Resource Management Didactic and written exams also useful for knowledge base Putting it all together or prep for complex cases … sim is the way to go

29. Large Scale Simulations

30. Questions?