1. Outcome-based research in Obstetric simulation Dr Jo Crofts Academic Clinical Lecturer in Obstetrics University of Bristol, UK

2. Outline Why simulation training is required 9 years of progress Simulation and clinical outcomes Characteristics of effective training Outcome based research is required Future of obstetric simulation

3. Childbirth is dangerous 1000 women die every day due to pregnancy and childbirth complications that ‘could have been prevented’

4. The Safety Problem 1 : 12 labours associated with adverse outcomes Nielsen P at al, Obstet Gynecol 2007 50% adverse outcomes preventable with better care CESDI – 4th Annual Report. 1997 CEMD – Why Mothers Die. 1998 CEMACH – Saving Mothers Lives 2007

5. Very expensive NHS Litigation Authority £633 million in settled negligence claims 2007-08 £221 million for Obstetric Claims £1 billion for additional bed days to deal with preventable harm Human costs ? House of Commons Health Committee: Patient Safety Report. 2009

6. Training Simulated emergencies should be organised to improve management of rare obstetric emergencies CESDI – 4th Annual Report 1997 CEMD – Why Mothers Die 1998 NHSLA. CNST Maternity Standards 2000 CEMACH – Saving Mothers Lives 2007 Kings Fund: Safer Births everybody’s business. 2008 Include teamwork training To Err is Human: building a safer health system. 2000

7. 9 years of progress 2003 No objective evaluation Difficult to demonstrate any benefit Decade after first recommendation - neither a national curriculum, nor a system for provision Black R & Brocklehurst P. BJOG 2003

8. Outcome based research Evidence of Effectiveness Level 1ReactionSatisfaction Level 2LearningMCQs, Skills Level 3BehaviourPatient care Level 4ResultsClinical Outcome Kirkpatrick, D. (1998). Evaluating Training Programs: The four levels. San Francisco, Berrett-Kochler Publishers.

9. Training Intervention Post-training Assessments 3 weeks, 6 months and 12 months Pre-training Assessment SaFE Study Local Hospital No team training One day Simulation Centre No team training One day Local Hospital Team training Two days Simulation Centre Team training Two days MCQ Clinical Scenarios MCQ Clinical Scenarios

10. Knowledge Summary Significant increase in knowledge following training 93% increased MCQ score Knowledge at 6 & 12 months was significantly higher than pre-training None of the training interventions appeared to be superior Crofts, J., D. Ellis, et al. (2007). "Change in knowledge of midwives and obstetricians following obstetric emergency training: a randomised controlled trial of local hospital, simulation centre and teamwork training." BJOG: An International Journal of Obstetrics and Gynaecology 114(12): 1534-1541.

11. Eclampsia 140 staff randomised to training on patient-actor or whole body simulator Following training completion of basic tasks (87% to 100%) administration of MgSO 4 (61% to 92%) medication given 2 minutes earlier No differences in training style except improved communication with actress Ellis et al. (2008). "Hospital, Simulation Center, and Teamwork Training for Eclampsia Management: A Randomized Controlled Trial." Obstet Gynecol 111(3): 723-731.

12. Shoulder Dystocia

13. Simulation of SD

14. SaFE: SD skills Action % achieved Pre-trainingPost-trainingSignificance All basic manouevres 81.494.7 P=0.002 Achieved delivery 42.983.3 P<0.001 Good communication 56.882.6 P<0.001

15. High v Low fidelity mannequin ActionPROMPTLowSignificance Achieved delivery 94%72%P=0.002 Delivery time 135 s161 sP=0.004 Mean peak force 102 N112 N P=0.242

16. Shoulder dystocia simulation 140 staff randomised Training is required Pre-training 43% successful shoulder dystocia Simulation improves performance Post-training 83% successful shoulder dystocia PROMPT mannequin Improved delivery rate (72% vs 94%) Shorter delivery time (161s vs 135s) Crofts, Bartlett, et al. (2006). Obstet Gynecol 108(6): 1477-85.. Crofts, Fox, et al. (2008). Obstet Gynecol 112(4): 906-12.

20. Not all training equal Two UK cities Similar demographic Shoulder dystocia training started in 2000 City 1: 70% decrease in OBPI City 2: 100% increase in OBPI Draycott et al. Obstet Gynecol 2008; 112: 14-20 MacKenzie et al. Obstet Gynecol 2007; 110: 1059-1068

21. Differences in training Effective 98% staff Multi-professional PROMPT model Simple algorithm Ineffective ~60% staff Separate Low fidelity model Mnemonic

23. Labour & Delivery CRM trial 15 hospitals (6 military, 9 civilian) 28,536 deliveries 4 month intervention 4 hour didactic training (CRM) Team structure implementation Primary outcome: reduction in overall frequency of adverse outcomes Nielsen PE, Goldman MB, Mann S, Shapiro DE, Marcus RG, Pratt SD, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol. 2007 Jan;109(1):48-55.

24. Adverse Outcome Index Adverse EventScore Maternal death750 Intrapartum / neonatal death > 2500g400 Uterine rupture100 Maternal admission to ITU65 Birth trauma60 Return to theatre or delivery suite40 Admission to NICU >2500g for >24 hours35 Apgar <7 at 5 minutes25 Blood transfusion20 3 rd or 4 th perineal tear5

25. Labour & Delivery CRM trial No difference in adverse outcomes (both groups improved) Problems CRM does not work / as implemented ? Short implementation period Wrong measures ? Hawthorne effect ? Underpowered ?

26. Nine years of progress What works Where Why What next……?

27. Common Effective Themes Simulation of emergencies High fidelity training tools Situated ‘Local’ training Nearly 100% staff Multi-professional Insurance based financial incentives Siassakos, Crofts, et al. (2009). "The active components of effective training in obstetric emergencies." Bjog 116(8): 1028-32.

28. Does Simulation work ? Yes Increasing retrospective data suggesting improvements in neonatal outcome after the introduction of simulation training (Some, but not all)

29. Can we do better ? Yes Increasing retrospective data suggesting improvements in neonatal outcome after the introduction of simulation training

30. Nine year vision Effective evidenced based training to reduce preventable harm All staff All mothers & babies Improved training materials Commit to more, and better research for the future Prospective Hard clinical outcomes

31. The Future Whole body mannequins Sepsis Maternal collapse Virtual reality Instrumental delivery

32. The Future Accessible training Simple training aids

33. Simulation training is required 1000 women die every day due to pregnancy and childbirth complications that ‘could have been prevented’ Almost all of them (99%) live and die in developing countries World Health Organisation

34. Thank you jo.crofts@bristol.ac.uk