1. Safer Surgical Services : Are system and culture interventions synergistic? Peter McCulloch, University of Oxford, England

2. Conflict of Interest Paid to give talks: No Paid for my advice: No Paid for me to attend Conferences: NIHR Funded my research: – NIHR (PGfAR Programme) – BUPA Foundation – Health Foundation

3. Waves of Safety Research Is there a problem? – IoM report How big is it? – Vincent et al, Gawande, What causes it? – Lingard, Sevdalis, Catchpole What works to fix it? Why does it work?

4. What do safety interventions do? Systems Improving Systems – Industrial QI engagement and PDCA approaches – SOP/Checklist approaches – IT based solutions Culture Enhancing Culture – CRM teamwork training – Supportive policies McCulloch, P. and Catchpole K. A three-dimensional model of error and safety in surgical health care microsystems. Rationale, development and initial testing. BMC Surg, 2011. 11: p. 23-7.

5. S3 Hypothesis Fixing CULTURE and SYSTEM provides SYNERGISTIC benefits to team performance

6. Testing the Hypothesis Multiple comparisons required Hugely variable environment/context Available process measures subjective Outcome measures very distal/insensitive Signal/Noise ratio therefore problematic

7. Programme Design Suite of identical CONTROLLED prospective studies Standardised, validated process and outcome measures used throughout. Planned meta-analysis to allow evaluation of over-arching questions

8. Study Design Pre-intervention data collection 3-6/12 Intervention Period 3- 6/12 Post-intervention data collection 3-6/12 Control group Control group in SAME TRUST, doing work of SIMILAR NATURE Observation and Intervention separation No blinding (Alas..) Observer pairs watch whole procedure

9. Measures Non-technical skillsOxford NOTECHS II Non-technical skillsOxford NOTECHS II Technical performanceGlitch rate Technical performanceGlitch rate Safety CultureWHO checklist adherence Safety CultureWHO checklist adherence LOS Clinical Outcome30 day complications 90 day readmissions PROMsEQ5D Measure of improvement: comparison of Before: After ratio for Active vs Control group

10. Interventions “Lean” Approach (System) – 1 day course plus coaching support during intervention period – Emphasised staff engagement and freedom to set agenda – Used 5S, process mapping, PDCA cycles, poka yoke etc Standardisation Approach (System) – 1 day course on standards and principles plus coaching support – Emphasised staff engagement and freedom to set agenda – Aimed to standardise and visualise work process Crew Resource Management training (Culture) – Based on previous study – 1 day course plus 6 weeks coaching – Partner with Atrainability

11. Studies and Settings 1.CRM alone: Kettering DGH (Ortho/Vascular) 2.SOP alone: Nuffield Orthopaedic Centre (Hip/Knee teams) 3.Lean alone: UHCW Trust (Trauma/Ortho) 4.CRM & SOP: UHCW Trust (Hip/Knee teams) 5.CRM & Lean: NOC (Plastics/Hip & Knee)

12. Results Orthopaedic, Trauma, Plastic and Vascular surgery observed 5 Hospital sites in 3 Trusts 453 operations observed >2,000 hours observer time in theatre 5,124 patient clinical outcomes analysed

13. Change in Oxford NOTECHS II [Change in Active – Change in Control] * p=0.047 * P = 0.002 * p=0.058

14. Change in Glitch Count * p<0.001

15. Change in WHO compliance % Relative Improvement * P<0.001 * P=0.032

16. Relative Changes in Clinical Outcomes (%)

17. Conclusions & Reflections Single intervention modalities do what you would expect – but weakly Dual intervention increased both BREADTH and STRENGTH of response CULTURE Motivation Understanding of aims Focus Buy-in SYSTEM Understanding of systems Confidence Ability to Make change

18. Importance of Study Design Control groups essential Valid standard measures important Lack of blinding a weakness Challenges: – Hawthorne effects – Contamination It CAN be done!

19. Barriers & Future Research Compartments LACK: – TIME for IMPROVEMENT – KNOWLEDGE – SUPPORT – MOTIVATION Nurses & PAMS Junior Doctors Senior Doctors Middle Management RESEARCH IMPLICATIONS  How to Upscale?  Measuring Resistance  Trialling Implementation plans  Step-Wedge trial OR?

20. Radical Change? Strengthen line management Clarify Accountability Abolish professional silos

21. Thank You Captain Trevor Dale Dr Karen Barker Prof Renee Lyons Prof Damian Griffin Surgical Staff at: Oxford University Hospitals University Hospitals Coventry & Warwick Kettering District General Hospital Dr Steve New Dr Ken Catchpole Dr Lauren Morgan Sharon Pickering Mohammed Hadi Eleanor Robertson Laura Blakely