1. Educational Solutions for Workforce Development Multidisciplinary Health Foundation SHINE Award 2012/13 Enhanced Significant Event Analysis: A Human Factors Systems Approach for Primary Care Paul Bowie, Elaine McNaughton, Deirdre Holly, David Bruce www.nes.scot.nhs.uk/shine/

2. Educational Solutions for Workforce Development Multidisciplinary What is a Significant Healthcare Event? Significant Event Patient Safety Incident Adverse Event (Avoidable Harm) Near Miss (Potential Harm) Purely Reflective Other Quality of Care Issue ‘Positive’ Event

3. Educational Solutions for Workforce Development Multidisciplinary Investigations of Significant Events Poorly Conducted: Problems and Impacts Incidents are highly selective (or non-engagement) Negative feedback (interferes with ability to assimilate & process information beyond the ‘self’ level) Second-victim syndrome (impact on health & wellbeing of clinician: guilt, embarrassment, shame…) Perceived blame culture (fear, distrust, punitive action, litigation…) Lack of a structured analytical framework (long standing issue) Many SEAs demonstrate a lack of ‘systems thinking’ Most clinicians attribute events to their own actions/inactions

4. Educational Solutions for Workforce Development Multidisciplinary Lack of meaningful and constructive investigations Missed opportunities to learn & improve (personal, team & organisational) SEA becomes a tick-box exercise Increased workforce stress, frustration & sick levels Wasted time, energy and resource Low engagement in formal incident reporting Investigations of Significant Events Poorly Conducted: Problems and Impacts

5. Educational Solutions for Workforce Development Multidisciplinary History of SEA (compared to hospital based techniques) Case-based discussion (Bradley, 1992; Buckley, 1992; Pringle et al, 1994) Flanagan’s Critical Incident Technique (John C Flanagan, 1954) Embedded in practice (most Scottish primary care professions) Quality of SEA (Bowie et al, 2009; McKay et al, 2007)

6. Educational Solutions for Workforce Development Multidisciplinary What We Set Out to Achieve We aimed to design, develop and test a theory- informed ‘guiding tool’ to support the SEA process in primary care settings.  Overcome SEA deficiencies by introducing human factors systems principles  Highlight and differentiate the interactions between the individual professional and the immediate workplace and wider organisational issues at play.  Individual Level: guide clinicians to reflect upon and contextualise their emotional reactions - achieve a state of psychological readiness to move on.  Team Level: a systems-centred analysis of the significant event.  Underlying assumptions: gain a deeper understanding of the human-system interactions at play may lessen –ve emotional reactions and apportioning of personal blame may lead to more meaningful and effective action plans for improvement.

7. Educational Solutions for Workforce Development Multidisciplinary How did we do it? Multi-professional steering group/project manager Tapped into existing human factors and safety science expertise Reviewed relevant literature Aim 1. Design a conceptual framework (error theory and ergonomic model) Aim 2. Develop ‘guiding tools’ (individual & team levels) Aim 3: Recruited clinicians and managers (qualified and in-training) to test our interventions Formal evaluation of impact Final project report

8. Educational Solutions for Workforce Development Multidisciplinary Methods – Recruitment, Testing & Evaluation Professional Groups: Community Pharmacy, General Dental Practice, General Medical Practice, Optometry and Clinical Psychology across NHS Scotland (n=180) Educational Leaders identified suitable groups to approach (e.g. speciality or vocational training, trainers/tutors) Participants undertook enhancedSEA as part of existing educational, appraisal or contractual arrangements (or none) Emailed Flyer and directed to dedicated website Website – one-stop-shop Evaluation (mixed methods): captured learning - attitudes, experiences, perceptions of feasibility/usability/impact of guiding tools etc.

9. Educational Solutions for Workforce Development Multidisciplinary Our Results 1. Development Outputs

10. Educational Solutions for Workforce Development Multidisciplinary CONCEPTUAL FRAMEWORK for enhancedSEA PEOPLEACTIVITYENVIRONMENT  Individual e.g. physical, psychological, personality or social issues; cognitive factors, competence, skills, attitudes, risk perception, training issues  Team e.g. roles, support, communication, leadership  Patient e.g. clinical condition, physical, social, psychological, relationship factors  Others e.g. other health and social services  Complexity of work process or task, guidelines, policies and procedures e.g. not up-to-date, not available, unclear/unusable, not followed  Design or organisation of work process of system e.g. level of complexity, workload, poor design  Equipment e.g. positioning, not available, not working, not calibrated, usability issues  Work setting e.g. staffing, environmental conditions, workload or hours of work, design of physical environment, administrative and/or time factors  Organisational e.g. safety culture, priorities, external risks, organisational structure  Communication e.g. verbal, written, non verbal systems, poor communication, failure to communicate  Education and training e.g. supervision, competence, availability/accessibilit y, appropriateness  Societal, cultural and regulatory influences

11. Educational Solutions for Workforce Development Multidisciplinary THE GUIDING TOOLs enhancedSEA Method The new approach is divided into three parts: 1. Small Personal Booklet (with 4 card inserts) 2.A3 size Desk Pad 3.New written format for enhancedSEA reports

12. Educational Solutions for Workforce Development Multidisciplinary 1. A small 12-page Personal Booklet (with 4 card inserts) to help individuals reflect on the potential emotional impacts of a significant event - and their own role in the event - by using human factors principles to gain a clearer understanding of all of the contributory factors involved.

13. Educational Solutions for Workforce Development Multidisciplinary 2. An A3 size Desk Pad for the care team, the sheets from which can be distributed to all those who attend a team meeting to analyse significant events. Each sheet contains instructions and prompts to guide the care team to take a systems-based approach to analysing the event in question and take notes on what was agreed – a small set of card prompts may also be used in conjunction.

14. Educational Solutions for Workforce Development Multidisciplinary 3. A re-designed written report format for enhancedSEA – to prompt a systems based analysis.

15. Educational Solutions for Workforce Development Multidisciplinary Our Results 2. Recruitment & Testing 242 registered via website 126 submitted enhancedSEA reports 117 completed post-study survey

16. Educational Solutions for Workforce Development Multidisciplinary Participation and Completion Rates

17. Educational Solutions for Workforce Development Multidisciplinary Our Results 3. Evaluation of Interventions

18. Educational Solutions for Workforce Development Multidisciplinary Evaluation (n=117) StatementsAgreement Levels (%) Using the enhancedSEA approach led to action that actually improved (or will improve) patient care 80 Personal Booklet I fully understood the purpose of this booklet 85 The booklet was practical to use in the workplace 72 I found the four cards inserted in the booklet to be helpful 55 I found the tool to be very relevant to dealing with the personal emotions related to a significant event 75 A3 Desk Pad I fully understood the purpose of this Desk Pad tool 72 The Desk Pad tool was practical to use in the workplace 55 Using the Desk Pad tool helped focus the SEA on system issues rather than just on the role of individuals 68 I found the tool to be very relevant to dealing with the personal emotions related to a significant event 60 enhanced-SEA Report Format The content of this Report Format was clearly written and easy to understand 77 I would recommend this Report Format to other colleagues 70 I will use this Report Format the next time I write up a significant event analysis 70

19. Educational Solutions for Workforce Development Multidisciplinary Statement and levels of agreement (%)Pre-Post- I have a good understanding of what a "significant event" is in the context of my healthcare role 8295* I fully understand how to undertake and lead a significant event analysis 6695* Generally, being involved in a significant event in the workplace has a strong emotional impact on me 4443 Generally, being involved in a significant event in the workplace heightens my personal stress levels 4036 The procedures in this workplace are not clear on how to highlight significant events 2822 When a significant event is analysed, it feels like the person is being written up, not the problem 1913 Poor design of systems, rather than the actions of humans, is the biggest factor contributing to significant events in the workplace 3950* I have a good understanding of the discipline of “human factors” 3577* Highlighting significant events is a good way of identifying staff who need additional training 2434* I think undertaking SEA is a demanding and difficult task 4852 Evaluation (n=117) *P<0.05

20. Educational Solutions for Workforce Development Multidisciplinary Comments related to the Personal Booklet “The step by step guide helped me to carry out significant event analysis in a quick and comprehensive manner making it more likely I will carry out formal SEA in the future with more confidence and frequency”. “I don’t think I looked at the cards. The booklet itself was very clear, breaking everything down nicely to help write the report. The enclosed example was also very helpful and I will certainly refer to the booklet again next time I complete a significant event analysis”. “I think that this tool might be more useful for some SEAs than others – for example it might be useful in understanding emotions surrounding a serious event, but many SEAs are more minor or administrative in nature. I did however find it helpful as an aide-memoire to ensure that I had thought of all possible aspects contributing to the event”. “I still found it difficult to analyse the situation and feelings but this did encourage me to be more objective and see why/how things happened a certain way”. “I found it a bit complicated, not all was relevant”

21. Educational Solutions for Workforce Development Multidisciplinary Comments related to the A3 Desk Pad “I think it is a persistent reminder of what eSEA is trying to achieve. I would not generally refer to it when doing my eSEA but having it on my desk continually reminds of the human factors approach and so was useful (similar to having a pen with a drug name on it!!)” “The content on the desk pad sheet was useful for our meeting but we ended up taking notes on a separate sheet and we could really have just used the personal booklet for guidance”. “Even in a meeting scenario-our staff are not around a table but just in our waiting room so big and bulky to use. Could do with more space for note taking if used. Found the smaller tools more appropriate in this setting/environment”. “Quite a large thing to have and wasn’t really sure where to keep it-significant events are hopefully relatively infrequent and not sure I needed something so big that don’t use that often”. “A cumbersome waste of space and paper”

22. Educational Solutions for Workforce Development Multidisciplinary Other Selected Quotes “I think SEA is a difficult and demanding process but these tools simplified the process for me and made it easier to try to be more objective” “I think the human factors model is a very useful approach to looking at an SEA and without doubt caused me to think differently about this SEA and feel the outcomes were more meaningful”. “Very worthwhile. Definitely made the SEA feel more in depth / thorough. Whole team very approving. Developed some really useful action points” “If eSEA was recognised, it would be more beneficial and more useful than existing SEA” “We’ll look at things more closely, when an SEA comes in use, use of the eSEA will have greater impact, will influence initial discussions” “eSEA breaks stuff down further that you wouldn’t have considered....which I wouldn’t have considered before” “I found it made the process more laborious and confusing in some ways. I think the booklet was helpful but the report format needs to be simplified. Some of it felt like writing in order to fill in boxes. I think we would have come to the same conclusions if I had used our normal format”.

23. Educational Solutions for Workforce Development Multidisciplinary Early Feedback from SEA Peer Reviewers Subjective/Objective Comparisons – ‘Old’ vs ‘New’ Acts as a ‘forcing function’ – deeper analysis More straightforward to actually understand and peer assess Feeling that more challenging events are being selected Overall standard has improved compared with ‘old’ system Potential for objective comparison (old vs new assessment ratings) Understanding of why event occurred Learning needs identified Change and improvements Overall global rating

24. Educational Solutions for Workforce Development Multidisciplinary Overall Impact of enhancedSEA “Use of eSEA might improve things a bit. Events are broken down into so many things that it would never be down to one person as there are so many links in the chain” “I think it was definitely the most thorough SEA that we’ve done for a while and everybody commented that it went very well and it had some very good outcomes for us all things that I really think will have a decent impact” “I think it’ll get people to think more into just why significant event analysis happens and it’s like a big elephant in the room, of course you’re embarrassed and you will have these emotions and I think it encourages people to realise that those emotions are there, we’re not working as robots if you like. And it gets people to think more about being human” “…it made you less stuck for sort of thinking about things that’d gone wrong and why there’s sort of different aspects had gone wrong because there was lots of problems there to get you going”.

25. Educational Solutions for Workforce Development Multidisciplinary What has gone well? We made the project aims more realistic and manageable. Our core group has stayed together and functioned well – migrating to broader Q&S organisational group National Educational Leaders clearly interested, very supportive and helpful, also Executive buy-in The approach developed based on solid theory and has high face validity (particularly amongst those informed in safety) Post-pilot implementation looks promising, but may take a while to embed for some professions (e.g. optometry and psychology) Infrastructure to improve and build upon – a lasting legacy TROJAN HORSE STRATEGY SEEMS TO HAVE WORKED!

26. Educational Solutions for Workforce Development Multidisciplinary What could have been better/different? Website design and access issues – bit rushed Tool content and design – took longer than expected, not sure if we had a shared mental model of final output (but a Pilot) Impacted slightly on testing period – knocked some out of sync. (e.g. Pharmacy) Quality of design could be much better Different interpretations of ‘human factors’ science and role in project/and also of evaluation expectations Test numbers plucked out of thin air – over-confidence and naivety? Email bombardment irritated a fair few GPs wearing different hats! Thematic analysis of enhancedSEA reports only just started.

27. Educational Solutions for Workforce Development Multidisciplinary Project Learning and Outputs Once evaluation feedback incorporated, enhancedSEA method ready for roll out – good interest from Educational Leaders/SPSP/Others Conference Presentations Basic e-learning (BMJ Learning, NES website) Book Chapters (NES book and Good Practice GP Training Guide) Journal Submissions (conceptual framework, evaluation…) NES team learning (human factors science, psychological safety, logic models, error theory…) Organisational impacts (e.g. inter-professional working, good publicity)

28. Educational Solutions for Workforce Development Multidisciplinary Positive External Interest GP Specialty Training/Dental/Pharmacy/ RCGP Vocational Training for GPNs/PMs SPSP-PC and Scottish Government Department of Health in England/PS Toolkit Enquiries Some NHS Boards – as part of local enhanced services

29. Educational Solutions for Workforce Development Multidisciplinary Thank You – Any Questions? www.nes.scot.nhs.uk/shine/ P Bowie & C de Wet [eds.] ‘Safety and Improvement in Primary Care: The Essential Guide’ Radcliffe Publishing Ltd.