Measuring perceptions of safety climate in primary care Carl de Wet Mark Russell Paul Bowie

Session aims Measuring safety climate (5 min) Safety climate vs. safety culture The rationale for SafeQuest 2. Practical application (10 min) Benefits of measuring Cautions 3. Interpreting the safety climate report (25 min) Perceptions within the practice team Perceptions of different professional groups Perceptions of different practice teams

What is safety climate? ‘Culture’ and ‘climate’ are often used interchangeably. ‘…The measurable features of safety culture…’ Factors (domains) are specific characteristics of climate Leadership Communication Workload Safety Systems Team work It is the shared perceptions of safety policies, procedures and practices held by a group. (Flin et al, 2006)

Measuring safety climate Various instruments: Qualitative (dimensional) or Quantitative (typological) USA, industry, secondary care BUT … few for Scottish primary care Specifically developed for intended users, geography and organisation Undergo psychometric testing Relevant factors: communication, team work

SafeQuest: Development and testing Written feedback (78) and interviews (46) with various staff groups. Endorsed by UK patient safety ‘expert’ group. Content validity index (CVI) 0.94. Psychometrically tested to a gold standard - 49 practices, 563 team members High validity and reliability of 30 items and 5 factors

The survey measures perceptions – not reality The survey measures perceptions – not reality. A ‘positive’ score does not necessarily mean that things are safe – only that staff thinks it is safe!

‘How bad is your pain from 1 to 10?’

Cautions Perceptions vs. reality High or low scores vs. ‘right’ and ‘wrong’ vs. ‘strong’ or ‘weak’ Numerical meaning of scores Snapshot Participation

Benefits of measuring safety climate Raise awareness ‘Diagnose’ your team’s safety climate Identify strengths and weaknesses by comparison to other teams Starting point for reflection and change – design and implement initiatives to build a strong, positive culture Evaluate – serial measures Encourage teamwork, participation and inclusion Organisational benefits High Reliability organisations (oil / aviation industries) Improved safety outcomes Improved safety behaviour Health care Emerging evidence of an association between safety climate and clinical outcomes in secondary care (but not yet primary care): shorter hospital stays, fewer medication errors, reduced rates of ventilator associated pneumonia, fewer patient falls, reduced bloodstream infection rates, increased adoption of safe work practices.

Section 1. Team perceptions Practice Y Practice X What factor is most positive? Which factor is least positive? What is the difference between most and least positive and the overall safety climate? (relativeness) What proof is there for these perceptions?

Section 1. Comparison with other practices Which factors are more or less positive than other practices? Are there any evidence for this? If all factors are more or less positive – what is the implication?

Section 2. Comparison by professional roles Practice X Practice Y Practice X Practice Y Are there any differences between the two groups? (The size of variation is more important than what group is right!) Are there any similarities? What does that mean? (Increases the reliability of the finding) Does any of the findings change when you now consider clinical vs non-clinical? Is there additional variation? How does this fit with the first section? Practice x = about 12-14% variation vs. Practice Y = about 3-4%

Section 3. Specific items Practice X Does any item ‘stand out’? Think about relations between items – if staff levels are really insufficient, how can performance NOT be impaired? An example of where perception may not reflect reality.

Practice X A statistical difference may not be a practical difference - Although the safety climate domains/factors may not have reflected a communication problem ?? Does this need to be addressed?

Practice X

Participation Practice x = 25 (100%) Practice Y = 10 (?%)

Practice X: Some findings and suggestions for change Excellent participation Low perceptions of ‘workload’ Specific concerns regarding staff levels and time Overall perceptions lower than other practice teams Substantial variation between staff groups Staff would like to be informed and engaged more often Audit current workload to compare ‘evidence’ with perception Agree an acceptable, internal communication forum to update staff Circulate agenda of meetings beforehand Consider inviting and giving opportunity for all staff groups to attend meetings. Increase interaction between clinical and non-clinical staff – breaks, recreational area, team-building events

Practice Y Although not statistically significant – could it have practical significance – this is the only question scored LOWER than average. Even though leadership is highest domain – is there an unresolved issue here?

Practice Y: Some findings and suggestions for change Participation questionable Perceptions are overall positive Little variation between staff groups ?Discrepancy Promote the climate survey more widely and consider a further measure. Find evidence to corroborate perceptions Formulate a specific plan to sustain positive culture Identify strengths that can be build on and strategy to disseminate to others Consider further reflection or investigation of practice management of ‘problem team members’.

Take home points Remember the cautions The results are yours - only you and your team can make sense of it. Statistical significance vs. practical significance Consider what evidence (if any) there are for the reported perceptions Involve as many team members as possible Keep an open mind !!

Safety Culture How could you use the tools you have seen and learnt about over the past two days to develop your practice culture? Identify a test of change

Health Foundation Safety Improvement in Primary Care SIPC