1. Measuring perceptions of safety climate in primary care
Carl de Wet
Mark Russell
Paul Bowie

2. Session aims Measuring safety climate (5 min)
Safety climate vs. safety culture
The rationale for SafeQuest
2. Practical application (10 min)
Benefits of measuring
Cautions
3. Interpreting the safety climate report (25 min)
Perceptions within the practice team
Perceptions of different professional groups
Perceptions of different practice teams

3. What is safety climate?
‘Culture’ and ‘climate’ are often used interchangeably.
‘…The measurable features of safety culture…’
Factors (domains) are specific characteristics of climate
Leadership
Communication
Workload
Safety Systems
Team work
It is the shared perceptions of safety policies, procedures and practices held by a group. (Flin et al, 2006)

4. Measuring safety climate
Various instruments:
Qualitative (dimensional) or Quantitative (typological)
USA, industry, secondary care
BUT … few for Scottish primary care
Specifically developed for intended users, geography and organisation
Undergo psychometric testing
Relevant factors: communication, team work

5. SafeQuest: Development and testing
Written feedback (78) and interviews (46) with various staff groups.
Endorsed by UK patient safety ‘expert’ group. Content validity index (CVI) 0.94.
Psychometrically tested to a gold standard - 49 practices, 563 team members
High validity and reliability of 30 items and 5 factors

6. The survey measures perceptions – not reality
The survey measures perceptions – not reality. A ‘positive’ score does not necessarily mean that things are safe – only that staff thinks it is safe!

8. ‘How bad is your pain from 1 to 10?’

11. Cautions Perceptions vs. reality
High or low scores vs. ‘right’ and ‘wrong’ vs. ‘strong’ or ‘weak’
Numerical meaning of scores
Snapshot
Participation

12. Benefits of measuring safety climate
Raise awareness
‘Diagnose’ your team’s safety climate
Identify strengths and weaknesses by comparison to other teams
Starting point for reflection and change – design and implement initiatives to build a strong, positive culture
Evaluate – serial measures
Encourage teamwork, participation and inclusion
Organisational benefits
High Reliability organisations (oil / aviation industries)
Improved safety outcomes
Improved safety behaviour
Health care
Emerging evidence of an association between safety climate and clinical outcomes in secondary care (but not yet primary care):
shorter hospital stays,
fewer medication errors,
reduced rates of ventilator associated pneumonia,
fewer patient falls,
reduced bloodstream infection rates,
increased adoption of safe work practices.

13. Section 1. Team perceptions
Practice Y
Practice X
What factor is most positive?
Which factor is least positive?
What is the difference between most and least positive and the overall safety climate? (relativeness)
What proof is there for these perceptions?

14. Section 1. Comparison with other practices
Which factors are more or less positive than other practices?
Are there any evidence for this?
If all factors are more or less positive – what is the implication?

15. Section 2. Comparison by professional roles
Practice X
Practice Y
Practice X
Practice Y
Are there any differences between the two groups? (The size of variation is more important than what group is right!)
Are there any similarities? What does that mean? (Increases the reliability of the finding)
Does any of the findings change when you now consider clinical vs non-clinical? Is there additional variation?
How does this fit with the first section?
Practice x = about 12-14% variation vs. Practice Y = about 3-4%

16. Section 3. Specific items
Practice X
Does any item ‘stand out’?
Think about relations between items – if staff levels are really insufficient, how can performance NOT be impaired? An example of where perception may not reflect reality.

17. Practice X
A statistical difference may not be a practical difference - Although the safety climate domains/factors may not have reflected a communication problem ?? Does this need to be addressed?

18. Practice X

19. Participation
Practice x = 25 (100%)
Practice Y = 10 (?%)

20. Practice X: Some findings and suggestions for change
Excellent participation
Low perceptions of ‘workload’
Specific concerns regarding staff levels and time
Overall perceptions lower than other practice teams
Substantial variation between staff groups
Staff would like to be informed and engaged more often
Audit current workload to compare ‘evidence’ with perception
Agree an acceptable, internal communication forum to update staff
Circulate agenda of meetings beforehand
Consider inviting and giving opportunity for all staff groups to attend meetings.
Increase interaction between clinical and non-clinical staff – breaks, recreational area, team-building events

21. Practice Y
Although not statistically significant – could it have practical significance – this is the only question scored LOWER than average. Even though leadership is highest domain – is there an unresolved issue here?

22. Practice Y: Some findings and suggestions for change
Participation questionable
Perceptions are overall positive
Little variation between staff groups
?Discrepancy
Promote the climate survey more widely and consider a further measure.
Find evidence to corroborate perceptions
Formulate a specific plan to sustain positive culture
Identify strengths that can be build on and strategy to disseminate to others
Consider further reflection or investigation of practice management of ‘problem team members’.

23. Take home points Remember the cautions
The results are yours - only you and your team can make sense of it.
Statistical significance vs. practical significance
Consider what evidence (if any) there are for the reported perceptions
Involve as many team members as possible
Keep an open mind !!

24. Safety Culture
How could you use the tools you have seen and learnt about over the past two days to develop your practice culture?
Identify a test of change

25. Health Foundation Safety Improvement in Primary Care SIPC