Building and Measuring Safety Culture: Getting Started & Planning Your Survey Process August 21 & 23, 2012

Polling Question Has your organization conducted a safety culture survey in the past 12 months? Yes No Do not know

Polling Question If yes, what survey instrument was used? Safety Attitude Questionnaire (SAQ) Hospital Survey on Patient Safety (HSOPS) Other survey or measurement tool Do not know

Learning Objectives Define safety culture Describe why safety culture is important for improvement efforts Identify best practices for measuring safety culture Define the role and responsibilities of the survey coordinator Identify characteristics of successful surveys and action items

A Brief overview of Safety Culture Part I A Brief overview of Safety Culture

The Armstrong Institute Model to Improve Care http://www.hopkinsmedicine.org/armstrong_institute

Culture provides the context for team success What is Safety Culture? Perceived priority of safety relative to other goals Culture is the compass team members use to guide their behaviors, attitudes, & perceptions on the job What will I get praised for? What will I get reprimanded for? What is the “right” thing to do? Culture provides the context for team success Learned, shared, tacit assumptions among members of a meaningful social group Lens through which “reality” is viewed Colors perceptions of what is and what should be Continuously evolving --Balance positive and negative…highlight “right” Image source: Marysia Tomaszewska, August 8, 2012, used under a Creative Commons License

What Are the Core Aspects of Safety Culture… -The notion of “bad” culture or “good” culture is somewhat of a myth. -As a multidimensional construct, there are several different things that contribute to culture…therefore, it is not that the entire culture is necessarily bad or good, rather that particular components may be areas of strength or may have room for improvement -When going through culture survey results then it is important to identify dimensions that are strong as well as those that need improvement in order to identify and leverage strengths to improve areas that need work. Schein, 2007

Why Safety Culture Matters Safety culture is related to outcomes Patient outcomes Patient care experience Infection rates, sepsis Postop. hemorrhage, respiratory failure, accidental puncture/laceration Treatment errors Clinician outcomes Incident reporting, burnout, turnover Huang et al., 2010; Mardon et al., 2010; MacDavitt et al., 2007; Singer et al., 2009; Sorra et al., 2012; Weaver, 2011.

Why Safety Culture Matters Safety culture influences the effectiveness of other safety and quality interventions Can enhance or inhibit effects of other interventions Safety culture can change through intervention Best evidence so far for culture interventions that use multiple components Haynes et al., 2011; Morello et al., 2012; Van Nord et al., 2010; Weaver et al., in press

Measuring Safety Culture Part II Measuring Safety Culture

CUSP & Safety Culture Safety Culture is typically measured “Pre-CUSP” (before interventions begin): Provides a baseline to diagnose barriers and facilitators that can impact improvement efforts Then can be measured 12-18 months following start of improvement efforts Use reliable and valid survey instrument Hospital Survey on Patient Safety (HSOPS) CUSP is the intervention that you will use to help you improve culture results

What is the Hospital Survey on Patient Safety (HSOPS)? Part of a suite of survey tools (SOPS) for hospitals, medical offices, nursing homes Sponsored by: Agency for Healthcare Research & Quality Developed by Westat, public release in 2004 The HSOPS App: An online survey tool Developed by the Armstrong Institute, in partnership with CeCity Allows participants to complete the survey online and survey coordinators to access detailed reports of results Participants are asked to choose 1 to 5 for each question: 1 Strongly Disagree 2 Disagree 3 Neither Agree nor Disagree 4 Agree 5Strongly Agree 1 Never 2 Rarely 3 Sometimes 4 Most of the time 5 Always

HSOPS Questions & Composite Scores (“Dimensions”) No. of Questions Example Question 1. Supervisor/manager expectations & actions promoting patient safety 4 B1. My supervisor/manager seriously considers staff suggestions for improving patient safety. 2. Organizational learning-continuous improvement 3 A9. Mistakes have led to positive changes here 3. Teamwork within unit A1. People support one another in this unit. 4. Communication openness C4. Staff feel free to question the decisions or actions of those with more authority. 5. Feedback & communication about error C1. We are given feedback about changes put into place based on event reports. 6. Nonpunitive response to error A8. Staff feel like their mistakes are held against them. (negatively worded) Staffing A2. We have enough staff to handle the workload. Hospital management support for patient safety F8. The actions of hospital management show that patient safety is a top priority. Teamwork across hospital units F4.There is good cooperation among hospital units that need to work together. Hospital handoffs & transitions F5.Important patient care information is often lost during shift changes. (negatively worded)

HSOPS Questions & Composite Scores (cont’d) 4 Outcome Variables No. of Questions Example Question 1. Overall perceptions of safety 4 A15. Patient safety is never sacrificed to get more work done. 2. Frequency of event reporting 3 D1. When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? 3. Patient safety grade (of hospital unit) 1 E1. Please give your work area/unit in this hospital an overall grade on patient safety. 4. Number of events reported in the last 12 months G1. In the past 12 months, how many event reports have you filled out and submitted? 6 Background questions

Scoring guidelines created by AHRQ HSOPS Scoring Scoring guidelines created by AHRQ Scores represent the % of positive responses % who gave a score of 4 or 5

Overview of the Survey Administration Process Part III Overview of the Survey Administration Process

Effective surveys… Are confidential and anonymous No way to link responses with email address or other individual information Results reported at unit-level Critical that the survey coordinator, project team, and leadership focus on survey as a tool for learning and improvement Have a clear reason and motivation to complete them Must be clear to staff why you are asking them to complete this survey, what will happen with results, and what will be done based on their input Are publically supported by both formal and informal unit leaders If leaders and colleagues view the survey as important and worthy of their time, frontline staff will be more motivated to contribute their input

Two Options for HSOPS Option 1 Option 2 Collect fresh HSOPS data from participating work areas Use online HSOPS App to survey work area team members Upload previously collected HSOPS data from participating work areas Upload excel file to online HSOPS App

What are the key tasks for planning our survey? Phase Task Prep (~3 wks) wk1 wk2 wk3 wk4 wk5 wk6 wk7 wk8 wk 9 Planning stage Logistical tasks Who will administer? Who will be surveyed? Publicize! Motivate participation. Create a debriefing plan Alert participants Provide pre-notification (post, email, meetings) Upload unit data Log-on and enter unit background info Start! Upload participant email addresses or previously collected data Track Monitor response rates (target >60%) Remind 1st reminder Remind again Final reminder End the survey Close the survey Collect data Download results report Execute Debrief & plan improvements

Who are key personnel in the HSOPS process?

Survey Coordinators: Key Responsibilities Help to coordinate the survey administration process, Learn how to use the online survey database (via training webinars and conference calls) Help survey participants with the survey process, Enter data about the work areas participating in the survey into the online survey database, Monitor the survey response rate using the online survey database, Work with hospital and work area leadership to distribute survey materials and information, Communicate with other HSOPS Coordinators in the project

Online HSOPS Application (“HSOPS App”)

Developing HSOPS Survey List Determine whom to survey Clinical staff only? Non-clinical support staff (e.g., environmental services)? Compile unit email list E-mail address Hospital area/unit Staffing category (e.g., physician, nurse, technician, administrative support) Make sure your list is up to date, check for: Staff on administrative or extended sick leave, Staff who appear in more than one staffing category or hospital area/unit, Staff who have moved to another hospital area/unit, Staff who no longer work at the hospital, and Other changes that may affect the accuracy of your list email addresses

Pre-Notification Materials to Inform Unit Clinicians and Staff of HSOPS Why the hospital is conducting the survey and how staff responses will be used, Which hospital staff were selected to be surveyed (e.g., all staff, nursing staff, all clinical staff, a random sample of staff, etc.), How much time is needed to complete the survey, Confidentiality or anonymity assurances, Suggested reply timeframe and how to return completed surveys, Incentives for which units will be eligible, if they respond (Optional), and Contact information for the main hospital point-of-contact. Sorra & Nieva, 2004

Why should we aim for a response rate of 60% or higher? Number of respondents who completed the online survey divided by the total number of unit members who were invited to complete the survey Safety culture reflects the shared perceptions among unit members Response rate of 60% is minimum necessary for a relatively representative sample Important for validity and reliability of results

Thinking ahead: Creating a Debriefing Plan for your Survey Results Debriefing is… A semi-structured conversation among frontline clinicians and staff that is usually led by a designated facilitator Purpose… Encourage open communication, transparency, and interactive discussion about the survey results across all levels To engage clinicians and staff in generating and implementing their ideas about how to create an effective safety culture in their work area

Some points to cover in your debriefing plan

We are here to support & guide: HSOPS Calls Getting Started & Planning the Process Intro to measurement, basics of survey administration The online HSOPS process A more detailed, technical call for survey coordinators that will walk through the details of how to use the online survey app Interpreting HSOPS data and debriefing Once data are collected and survey coordinators have download their results, another call will be held to walk through how to read reports, debriefing staff ,and interpret results

What if we have other surveys going on? It is common for hospitals to use several different surveys to elicit input from clinicians and staff Efforts to measure safety culture can be aligned with other surveys E.g., JHM measures both safety culture and employee engagement annually Some tips: Try to not have more than 1 survey going on at the same time if possible (Lack of time and survey fatigue can negatively impact results) Consider similar trends observed across surveys You can debrief results of similar surveys at the same time if similar trends or common themes are observed in results

Some Additional Tips for Effective Surveys Communicate to all staff on the unit the importance of their valuable input Check response rates periodically and send reminders at different times throughout the survey administration process: before the survey is administered and a few times during the survey administration process Send periodic reminders through various communication channels, such as emails, announcements, during staff meetings, etc. Create a debriefing plan for sharing results with clinicians and staff Don’t be afraid to reach out for assistance when in doubt

Action Items: Identify a survey coordinator for participating work areas …Or coordinators if your team desires more than one coordinator Send collated list of survey coordinators to your coordinating entity (HEN, Hospital association, or affinity group leader) If collecting new data or uploading previously collected data Coordinating entity to compile and send to JHU Determine whom to survey and sample size …Or locate previously collected data file If collecting fresh HSOPS data.: Compile unit email list Create informational materials to publicize the survey (posters, flyers, newsletters, meeting agenda items) Send pre-notification letter, post pre-notification information, or hold pre-notification meeting Create draft debriefing plan

Questions? Reminder… You can access all slides, call recordings, and project tools at the following website: http://www.hopkinsmedicine.org/quality_safety_research_group/our_projects/action_II/SUSP/index.html

On-boarding Call Evaluation We want to ensure that the on-boarding calls provide useful and pertinent information for the SUSP teams. For this reason we request that you complete a brief evaluation following each call. The evaluation may be found at the following link: https://www.research.net/s/Onboarding_Evaluation

References Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Dziekan G, Herbosa T, Kibatala PL, Lapitan MC, Merry AF, Reznick RK, Taylor B, Vats A, Gawande AA; Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011 Jan;20(1):102-7. Huang DT, Clermont G, Kong L, Weissfeld LA, Sexton JB, Rowan KM, Angus DC. Intensive care unit safety culture and outcomes: a US multicenter study. Int J Qual Health Care. 2010 Jun;22(3):151-61. MacDavitt K, Chou SS, Stone PW. Organizational climate and health care outcomes. Jt Comm J Qual Patient Saf. 2007 Nov;33(11 Suppl):45-56. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010 Dec;6(4):226-32. Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf. 2012 Jul 31. [Epub ahead of print] Schein E. Organizational culture and leadership, 4th edition. San Francisco, CA: Jossey-Bass. 2010.

References Singer SJ, Falwell A, Gaba DM, Meterko M, Rosen A, Hartmann CW, Baker L. Identifying organizational cultures that promote patient safety. Health Care Manage Rev. 2009 Oct-Dec;34(4):300-11. Sorra J, Khanna K, Dyer N, Mardon R, Famolaro T. Exploring Relationships Between Patient Safety Culture and Patients' Assessments of Hospital Care. J Patient Saf. 2012 Jul 10. [Epub ahead of print]. Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. (Prepared by Westat, under Contract No. 290-96-0004). AHRQ Publication No. 04-0041. Rockville, MD: Agency for Healthcare Research and Quality. September 2004. van Noord I, de Bruijne MC, Twisk JW. The relationship between patient safety culture and the implementation of organizational patient safety defences at emergency departments.. Int J Qual Health Care. 2010 Jun;22(3):162-9. Weaver SJ. A configural approach to patient safety climate: The relationship between climate profile characteristics and patient safety. Doctoral dissertation. University of Central Florida. 2011. Weaver, S. J., Dy, S., Lubomski, L., & Wilson, R. Promoting a culture of safety. In R.M. Watcher, P.G. Shekelle, P. Pronovost (Eds.). Making healthcare safer: A critical analysis of the evidence of patient safety practices (AHRQ report # TBD). Rockville, MD. In press.