Improvement 101 Learning Series Module 3: Measurement and Using Data for Improvement Your Host: Leanne Couves Improvement Advisor Hello and welcome back to the BCPSQC webinar series on improvement topics.  Thanks for listening.  My name is Leanne Couves and I will be your host today.  I am a Faculty member for Quality Academy and work on a number of improvement projects in the healthcare system.   This series is specifically designed for patient partners and volunteers who work within BC’s healthcare system to help improve care.   Authentic engagement with patient partners is not something that happens by accident. It requires planning and well-intentioned, principled actions.  A necessary shift is taking place in our health care system – inviting patients into improvement work is not something that is nice to do or a privilege to patients, but rather something that needs to be done whenever decisions that impact patients are being made. Patient Partners are recognized as part of the care team and increasingly, an important part of our improvement teams.   We hope that these learning modules will provide a deeper understanding of some of the language, methods, processes, tools and techniques that healthcare teams are using to make system improvements. There are a total of 4 modules in the series.  Each module tackles a specific improvement topic.  The first five minutes of the recording will provide an overview of the topic.  For some, this content may be enough to get you started and answer your initial questions.   For those who want to go deeper, the next 15 minutes will provide more details, information and resources. 

This module focuses on measurement and using data for improvement This module focuses on measurement and using data for improvement. It emphasizes the importance of collecting data over time and covers how to choose measures for a project and develop a plan to collect data.

What Are Data? A first question might be “What are data anyway?” We use data all the time in our daily life and often in our improvement projects.

WHAT Are Data? Often we think of data as being NUMBERS. Like the weight we see on a bathroom scale, the price we pay for groceries each week or the number of strokes in a friendly game of golf.

WHAT Are Data? Numbered data can be Counted or Measured Numbers could be things we COUNT, like number of patients that arrived at an Emergency Department each day or number of errors that caused harm to a patient. Other things we MEAURE. In our daily lives, we might measure daily temperatures or the amount of rain that fell last week. In healthcare, we might measure the time that a patient might wait in an Emergency Department.

WHAT Are Data? Numbered data can be Counted or Measured These numbers are often presented visually in a graph. For improvement projects, that data is often displayed in graphs over time, such as waiting time in Emergency, plotted every day for the last 3 months.

Data can also be in the form of FEEDBACK, presented in words, stories, pictures or observations. These data are a rich source of information for our improvement work. The source of these data may be surveys, interviews, audits, focus groups, video recordings, or just asking people what they think or feel. Whether data comes in the form of numbers, words or observations, a common feature is that data are RECORDED and used for learning and improvement.

WHY Measure? Where are we now? So, why do teams measure? Remember that improvement is the process of making things better. Measurement and the collecting of data can help improvement teams make things better in a number of ways: First, data can tell an improvement team where things are at before an improvement project begins. What are patients saying about their care? How long are people waiting in the Emergency Department now? Teams can then compare what is called BASELINE data with what patients’ expect, what external standards (like accreditation) are out there or what we would ideally like to see. These initial assessments can also give some ideas about potential goals an improvement team might want to set.

WHY Measure? Where are we now? Are the changes we are trying working or not? As we mentioned in the previous sessions, “all improvements require a change but not all changes lead to improvement”. How will teams know that the changes they are trying actually makes things better? Data can help teams understand if things are getting better or worse and by how much. Data provides a basis for action and helps teams learn more about their changes. Do they keep doing what they are doing? Do they modify the changes? Do they abandon these ideas and perhaps try something else.

WHY Measure? Where are we now? Are the changes we are trying working or not? Have we have reached our goal? Finally, data can tell us when they have reached their goals and if those goals are maintained.

WHY Measure? “What counts to be counted?” Where are we now? Are the changes we are trying working or not? Have we have reached our goal? “What counts to be counted?” “Our theories determine what we measure” An unappreciated yet beneficial effect of measuring and collecting data is that it prompts important conversations about what really matters and what counts to be counted. Once we have those conversations and an agreement on what will be measured, the collected data can facilitate learning and communication amongst team members that has meaning for all.

Types of Measures Outcome Measures: Process Measures: Show if changes are leading to improvement and achieving the overall aim of the improvement initiative Process Measures: Show whether a specific change is having its intended effect Balancing Measures: Help ensure that changes to improve one part of the system are not causing new problems in other areas Improvement teams find it useful to several measures to get insight from a variety of perspectives. Improvement initiatives often have three types of measures: • Outcome measures help the team understand if goals are being reached. For example, are Emergency Wait Times getting better? How is our patient satisfaction with care improving?

Types of Measures Outcome Measures: Process Measures: Show if changes are leading to improvement and achieving the overall aim of the improvement initiative Process Measures: Show whether a specific change is having its intended effect Balancing Measures: Help ensure that changes to improve one part of the system are not causing new problems in other areas • Process measures tell the team more about the changes being tried. For example, when a new form or checklist is introduced, are staff completing it? Is it being filled in accurately? We can track data like % of forms completed.

Types of Measures Outcome Measures: Process Measures: Show if changes are leading to improvement and achieving the overall aim of the improvement initiative Process Measures: Show whether a specific change is having its intended effect Balancing Measures: Help ensure that changes to improve one part of the system are not causing new problems in other areas • Balancing measures tell us if there is anything else that has changed, or is different, because of the changes introduced. Have their been negative impacts in other parts of the system? Some teams measure staff satisfaction, workload, or money. As a guideline, 3-8 measures are usually sufficient, including at least one outcome measure and one process measure.

Data Collection Tips Start right away Small and frequent samples Integrate into workload Timely Plot data over time Sometimes, improvement teams get overwhelmed with what we call the “data burden”. But there are ways to reduce the effort and still get useful data! As soon as an aim or goal is set, the team can start defining what measures might be needed and how to collect it. Some teams go back in time and gather some data from previous months or years. Teams don’t need to measure everything and don’t need to measure every patient, record, interaction to learn. A colleague once said “We don’t need to be hit by every raindrop to know that we need to get an umbrella”. Let’s say we have 100 new admissions a month. Surveying all 100 patients requires a lot of resources. Surveying only 1 patient might not offer enough diversity of perspective. Improvement teams need to decide how to balance the need for data with the resources to collect and interpret that data.  The team might survey about 10 patients a month, a smaller sample. Teams will try to make data collection as easy as possible. Sometimes they can can get useful data in computer systems. Other times, they may need to add new processes. Teams will also try to collect data in a timely way. Data that is months or years out of data will be less useful. Dr. Donald Berwick once said “If you only do one thing as a result of this improvement stuff, plot the dots. Use time as your teacher. Changes happen over time. Plot your data over time to see trends, patterns and effects”.

Thank you! PatientVoicesBC.ca @PatientVoicesBC Thank you for listening to the first 5 minutes of Module 3. It hopefully provided “just enough” information to answer your initial questions about Measurement and Using Data for Improvement . You are welcome to stop the recording now and enjoy the rest of your day. However, if you want to dig deeper, I welcome you to continue with the recording and learn more about Measurement and Using Data for Improvement .