Quality Improvement and the Model for Improvement Hunter Gatewood, Improvement Advisor MFI is an organized approach to getting better results when you are working with other people. Reference for all Model for Improvement slides: The Improvement Guide (Langley, Nolan, Nolan, Norman, and Provost, Jossey-Bass, 1996). Most of the content of this presentation of “Quality Improvement and the Model for Improvement” comes from a day-long course “The ABCs of Quality Improvement” which was co-authored by the Institute for Healthcare Improvement and the California HealthCare Foundation in 2007. Slides on “Understanding Variation” in 30s of slide deck adapted from NHS Scotland Tutorial Guide on Statistical Process Control. Learning Session 1 DSRIP RHP10 Learning Collaborative on Behavioral Health – Primary Care Integration January 29, 2014

What is Quality Improvement? Before showing this slide, ask participants how they define quality improvement. Build off their comments. System level: not personal “be better” but system change that makes quality the default, decreases chance of error, dropped balls, etc.

What is Quality Improvement? Change at system level Work at the frontline level By multi-disciplinary staff teams Regular, ongoing assessment and measurement Reduction of variability Process focus, not individual as good/bad Examples Establish reminder system to reduce no shows Develop system to identify patients needing LDL test How do we do things better? QI is an organized approach to getting better results when working with other people, when it’s more than just you.

What is Quality Assurance? “The planned and systematic activities put in place to ensure that (quality) requirements for a product or service will be fulfilled.” Ensure that requirements, guidelines, regulations met Uses inspection (vs. regular measurement) Also known as quality control Examples: Health code for restaurants Licensing of providers Audits of clinical facilities Are we (you) doing things right? QA more regulatory in nature. The philosophy is grounded more in inspection and making sure services or products meet standards, as opposed to QI, which continuously tries to improve the status quo through regular measurement and change at the system level. QA may be static, vs QI which is rooted by it’s nature in continuous change

change is an improvement? Model for Improvement What are we trying to AIM accomplish? How will we know that a MEASURES change is an improvement? What changes can we make that CHANGES will result in improvement? Act Plan Study Do RAPID TEST OF CHANGES Today, we’re going to talk about the philosophy and key concepts to quality improvement, starting with the “Model for Improvement.” The Model for Improvement was first developed by Tom Nolan and colleagues, and is described in detail in The Improvement Guide (Langley, Nolan, Nolan, Norman, and Provost, Jossey-Bass, 1996). The Model for Improvement is based on the idea that every system is designed to give the results that it gets. For different results, you have to change the system. The three questions and the Plan, Do, Study, Act Cycle give you a method to learn how to make the changes that will result in improvement. From Associates in Process Improvement.

Model for Improvement, Part 1 Aim Statements Monthly Measures and Run charts but first, 3 more distinctions

Learning Collaboratives, DSRIP projects, and the Model for Improvement Model for Improvement is a powerful structure to use with any improvement goal. It’s central to a Learning Collaborative, but you can use it outside of a Learning Collaborative. In a Learning Collaborative, all teams focus on the same improvement topic: Care Transitions. So Aim Statements and measures are focused on improvement in Care Transitions. DSRIP projects cover a huge and impressive range of topics, many are related to Care Transitions, and many are not. Many DSRIP projects’ progress are crucial to powerful sharing with each other to drive better care in care transitions. And others you will want to keep separate from LC work, because it’s not relevant and you don’t want to do any more reporting and sharing work than is useful to you and to your R10 colleagues. Diagram credit: hhs.gov, from original at www.ihi.org

Three DIFFERENT Uses of Numbers Research Judgment Improvement Research: hypothesis, measure, analyze, report, done. Judgment: audit, inspect, maintain standards, often for public good Improvement: we fix our work ourselves, with our ideas and our approaches. Question: In DSRIP measurement and numbers reporting, which use of data is that? Judgment, because it’s overseen by state and federal regulators and there’s money at stake. Money given or withheld, it’s always about judgment. AND you use Improvement-type measurement and activity to get the results to get the money.

Other Improvement Models Six Sigma LEAN (Toyota Production System) TQM CQI The rest of the slides on Six Sigma and LEAN are optional based on the audience background and desire for participants to have overview of these models.

change is an improvement? Model for Improvement What are we trying to AIM accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do You’re all familiar with the Model for Improvement now. We are now going to break down each of the three questions for the step-by-step processes to answer them. And, here it is again – the Model for Improvement. From Associates in Process Improvement.

Characteristics of Good Aim Focused: Sets a clear goal to focus the team Measurable: can develop clear measures to track progress toward aim Defines success numerically Time specific: establishes time frame (6-12 months) Clinically relevant, compelling Defines patient population Adapted from the Institute for Healthcare Improvement Breakthrough Series College.

Example Aims Within 12 months, decrease the percent of patients with >3 ER visits a year. Within six months, reduce waiting time for primary care appointments in SCC Clinic to Third-next Available Appointment Delay of 3 days or less. By December 2012, decrease the percentage of diabetic patients with HbA1c > 9 to ten percent. By {When}, increase/decrease {What} for {Whom}

Assess the Aim: Example 1 The QI team will meet five times in the next month Focused: Sets a clear goal to focus the team? Measurable and defines success numerically? Time specific? Clinically relevant? Defines patient population? Measurable-meeting /5 times time specific-next month(May 2012 would be better) clinically relevant-? Pt pop-QI team

Assess the Aim: Example 2 By December 2014, increase by 50% the percentage of patients with diabetes who got an HbA1c test in the past 12 months AND have HbA1c < 9%. Focused: Sets a clear goal to focus the team? Measurable and defines success numerically? Time specific? Clinically relevant? Defines patient population? Wordy sounds good. Measure- percentage of patients who have had a HBA1c test/increase by 50% time-Dec 2008 Clinically relevant? 95% then no. 10% then yes pop-DM patients reword to actual percentage

change is an improvement? Model for Improvement What are we trying to accomplish? How will we know that a MEASURES change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do You’re all familiar with the Model for Improvement now. Breaking down each of the three questions. Working on #2 You’re all familiar with the Model for Improvement now. We are now going to break down each of the three questions for the step-by-step processes to answer them. Change is not always an improvement We are going to give you the tools so that as you make changes you can figure out if they move you closer to your aim, no effect, or away from it We will do this by looking at our measures

Measurement for Improvement Purpose: To track progress (improvement!) over time and to promote buy-in. Audience: QI Team, Front-line staff and providers, Senior sponsors Measure things that have meaning Audience: anyone interested or effected by the process of achieving the aim 16

RHP10 shared BH-PC Integration measures Pre-work, Appendix D % patients screened with cross-specialty tool Ex. PHQ for depression in primary care, or diabetes screening on behavioral health 2. % patients receiving integrated care intervention in past 12 months Ex. shared care plan at both BH and PC sites 3. % patients receiving integrated care whose condition improved (tracked cross-specialty) Ex. PHQ #s from + to – in primary care, or HbA1c down and tracked/reported in BH 17

Characteristics of a Good Measure Directly relates to aim Specifies patient population Data are available Able to collect data frequently Worth measuring for at least 12 months Data are available- important use what you have, as you develop a registry or EMR you may collect different data Collect frequently-saw this with PDSA cycles. What we are showing you today is a different tool to use for quality improvement 18

Example Measures Number of days until third next available appointment. Aim: Within six months, reduce waiting time for primary care appointment to 3 days. Percentage of patients with HbA1c >9. Aim: By December 2012, decrease the percentage of diabetic patients with HbA1c > 9 to 10% or less. As part of measure development, also think about sources of the data that will need to be collected. Establishing measures goes hand-in-hand with starting to think about data sources, and how to collect and use the data. A little bit of a review What are you actually measuring? Sample measures from audience. 19

Assess the Measure: Example 1 Number of times the QI team meets each month Directly relates to aim? Specifies patient population? Data are available? Able to collect data frequently? Aim: The QI Team will meet 5 times in the next month In reality…..Not a good measure because it is not clinically relevant – there is no patient population and the number of meetings held does not correlate to process improvement. The QI Team will meet 5x in the next month. The aim is the problem. 20

Assess the Measure: Example 2 Number of improvements to the care of patients with diabetes Directly relates to aim? Specifies patient population? Data are available? Able to collect data frequently? Aim: Improve the care of diabetic patients over the next 12 months “Improve the care of diabetic patients over the next 12 months” Data available on what/ what is an improvement? Need to define “improvement” Quote Foot care study 21

Assess the Measure: Example 3 Percentage of patients with HbA1c testing during past 12 months Directly relates to aim? Specifies patient population? Data are available? Able to collect data frequently? Aim: By December 2012, 90% of DM pts. will have had an A1c in the past 12 months. “By December 2008, increase by 50% the percentage of patients with diabetes who have had an HbA1c test in the past 12 months.” 22

Sample Run Chart Median

Run Charts Display ordered sequence of data and provide running record over time Can be used for any data that are sequenced over time (trending) Require no statistics Visually illustrate progress toward goal Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control. http://www.indicators.scot.nhs.uk/SPC/SPC.html

Understanding Variation All sets of data demonstrate variation. Two types of variation Random/common cause normal, no impt change, NOT special Special cause something going on Examples of variation in a set of numbers: how long it takes to get to work each day The volume of cereal in a box The size of apples The length of hairs in a buzz cut The height of kids in a 2nd grade class The height of waves at the beach The amount of coffee in a large Starbucks drink Adapted from, NHS Scotland Tutorial Guide on Statistical Process Control. http://www.indicators.scot.nhs.uk/SPC/SPC.html

Detecting Special-Cause Variation Four run chart rules Astronomical point Shift Trend Number of runs

Did the change result in improvement? 27

1. 2. 3. Change Made Change Made HG note: what conclusions can we draw about significance of The Change in these charts? First, need the median line drawn, then need to use run chart rules. 3. Change Made 28

Benefits of Run Charts Tells story visually Focuses on the process Prevents jumping to unfounded conclusions about what does/doesn’t work Motivates people to think like improvers

Team Meeting 1: Aim Statement and Measures Put slide of LC measures back up.

Sample Aim+Measures for BH-PC Integration By Dec 31, 2014, Super BehavioralHealth Clinic will improve the integration of behavioral health and primary care for our patients seen at Advanced Primary Care Clinic. We will improve: Increase % patients with diabetes with a diabetes self-mgt goal from 30% to 90% Increase % patients with a shared care plan at both SBHC and APHC from 0% to 80% Decrease % patients with diabetes with HbA1c over 9 from 20% to 10% 31

Model for Improvement, Part 2 Test changes small, using the Plan-Do-Study-Act Cycle What Changes Can We Make that Will Result in Improvement?

Model for Improvement Act Plan Study Do What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do Drilling down into your data will hopefully give clues for changes that actually do make an improvement. The MFI and PDSA cycle provide a common language for testing the effect of your changes. Emphasis here on methods that help you work on activities that will bring not just CHANGE, but CHANGE FOR THE BETTER. The AIM and initial change activities are selected based on past experience, “best guess”, borrowing best practices from others, etc. the MEASURE is our tool to see if initial and ongoing activities cause a change from the status quo. Now comes the third step; using tools and methods that tell us if we are going in the right direction (are things getting better, or are they staying the same, or actually getting worse?) We need a process to do this analysis, and to incorporate the analysis into planning for the next set of improvement activities. From Associated in Process Improvement

Learning Objectives Learn the value of testing changes in care on a small scale before implementing widely Learn how to turn your ideas into tests (PDSAs) Plan a PDSA test to complete when you return to work

Why do Small Tests of Change? Understand the likelihood that change will result in improvement Understand the extent and limitations of the change Learn to adapt the change to local environment Evaluate cost Address unexpected consequences Gain buy-in and minimize resistance if change is implemented and spread 1. Testing provides evidence that a change really does result in the improvement that was expected. Even though a change may sound like a good idea, you don’t know until you actually use it in practice. 2. There can be multiple changes needed in order to produce the desired effect on your system. 3. Even though a change may have produced the desired effect in a different setting, you don’t really know how it will work in your particular environment until you try it. 4. Change sometimes produces unintended consequences. Testing lets you to observe the costs (resources, time, equipment, etc.) that the new process might involve, as well as the side-effects that might come with the change. For example, providing same-day access for clinic patients may affect the process for locating medical records. 5. It is often easier for people to agree to try a new way of doing something if the change is presented as a short-term, small scale trial. “Let’s just try this for three days.” In this way, they don’t have to immediately abandon the old way of doing something. Testing often shows people that the new way is really better, and they are then more willing to embrace the new process. Adapted from the Institute for Healthcare Improvement Breakthrough Series College. 35 35

PDSA – Rapid Cycle Improvement Act Questions & predictions (why?) Plan to carry out the cycle Plan Study Do Carry out plan Document Begin data analysis Complete analysis Compare data to predictions Summarize what was learned Adapt Adopt Abandon Four parts of the cycle: Plan: Decide what change you will make, who will do it, and when it will be done. Formulate an hypothesis about what you think will happen when you try the change. What do you expect will happen? Identify data that you can collect (either quantitative or qualitative) that will allow you to evaluate the result of the test. Do: Carry out the change. Study: Make sure that you leave time for reflection about your test. Use the data and the experience of those carrying out the test to discuss what happened. Did you get the results you expected? If not, why not? Did anything unexpected happen during the test? Act: Given what you learned during the test, what will your next test be? Will you make refinements to the change? Abandon it? Keep the change and try it on a larger scale? Adapted from the Institute for Healthcare Improvement Breakthrough Series College.

PDSA STEP by STEP Identify a daily process need/problem Brainstorm possible solutions Choose one to test Write a PLAN for your test DO it! Conduct the test and document the data. STUDY the data to determine whether or not this test solved the problem Take ACTion-your most logical next step to solve the problem or retest for effectiveness

Example of PDSA Subject: Diabetes: Planned visits for blood sugar management. Plan: Ask one patient if he or she would like more information on how to manage his or her blood sugar. (Predict: Patient will say “yes”) Do: Dr. J. asked his first patient with diabetes on Tuesday. Study:  Patient was interested; Dr. J. was pleased at the positive response. Act: Dr. J. will continue with the next five patients and set up a planned visit for those who say yes. Source: ihi.org

Get one tennis ball for your group Pick team’s timekeeper Exercise Stand in groups of 8 Get one tennis ball for your group Pick team’s timekeeper Name your team, quickly Plan multiple cycles for a test of a change, and think a couple of cycles ahead Scale down size of test (# of patients, location) Test with volunteers – don’t start with uninterested or resistant staff Don’t try for buy-in, consensus for every small test Don’t necessarily need consensus for tests. Run a set of tests if people have more than one opinion; the team can learn more from a range of tests. Consensus only needed at implementation time, when we believe we have a lot of evidence for our theory of how the system works. Be innovative to make test feasible Collect useful data (qualitative or quantitative) during each test How will you use the results of test? If you aren’t going to use results, don’t waste your time doing the test Test over a wide range of conditions (days of week, staffing variables, etc.) 25

“Hey, Let’s go See Homer Blow Himself up.” DVD at Chapter 6 “Rocket Roulette”, from 34:37 to ~ 41:00 (6 ½ min) It is based on the true story of Homer Hickam, a coal miner's son who was inspired by the launch of Sputnik 1 to take up rocketry against his father's wishes, and eventually became a NASA scientist. Video: “October Sky” Scene 6, “Rocket Roulette” (from 34:37 to 41:00)

Repeated Uses of PDSA Cycle Hunches Theories Ideas Changes That Result in Improvement A P S D DATA Very Small Scale Test Follow-up Tests Wide-Scale Tests of Change Implementation of Change Testing or adapting a change idea – start with one patient or provider, tweak it, then spread to 2-3 providers Implementing a change Spreading the changes to the rest of your system Adapted from the Institute for Healthcare Improvement Breakthrough Series College.

Keys to Successful Small Tests of Change Scale down: do small tests Collect useful data during each test Test over a wide range of conditions Plan multiple cycles for a test of a change, and think a couple of cycles ahead Scale down size of test (# of patients, location) Test with volunteers – don’t start with uninterested or resistant staff Don’t try for buy-in, consensus for every small test Don’t necessarily need consensus for tests. Run a set of tests if people have more than one opinion; the team can learn more from a range of tests. Consensus only needed at implementation time, when we believe we have a lot of evidence for our theory of how the system works. Be innovative to make test feasible Collect useful data (qualitative or quantitative) during each test How will you use the results of test? If you aren’t going to use results, don’t waste your time doing the test Test over a wide range of conditions (days of week, staffing variables, etc.) Adapted from the Institute for Healthcare Improvement Breakthrough Series College. 25

Run PDSAs in Parallel Specific Test Cycles Implement registry Group Visits Workflow redesign Team care approach This diagram illustrates how our hypothetical team that is trying to achieve improved access actually carries out tests in several different categories of changes. In order to Improve timeliness of care for patients with diabetesAim: Improve % of diabetics getting HbA1C, lipid tests and foot/eye exams within recommended interval , the team probably cannot rely solely on implementing the registry. Changes in other areas such as developing group visits, workflow redesign, and instituting a team care approach should be considered. These multiple changes can be made simultaneously or can be folded in at planned intervals. We encourage teams to try “packages” of changes to achieve maximum results. At a later stage, specific changes can be removed to assess the impact of a particular change on the overall result. Adapted from the Institute for Healthcare Improvement Breakthrough Series College.

Using a PDSA Worksheet PLAN: activities and timelines, including person responsible DO: describe what actually happened during test STUDY: review data collected during plan phase and compare to predictions ACT: determine what to change and what to keep based on previous plan cycle (this is a new PLAN) 1. Make sure that the team meets to develop the aim, measures, and plans for testing the changes. Everyone needs to feel ownership. 2. The first team meeting should make a plan for completing the first test cycle. It’s important for the team to move as quickly as possible to action…testing a change that makes a difference in the process to be improved. 3. Once the test has been completed and the plans for the next test have been discussed, the team can share its learning process with others in the organization, such as the sponsor or senior leader. [Use the P-D-S-A format as the template for these reports.] 4. The team should meet with the senior leader to share its project plans with the senior leader giving guidance and feedback. BREAK FOR A MOMENT TO TALK ABOUT: WHO IS SENIOR LEADER IN YOUR SYSTEM?? WHAT DOES THIS MEAN FOR YOU? The team can also identify specific resources it may need to support the work of the team or specific issues that need to be resolved. Schedule the next meeting at the end of the first team meeting to ensure regular contact between the team and the senior leader. Take status quo off the table Make project visible to others Ensure success of QI team - remove barriers and obstacles Plan and manage spread