California Chronic Care Learning Communities Initiative Collaborative Learning Session I How Do We Get There Quickly? Model for Improvement - Part 2: Testing Changes with PDSA Cycles Angela Hovis, Improvement Advisor

© 2004 Institute for Healthcare Improvement Session Objectives Participants will be able to: Describe the Model for Improvement and its utility in accelerating improvement initiatives Use the change package Describe the importance of testing changes for learning and improvement Test Changes with PDSAs

© 2004 Institute for Healthcare Improvement “Not all changes are improvements, but all improvements are a result of changes!” Testing Changes

Test Trial and Learn - try and adapt ideas Implement make a change part of the day to day operation of the system in the pilot Spread make a change part of the day to day operation of the system outside of the pilot Side Bar: Test vs Implement vs Spread

© 2004 Institute for Healthcare Improvement Fundamental Questions for Improvement What are we trying to accomplish? Team Aim Statement How will we know that a change is an improvement? Measures What changes can we make that will result in an improvement? Changes What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Model for Improvement ActPlan Stud y Do

3. What Changes Can We Make That Will Result in Improvement? The “change package” contains good concepts and ideas (based on best practices and research) for changes that can help us accomplish our goals. Use the change package to identify the changes you want to make to your system to achieve your aim.

Informed, Activated Patients and Caregivers Productive Interactions Prepared, Proactive, Practice Team Improved Outcomes for Patients with Diabetes/CAD Delivery System Design Decision Support Clinical Information Systems Self- Management Support Health System Resources and Policies Community Health Care Organization Using the The Care Model for Self-Management Support

Change Concept: A general notion or approach to change that has been found to be useful in developing specific ideas for changes that lead to improvement.

Change Concepts vs. Specific Changes Vague, Strategic, Arrange: Provide follow up to patients after they have set goal (agree) Conceptual Contact patients one week after they have set goal Find out how patients are doing with their goal - reassess confidence level and see if they have questions or concerns Specific Ideas, Actionable At time of office visit, solicit patient’s preference for follow up and have MA either call or patient within 3 weeks office visit

Using the Change Package Self-management Component of Care Model Concept Key Change Example of Idea to Try

© 2004 Institute for Healthcare Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Model for Improvement ActPlan StudyDo

© 2004 Institute for Healthcare Improvement What is the PDSA Cycle? Act What changes are to be made? Next cycle? Plan Objective Questions and predictions (why) Plan to carry out the cycle (who, what, where, when) Study Complete the analysis of the data Compare data to predictions Summarize what was learned Do Carry out the plan Document problems and unexpected observations Begin analysis of the data

© 2004 Institute for Healthcare Improvement The PDSA Cycle Why Test? ActPlan StudyDo

Three Doors and Goat: Experiencing PDSA Presented by Angela Hovis Office Practices Summit March 27, 2004 Created by: Roberto Colacioppo IMECC / UNICAMP ( Campinas – São Paulo – Brazil

The Three Doors Game Learning how to run PDSA and use forms

Pick a door, please. 123

Without opening your door, I will show you a goat, ok? 123

Do you want to switch? 123

Wow! You got the car! 123

Do you want to switch? 123

Sorry, You got the goat! 123

Flowchart The host of the show shows the three doors Contestant chooses a door Host opens a door with a goat from the two doors not chosen Host asks if the contestant wants to switch doors Contestant decides yes or no Host opens the other two doors      

Background info This story is true, and comes from the American T.V. game show “Let's Make a Deal”, presented by Monty Hall in the 70’s. This situation was discussed in the popular "Ask Marylin" question-and-answer column of the Parade magazine in the 90’s. Her answer was: "Yes, you should switch." To date, Ms. vos Savant has received over 10,000 letters, mostly disagreeable. Marilyn vos Savant Monty Hall

Is Ms. vos Savant correct? What does your intuition tell you about this problem? Does the contestant have a chance of winning with either choice? Or is it better to switch from the original choice? To learn about this question, we will work on a PDSA cycle.

The PDSA Cycle Act What changes are to be made? Next cycle? Plan Objective Questions and predictions (why) Plan to carry out the cycle (who, what, where, when) Study Complete the analysis of the data Compare data to predictions Summarize what was learned Do Carry out the plan Document problems and unexpected observations Begin analysis of the data

Let's use a basic PDSA form © Associates in Process Improvement 2002 All Rights Reserved

Project : Monty Hall Problem Cycle #: 1 Date: 03/27 Objective: Finding a solution to the Monty Hall Problem PLAN QuestionsPredictions Does the contestant have the same chance of winning with either staying or changing the original choice?  Yes, it doesn't matter  No, staying with original is better  No, switching is better What data will be collected during this time? (Forms to be used) Fill in the objective, questions and predictions

Project: Monty Hall Problem Cycle #: 1 Date: 03/27 Objective: Find a solution to the Monty Hall Problem PLAN QuestionsPredictions Does the contestant have the same chance of winning with either staying or changing the original choice?  Yes, it doesn't matter  No, staying with original is better  No, switching is better What data will be collected during this time? (Forms to be used) Work in pairs (host and contestant) Each pair will simulate both situation 10 times with playing cards Data will be collected in the form attached A Run Chart (attached) will be filled with results from all pairs Observations focused on the difference between the two situation will be collected Make a plan to collect data

EXAMPLE Using playing cards to collect data original choice Staying: 10 times Switching: 10 times Here we have a goat (red card)

Project: Monty Hall Problem Cycle #: 1 Date: 03/27 Objective: Find a solution to the Monty Hall Problem PLAN QuestionsPredictions Does the contestant have the same chance of winning with either staying or changing the original choice?  Yes, it doesn't matter. Chances are equal.  No, staying with original is better  No, switching is better What data will be collected during this time? (Forms to be used) Work in pairs (host and contestant) Each pair will simulate both situation 10 times each with playing cards Data will be collected in the form attached A Chart (attached) will be filled with results from all pairs Observations focused on the difference between the two situation will be collected Make a plan to collect data

Worksheet Data Collection for 3 doors and Goat

Project : Monty Hall Problem Cycle #: 1 Date: 03/27 Objective: Find a solution to the Monty Hall Problem PLAN QuestionsPredictions Does the contestant have the same chance of winning with either staying or changing the original choice?  Yes, it doesn't matter. Chances are equal.  No, staying with original is better  No, switching is better What data will be collected during this time? (Forms to be used) Work in pairs (host and contestant) Each pair will simulate both situation 10 times each with playing cards Data will be collected in the form attached A Chart (attached) will be filled with results from all pairs Observations focused on the difference between the two situation will be collected Make a plan to collect data

Chart for Data

Project: Monty Hall Problem Cycle #: 1 Date: 03/27 Objective: Find a solution to the Monty Hall Problem PLAN QuestionsPredictions Does the contestant have the same chance of winning with either staying or changing the original choice?  Yes, it doesn't matter. Chances are equal  No, staying with original is better  No, switching is better What data will be collected during this time? (Forms to be used) Work in pairs (host and contestant) Each pair will simulate both situations 10 times each with playing cards Data will be collected in the form attached A Chart (attached) will be filled with results from all pairs Observations focused on the difference between the two situation will be collected Make a plan to collect data

Let's carry out the plan DO What went wrong? What happened that was not part of the plan? Begin analysis. Write your observations made during simulation: What happened? What are the differences between staying with and changing the original choice? STUDY Complete analysis of data. Summarize what was learned include results of predictions

Let's study! STUDY Complete analysis of data. Summarize what was learned include results of predictions. 1. What does the variation the data graph indicate? 2. What hints can we extract from the observations to explain such behavior of the data? 3. Can we conclude and formulate a theory about the Monty Hall Problem? 4. Summarize what was learned. Compare to predictions.

Decisions... ACT What decisions were made from what was learned? What will the next cycle be?

What did we learn from this exercise?

© 2004 Institute for Healthcare Improvement Why Test Changes? To increase the belief that the change will result in improvements in your setting To avoid surprises or challenge assumptions To learn how to adapt the change to conditions in your setting To evaluate the costs and “side-effects” of changes To minimize resistance when spreading the change throughout the organization

Some comments and feedbacks: PDSA… … helps to focus and to organize our job (objective, questions and predictions). (P) … helps to plan which data will answer our questions (it stimulates using data). (P) … stimulates to plan the analysis of the data before obtain it (avoid waste). (P) … stimulates to document observations (D) … stimulates insights and creativity. (D) … stimulates to test paradigms by comparing predictions and actual data. (S) … helps to learn what is necessary to go ahead. (S) … stimulates to apply the specific knowledge obtained. (A) … is an efficient way to document our improvement project. (form)

WATCH OUT!!

Some misuses of PDSA by beginners Tendency to make objective statements too broad Confusion between objective and questions Tendency to formulate many irrelevant questions to the PDSA objective Tendency to put Plan issues (e.g. What data do we need? Who will collect data?) mixed with PDSA questions Tendency not to anticipate in the Plan step how they will analyze data to be collected Tendency not to record useful observations in “Do” In Study, they draw conclusions that don't answer PDSA questions (they forget the original focus) Tendency to do too much in one THE PDSA, instead of cycles Tendency to use PDSA for collecting data on a measure rather than to test an idea for change

© 2004 Institute for Healthcare Improvement “Persuasion is not enough…… After adopters understand the new ways, they must be provided with the necessary guidance and ample opportunity to perfect the modeled activities under circumstances where they need not fear making mistakes.” Albert Bandura Promoting Adoptive Behavior : Moving to Action

© 2004 Institute for Healthcare Improvement Bon Secours First PDSA Cycle (Office Practices - October, 2002) Plan Objective of first test: To know how to motivate patients to healthier behavior with respect to HbA1c education Question: What is the patient's perception related to HbA1c>7? Prediction: Lack of knowledge (pt) and not motivated to lose weight, test blood, etc. (who, what, where, when, how) Pt identified in teams first data collection (pre-IMPACT meeting) contacted to come in for visit on 10/14 to identify barriers. Measure(s) to assess the success of this test - How's your health?" tool and patient interview.

© 2004 Institute for Healthcare Improvement Bon Secours First PDSA Cycle Do One patient was interviewed and used the How's Your Health tool. The pt is well educated and had insight that when the MD calls him when his HbA1c is going up-he adheres to diet and exercise. He also identified that travel is a problem with compliance. They mapped a trend related to his blood. He also identified from the tool some areas that had not really focused on such as foot care.

© 2004 Institute for Healthcare Improvement Bon Secours First PDSA Cycle Study See above-we disproved our theory-pt is knowledgeable but we need to customize teaching relative to travel and staying on the path. Pt more motivated when someone is calling him back or contacting him and he is motivated to be contacted by so this will be part of his plan.

© 2004 Institute for Healthcare Improvement Bon Secours First PDSA Cycle Act As noted in first test- pts surveyed at the practice in a day >50% do want to be contacted and wish to contact office by . Computer being set up in office space to go ahead and arrange to answer by staff and gather addresses. Jeff Glover developed a questionnaire for diabetic pt that he is giving as they are being scheduled to come in (those with HbA1c>7) to start. Letter drafted to introduce the survey tool and we will look to add some incentive for returning the survey such as return envelope included or perhaps movie tickets? Will reformat tool to 1 page and we will to Marie after reformat. Actively investigating the registry tools and expanding review of diabetic charts to reschedule pts to be seen.

© 2004 Institute for Healthcare Improvement Tips for Testing Changes Identify changes that will assist the team to reach goals Harvest good ideas and adapt them to your setting Solicit team members ideas Start with changes that are easy to implement Link multiple testing cycles to accelerate learning

© 2004 Institute for Healthcare Improvement Tips for Testing Changes Test large innovative changes on a small scale (e.g. one MD, one patient) Collect useful data during each test Learning from “failed” tests Test over a wide range of conditions Develop plans to simulate the change Make changes side-by-side with the existing system

© 2004 Institute for Healthcare Improvement Move Quickly to Testing Changes Year Quarter Month Week Day Hour “What tests can we complete by next Wednesday?”

Repeated Use of the PDSA Cycle Hunches Theories Ideas Changes That Result in Improvement AP SD A P S D AP SD D S P A DATA Very Small Scale Test Follow-up Tests Wide-Scale Tests of Change Implementation of Change

Build knowledge sequentially with multiple PDSA cycles for each change idea. Peak flow meters for high-risk patients Routine use of flow meters by high-risk patients AP SD A P S D AP SD D S P A DATA D S P A Cycle 1:Test communication on use of flow meters with 1 patient Cycle 2: Revise and test with 2 provider and 6 patients Cycle 3: Revise and test with 3 providers and their patients for one week Cycle 4: Train providers on teaching patients to use flow meters Cycle 5: Monitor communication and use of flow meters with high-risk patients

A P S D A P S D A P S D D S P A A P S D A P S D A P S D D S P A A P S D A P S D A P S D D S P A A P S D A P S D A P S D D S P A Recruitment Size and Location The Right Team Example of Testing Multiple Changes Program Content Cycles for Implementing Group Visit

Table Exercise Plan for Action Period 1 and First PDSA 1st PDSA By next Wednesday, November 3

© 2004 Institute for Healthcare Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Model for Improvement ActPlan StudyDo

© 2004 Institute for Healthcare Improvement References The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. G. Langley, K. Nolan, T. Nolan, C. Norman, L. Provost. Jossey-Bass Publishers., San Francisco, Quality Improvement Through Planned Experimentation. 2nd edition. R. Moen, T. Nolan, L. Provost, McGraw-Hill, NY, “Understanding Variation”, Quality Progress, Vol. 13, No. 5, T. W. Nolan and L. P. Provost, May, A Primer on Leading the Improvement of Systems,” Don M. Berwick, BMJ, 312: pp , “Accelerating the Pace of Improvement - An Interview with Thomas Nolan,” Journal of Quality Improvement, Volume 23, No. 4, The Joint Commission, April, 1997.