1. Improvement Forum    A webinar series for QI Managers, Nurse Leaders and others supporting healthcare improvement in Wisconsin’s hospitals    June 2012

2. How do you know you have improved?    Our Topic for June 2012    Travis Dollak, Quality Coordinator Tom Kaster, Quality Coordinator

3. Today’s Agenda 3 Introduction Content Sharing – Measurement is for Learning – What to measure – Measuring what matters – The problem with “drift” – Resources Discussion Questions

4. Focus on Measurement Why measure? The main reason for conducting an improvement project is to achieve results, no matter the issue or topic. And how do we know we have achieve a desired result that can be proven to others? We must demonstrate change from a baseline, or initial measurement, and assess the degree of change after an intervention. 4

5. 70 Million Americans Benefit from Quality Measurement 96% of heart attack victims were prescribed beta-blocker treatment in 2005, up from 62% in 1996* 77.7% of children enrolled in private health plans received all recommended immunizations, up 5% from 72.5% in 2004* Evidence-based guidelines from the American College of Cardiology and the American Heart Association have reduced mortality among patients who have had a heart attack * National Committee for Quality Assurance

6. Areas of Measurement Relies on the actual execution of the PDSA cycle Disclaimer information here… 6 Aims Measurement Change ideas Testing ideas before implementing changes Process Measures

7. Diet Driver Diagram 7

8. Reducing Falls Driver Diagram Disclaimer information here… 8

9. Poll Question 1: Process Measures How often does your facility measure processes for your improvement projects? – Always – Almost Always – Sometimes – Never Disclaimer information here… 9

10. How to develop process measures Ask: – How does the work get done? – How would I know? – What is important to know? – What is the easiest way to know? – What is already collected? Is it good enough?

11. Real Word Example – Losing Weight 11 Outcome Measure: I want to loose 10 lbs by July 4, 2012 – Stepping on the Scale can lead to moderate improvement but will plateau Process Measures: To lose 10 lbs by July 4 th, I will measure: – Calorie intake – Analyze what I eat – Time spent exercising – Analyze how often and what type of exercise

12. Clinical Example – Falls Prevention Outcome Measure: Reduce all falls by 50% by 12/31/2013 Process Measure: To reduce all falls by 50% we will measure: – The prevalence of a daily fall risk assessment being completed – How often the care plan identified in the risk assessment is in place and adhered to 12

13. Poll Question 1 Results: Process Measures How often does your facility measure processes in your improvement projects? – Always – Almost Always – Sometimes – Never Disclaimer information here… 13

14. AspectImprovementRegulatoryResearch AimImprove careCompare, reassure, spur change New knowledge Methods Test Observable YesN/A. Evaluate current performance Test blind or controlled BiasAccept stable biasAdjust data to reduce bias Design to eliminate Sample SizeJust enough data, small sequential samples N/A. Report 100%Just in case data Hypothesis Flexible No. Revised as learn and test No hypothesisFixed hypothesis How to determine improvement Run or control chartsNo focus on changeHypothesis, Statistical tests: F-test, t-test, chi square, p value Testing StrategySmall sequential testsNo tests1 large test Data confidential Data used only by those involved in improvement No subjects. Data is for public Subjects protected

15. Measuring Effectively Seek usefulness, not perfection Use sampling Plot data over time Don’t wait for the information system 15

16. Usefulness, Not Perfection Usefulness means measuring just enough to tell you what direction you are headed Perfection can lead to paralysis by analysis State/Federal Criteria can cause us to focus efforts on perfect data and less on improvement 16

17. Keeping measurement simple Use Simple Visuals Use Tic and Tally Sheets Make your measures easy to track on a daily or weekly basis

18. Why Sample? Benefits: Lower cost Saves time (receive information faster) With smaller data set, its easier to improve the accuracy/quality of the data Example: Sample 20 pts/month using IHI trigger tool to identify ADEs yields the same results as sampling entire population http://www.1000livesplus.wales.nhs.uk/sitesplus/documents/1011/T4I%20%284%29%20How%20to%20use%20Trigger%20Tools%2 0%28Feb%202011%29%20Web.pdf 18 Example: Finding ADEs w/ IHI Trigger Tool Lower cost Saves time (receive information faster) With smaller data set, its easier to improve the accuracy/quality of the data

19. Displaying Data Over Time Why use graphs & charts? Graphing and charting are useful tools when there is a lot of data to display, or a simple comparison of data in a table is not adequate to explain changes in the data. Some methods to display data are more appropriate than others. 19

20. Why be visual? 20 # of ADEs per 1,000 Doses

21. 21 Remember to “tell the story” about how you achieved these results….

22. Poll Question #2: Annotated Run Charts How comfortable are you with developing and using annotated run charts to measure your improvement projects? – Very comfortable – Somewhat comfortable but would like more help – Not comfortable and need more help – What is an annotated run chart? Disclaimer information here… 22

23. When Reaching Your Goal Measurement does not stop Staying at ‘zero’ – Continuous monitoring Monitoring early warnings – New orientation* – Revisit training* 23

24. Summary Measure to learn – use process measures Seek usefulness, not perfection Display your data in a meaningful way Connect your driver diagram to your process measures Avoid drift – continuously monitor 24

25. Questions and Answers    What can we learn from each other?    Stephanie Sobczak, MS, MBA Manager QI, Wisconsin Hospital Association Next Month’s Topic: Accelerating Change through small tests