1. Analyzing & Presenting Performance Improvement (PI) Data

2. Objectives  Demonstrate an exercise that uncovers how leaders make managerial decisions based upon data  Identify barriers to effective analysis and reporting of PI data  Share 2 data analysis/reporting educational tools targeted for staff  Provide sample PowerPoint slides for staff training re: data analysis and process variability  Discuss PI information needs of leadership CSR ©2011

3. 123

8. Why aggregate and analyze?  Transform data into information  Identify current performance levels, patterns, or trends  Determine  Whether or not improvement is needed  If a strategy to stabilize or improve performance was effective  If design specifications met  Judge a particular process’s stability or a particular outcome’s predictability in relation to performance expectations

9. Problem #1 Lumping data together is usually not enough!  Aggregate #’s do not show any “unusual” circumstances.  If leaders take action based on data assumptions without taking into account unusual circumstances – they can muck up a perfectly good process!

10. Time to work each day MinutesMinutes Should I change the route to work each day?

11. September’s Rates – Minutes to work

12. Problem #2 Before and after measures aren’t enough!  Two aggregate measures taken before and after a change do not in themselves demonstrate that a process has improved.  One needs to know the stability of the processes that produced these aggregate measures.  To determine process stability, it is necessary to look at data over time i.e., in a time series design.

13. Staff Turnover Intervention Begins 10/2009

14. Staff Turnover – the same data using the # of staff over time! Intervention Begins 10/2009

15. Sheward & Deming Points  Variation exists in all we do  Processes that exhibit common causes of variation are predictable within statistical limits  Processes often have both common and special cause variation  How we respond to special causes is different than our response to common cause variation  Attempting to improve processes that contain special causes will increase variation and waste resources  Once special causes have been “eliminated”, it is appropriate to consider changing the process

16. Common vs. Special Cause Variation Common Cause  Is inherent in the design of the process.  Is due to regular, natural, or ordinary causes.  Results in a stable process. The variation is predictable.  Also known as random or unassignable causes. Special Cause  Is due to causes not inherent in a process.  Results in an unstable process, because the variation is not predictable.  Also know as non-random or assignable causes.  Might be described as a “signal” that the process has changed. CSR ©2011

17. Neither type of variation is “good” or “bad” in itself! Common Cause  Only tells you that a process is stable and predictable within certain limits  However, it may be functioning at an unacceptable level! Special Cause  Usually undesirable when you did not plan for it.  Can also be a “signal” that a planned change was effective.

18. When people do not understand variation  See trends where there are no trends  Blame and give credit to others for things over which they have little or no control  Build barriers, decrease morale, and create an atmosphere of fear  Never be able to fully understand past performance, make predictions about the future and make significant improvements in processes

19. How Do We Analyze Variation?  Run charts and control charts are the tools used to determine whether variation is:  Common cause, or  Special cause  They tell us what the process is actually doing –  Not what we would like it to do!

20. Bar Graphs  A preliminary exploration of data  may be time- ordered  Are a common graphical display format  Can be difficult for trend detection 20 JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC 0 200 400 600 800 1000 1200 1400 2009 2010

21. Same bar graph data displayed in a simple Line Graph  Offers a preliminary view of time ordered data  Stock market trends are viewed in line graphs  Common mistake is to see trends where they statistically don’t exist 21 JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDECJANFEBMARAPRMAYJUN 400 500 600 700 800 900 1000 1100 1200 1300 LINE GRAPH

22. Run Chart  Used to detect common cause vs. special cause variation  Easy to construct and evaluate  Less sensitive than control charts for identifying extreme data points as a special cause 22 JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDECJANFEBMARAPRMAYJUN 400 500 600 700 800 900 1000 1100 1200 1300 RUN CHART: Monthly Calls Received 19992000 MEDIAN "COMMON CAUSE VARIATION"

23. Run Chart = Line Graph + Center Line* 23 *The center line in a run chart is typically the median point for the data, but some use the process average or mean as the center line JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDECJANFEBMARAPRMAYJUN 400 500 600 700 800 900 1000 1100 1200 1300 RUN CHART: Monthly Requests for Services 19992000 MEDIAN "COMMON CAUSE VARIATION"

24. Run Chart Terminology  RUN  Defined as one or more consecutive data points occurring on the same side of the center line  TREND  Defined as an unusually long series of data points steadily increasing or decreasing 24

25. EASY Run Chart Tests for Special Causes  TREND of 6 consecutive data points steadily increasing or decreasing  RUN of 8 consecutive data points on one side of the center line (median or mean)  OUTLIER POINTS – use your judgment whether to expend resources to investigate further to understand cause and determine if improvement is needed 25 CSR ©2011

26. Median Test #1 TREND of 6 consecutive data points steadily increasing or decreasing Sequential Run of only 5 – Common Cause Variation

27. Median Test #2 RUN of 8 consecutive data points on one side of the center line (median or mean) Run of 9 – Special Cause Variation

28. median Test #3 OUTLIER POINTS – use your judgment whether to expend resources to investigate further to understand cause and determine if improvement is needed Is May’s result a special cause????

29. Improvement Strategies: After making a run or control chart, what’s next? 29 The type of variation determines your approach: Special cause variation?  If negative, eliminate it.  If positive, emulate it.  But don’t change the process! Common cause variation?  If process is functioning at an unacceptable level, change the process!  Don’t “tamper” with individual data points!

30. How will you know your intervention is a success? 30  A Special cause in the desired direction will signal that the old process is changed for the better.  A Special cause in the wrong direction will indicate that your intervention was counterproductive.  Continued common cause variation will indicate that your intervention did not help – but did not hurt either. CSR ©2011

31. Improvement Strategy 31 Time 1Time 2Time 3Time 4 Conduct Initial Investigation Standardize The Process Introduce Improvement - 1 Introduce Improvement - 2

32. Targeting Your Message Hospital boards should hold accountable and require full and complete explanations from management when safety and quality performance levels differ significantly from national benchmarks or fall below expectations, with specific attention devoted to the organization’s plan for improvement (e.g., its development, performance expectations, and the basis on which expectations are established). ​ Hospital Governing Boards and Quality of Care: A Call to Responsibility. Washington, DC: National Quality Forum; 2004.

33. Leadership should…. Create alignment between organizational strategy, measures, and improvement projects Unify Quality Improvement Plans, Strategic Plans, and Financial Plans within the organization Ensure that the daily work of employees is organized to support deployment of strategies and improvement projects chosen because of their direct impact on system-level measures or direct support of strategic objectives. Leaders should then implement, monitor, and revise the strategy as needed if the desired changes are not occurring. Botwinick L, Bisognano M, Haraden C. Leadership Guide to Patient Safety. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2006. (Available on www.IHI.org)

37. Use of Lean/6 Sigma Potential Topics to Report: Voice of the customer, suppliers and process workers Key critical customer requirements Outputs which are “Critical to Quality” (CTQ) Rating of relationship between the process steps [inputs] to the customer requirements Current Process Controls Prevention Current Process Controls Detection FMEAs – findings about severity, occurrence and detection Description of standard work – current and future states (i.e., value stream map)

38. Questions? Richard Scalenghe, CPQH rscalenghe@jcrinc.com 630-740-7914