1. Statistical Process Control (SPC) and Shewhart Charts QA Cohort 6 Residency 3.

2. Shewhart Charts aka Control Charts

3. Objectives By the end of this session, you will be able to: 1.Determine when to use a control chart in your work 2.Explain key features of a control chart 3.Choose the right chart for your type of data 4.Apply tests to analyze a control charts 5.Understand purpose of revising control limits on the chart

4. Reference for all content:

5. A Control Chart

6. A Control Chart with Different Sample Sizes

7. Methods for Improvement, Accountability and Research Improvement Accountability/Performance Measurement Clinical Research Purpose Improvement of care; application of evidence Comparison between organizations or to a criteria, reassurance, spur for change Generate evidence or new knowledge Data Collection Accept consistent bias in data collection Statistical adjustments to reduce bias Design of experiment to reduce bias Sample Size Small sequential samples of data 100% of available data Large samples so there is enough data for possible hypothesis Analysis Run Charts of Shewhart ChartsNAInferential statistics and hypothesis testing (assumes same : T-tests, F-tests, Regression Analysis, etc). with p-value Adapted from: Solberg, L.I., Mosser, G., McDonald, S. (1997). The three faces of performance measurement: improvement, accountability, and research. Joint Commission Journal on Quality Improvement, 23(3), 135-147.

8. Control Charts: For Improvement or for Accountability/Performance Measurement? Local level improvement:  Shewhart Chart – use to learn about factors that are influencing your system  Can use either a Shewhart Chart or a Run Chart to show evidence of improvement Accountability  Use to show the impact of a large improvement effort, or a series of improvement efforts, over time

9. Review of Run Charts

10. What do run charts tell us? 1. Make performance visible 2. Is there improvement – probability based tests. 3. Is the improvement holding? 4. Gives a sense of impact of various PDSA cycles.

11. 11 1. Make performance visible

12. 12 2. Evidence of improvement: this “trend” is non-random pattern of data based on probability

13. Non-Random Signals on Run Charts A Shift: 6 or more An astronomical data point Too many or too few runs A Trend 5 or more The Data Guide, p 3-11 Evidence of a non-random signal if one or more of the circumstances depicted by these four rules are on the run chart. The first three rules are violations of random patterns and are based on a probability of less than 5% chance of occurring just by chance with no change.

14. 14 3. Is the improvement holding?

15. 15 4. Impact of PDSA cycles or specific changes

16. Why would you need a control chart? Variation Intended Variation Unintended Variation Common Cause Variation Special Cause Variation

17. Control Charts A control chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes

18. Differences between a Run Chart and a Control Chart

19. Control Limits: use to identify Special Cause Variation 3 lines on a control chart: - centre line (mean) - upper control limit (mean+ 3 sigma) - lower control limit (mean – 3 sigma)

20. Control Limits: use to identify Special Cause Variation Calculations for limits: - depend on the type of data - not the same as confidence intervals - recommend using SPC software

21. Control Limits: use to identify Special Cause Variation Statistical properties: probability of misinterpretation is low (less than 11% overall; less than 5% for some data)

22. When to use a control chart instead of a run chart? 1.To determine change quickly (improvement will be picked up more quickly than in a run chart) 2.Different samples sizes in each period 3.To see if results are stable and predictable? 4.Predict performance in coming weeks or months. 5.Reasons for variation

23. 1. Evidence of Special Cause Variation (Improvement)

24. 2. Different Sample Size for Each Time Period

25. 3. Is system stable and predictable?

26. 4. What will result be next month?

27. 5. What are the sources of variation? Rational ordering – covered in Res 4

28. Is Special Cause Variation Good or Bad? For improvement work: 1.Bad: evidence of SCV in our existing system (baseline) means system is not stable. 2.Good: if we start off with a stable system, then we introduce improvement, SCV is evidence of improvement.

29. Special Cause Variation as Evidence of Improvement – Plain Language Interpretation We can be confident that the variation we are seeing is not merely due to chance, nor due to the effect of common causes – it is likely due to the improvement effort.

30. Reflection 1. Opportunities for control charts in your improvement project – what will they tell you? 2. Opportunities for control charts in other parts of your work – what will they tell you?

31. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

32. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

33. Commonly Used Type of DataType of Chart ExampleSpecs Percent dataP chartPercent of patients who received the flu shot. Need numerator and denominator for each time period Count Data (usually counts of something we are trying to reduce) C chartNumber of complaints received Need to count the complaints each time period Count Data as a Rate (usually infections or injuries) U chartInfections per patient days Need to count the infections and know what your base is (how many patient-days in the time period) AveragesX bar and SAverage length of stayNeed raw data – length of stay for each patient in that time period. You will get 2 charts displayed – one showing the average plotted on the control chart and one with the variation between points plotted on the chart. Individual Data or Data Already Calculated I ChartLength of Stay for Each patient plotted separately Average length of stay Percent of patients who received flu shot. Use if you only have one number for each time period. Either because you are only including one patient or if all you have is data that was already calculate.

34. Type of Data Count or Classification (Attribute Data) Count (nonconformities) 1,2,3,4, etc. Classification (nonconforming) either/or, pass/fail, yes/no Equal area of opportunity Unequal area of opportunity Unequal or equal subgroup size Continuous (Variable Data) Time, money, scaled measures, Workload or productivity Subgroup size of 1 (n=1) Unequal or equal subgroup size (n>1) C Chart U Chart P Chart I Chart (also known as X Chart) X Bar and S Number of nonconformities Nonconformities per unit Percent nonconforming Individual measurement Average and standard deviation Shewhart Chart Selection Guide Each subgroup is composed of a single data value Each subgroup has more than one data value - Health Care Data Guide, p. 151

35. Adapted from Health Care Data Guide, p. 151 & QI Charts Software

36. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 1. You have length of stay for all patients on your unit. You want to display it by month.

37. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 2. Each week you calculate the number of patients who have a follow up booked (of all the patients you saw in the clinic)

38. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 3. Every week, you plot the number of patient complaints received by your office

39. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 4. Every month, you plot the number of times a restraint or seclusion was initiated, taking into account patient-days in that month.

40. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 5. Every day in January, you calculate and plot the number of minutes between the 1 st scheduled surgery start time and actual start time for the 1 st surgery.

41. Practice Choosing a Chart Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart 6. Every day in January, you want to plot the number of minutes between the scheduled surgery start time and actual start time for all surgeries that day.

42. Homework: fill this table out for each key measure on your charter Attribute or Continuous If Attribute, is it a count or classification (2 buckets) If count, is there an equal or unequal ‘opportunity’ If continuous– is subgroup size=1 or >1? Type of Chart

43. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

44. Using SPC Software

45. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

47. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

49. Case Study #1a

50. Case Study #1b Percent of cases with urinary tract infection

51. Case Study #1b Percent of cases with urinary tract infection

52. Case Study #1c Percent of cases with urinary tract infection

53. Case Study #1c Percent of cases with urinary tract infection

54. Case Study #1d

56. Case Study #1e Percent of cases with urinary tract infection

57. Case Study #1e Percent of cases with urinary tract infection

58. Outer third for Dec 2012 point is anywhere between 31 and 39 Subtract UCL-Centre Line (39-14=25) Divide by 3 (thirds) (25/3 =8) Subtract result from UCL (39-8 =31) Outer third

59. Analyzing a Control Chart 1.Choose type of chart appropriate for the data (e.g. a P chart or a C chart) 2.Create a control chart once you have about 12 data points 3.Should have approximately 20 data points before establishing initial limits (freeze baseline limits) 4.Analyze using tests to detect special cause variation 5.Revise control limits as required

60. Revising Limits After Evidence of Improvement Health Care Data Guide, p. 124

61. Investigating Special Cause Analyzing control charts is an iterative process Based on analysis: 1.If SCV indicates a new system – revise limits 2.If SCV is a problem that you have dealt with – take out point and update limits

62. As an example, let’s say we discovered that the data in Period 8 was really low because there was a new clerk on that month. We now train all our new and casual staff on Med Rec. We can now actually remove this point. Investigating Special Cause Percent of Transfers with Med Rec Complete

64. More examples – interpreting SPC 64 Is there evidence of SCV? What do you think it means?

65. 65 This system is unstable. Should investigate the reasons for special causes before trying to implement improvement.

66. Is there SCV in this baseline data? What is the interpretation? Percent of cases seen by appropriate care provider.

67. Might be evidence of improvement in P13. Revise limits after 20 data points and then interpret. Percent of cases seen by appropriate care provider.

68. This hospital is part of a falls Collaborative. Is there SCV? What is the interpretation? What do the extended control limits tell you?

69. Stable system. No evidence of improvement (or no evidence it is getting worse). If no changes to the system, falls can be anywhere from 2 to 24 per month.

70. As part of a project to increase teams with patient reps, we went back and collected baseline data for one year then “extended” and “froze” these limits. New data that is added (post implementation) will be compared to the baseline limits.

71. The new data is now being compared to the old data. Note how the limits did not change with this new data (because we froze them). Is there evidence of improvement? Starting where?

72. New system. We will compare new data to these new limits now – which make more sense for the new system we have created.

73. Your updated job description….. as a QI expert Knowing what kind of Shewhart Chart to use for different data Knowing how to create the chart with control limits Applying tests for analyzing Shewhart Charts (evidence of SCV) Collaborating with others to understand the source of SCV Deciding when/if to revise the limits – using software to do revisions Explaining to others the benefit of SPC and why it is more appropriate than inferential statistics for healthcare improvement

74. Practice Analysis and Interpretation – Case Studies

75. Case Study #1 1.What is the appropriate chart – P, C, U, I, or Xbar and S? 2.Is there evidence of special cause variation? Mark it on the chart. Would you consider revising limits – where and why? 3.If you were discussing this with the manager, what would you suggest in terms of next steps? 4.Note data display principles (no gridlines, use of white space). Any issues? Turnaround time to clean exam room after each patient. Manager is considering an improvement project to reduce turnaround time to 6 minutes or less. She saw an average turnaround time for last month and says the average is 8 minutes, so she doesn’t think they need any major changes. They measure turn around times for 60 consecutive patients and you plot it on a control chart.

76. Case Study #2 1.What is the appropriate chart – P, C, U, I, or Xbar and S? 2.Is there evidence of special cause variation? Mark it on the chart. 3.Would you consider revising your limits? 4.What would your interpretation be. Write a one paragraph news article. This is the number of falls (indoors) reported in PSLS in residential care facilities (province wide). PSLS office is being asked to comment to the Vancouver Sun on the number of falls and whether they have been decreasing over the last 4 years, since there was a campaign to prevent falls.

77. Case Study #3 1.What is the appropriate chart – P, C, U, I, or Xbar and S? 2.Is there evidence of special cause variation? Mark it on the chart. (note – the upper limit is “artificial” because it caps at 100. Do not use upper line in your interpretation. 3.Would you consider revising your limits? What would this help you determine? 4.Your staff are frustrated and don’t think it is possible, because when patients have lots of meds it takes too long. Also, they say that when the project first started, the nurses were more keen to do it but now too many other improvement projects have taken precedence. This is the percent of patients with med reconciliation completed upon discharge (by week). Your unit just started this practice, and you set a goal of 60% by the end of this year.

78. Case Study #4 1.This is an Xbar and S chart. Can you describe what each chart is displaying? 2.Is there evidence of special cause variation? Mark it on the chart. 3.Do you think the project has worked? 4.Would you consider revising your limits? What would this help you determine? This is the average wait time for first time patients to see a occupational therapist in your clinic. Your clinic started a system of pooled referrals in order to decrease wait times to under 21 days.