1. Interpreting Run Charts and Shewhart Charts

2. Agenda Features of Run Charts Interpreting Run Charts
A quick mention of variation
Features of Shewhart Charts
Interpreting Shewhart Charts

3. Displaying Key Measures over Time – Run Chart
Data displayed in time order
Time is along X axis
Result along Y axis
Centre line = median
One “dot” = one sample of data

4. Three Uses of Run Charts in Quality Work
Determine if change is an improvement
Three Uses of Run Charts in Quality Work
Median 429
Slide source: L. Provost, used with permission
The Data Guide, p 3-18 4

5. Three Uses of Run Charts in Quality Work
2. Determine if improvement is sustained
Median 429
Slide source: L. Provost, used with permission
The Data Guide, p 3-18 5

6. Three Uses of Run Charts in Quality Work
3. Make process performance visible
Median 429
Slide source: L. Provost, used with permission
The Data Guide, p 3-18 6

7. How Do We Analyze a Run Chart?
Visual analysis first
If pattern is not clear, then apply probability based rules
Slide source: L. Provost, used with permission
The Data Guide, p 3-10 7

8. Non-Random Signals on Run Charts
A Trend
5 or more
A Shift:
6 or more
Too many or too few runs
An astronomical data point
Slide source: L. Provost, used with permission
Evidence of a non-random signal if one or more of the circumstances depicted by these four rules are on the run chart. The first three rules are violations of random patterns and are based on a probability of less than 5% chance of occurring just by chance with no change.
The Data Guide, p 3-11 8

9. Source: Swed, Frieda S. and Eisenhart, C
Source: Swed, Frieda S. and Eisenhart, C. (1943) “Tables for Testing Randomness of Grouping in a Sequence of Alternatives.” Annals of Mathematical Statistics. Vol. XIV, pp , Tables II and III.
The Data Guide, p 3-14 9

10. Trend? Note: 2 same values – only count one
Slide source: M. Rathgeber, MERGE consulting, used with permission

11. Shift? Note: values on median don’t make or break a shift
Slide source: M. Rathgeber, MERGE consulting, used with permission

12. Shift?
Slide source: M. Rathgeber, MERGE consulting, used with permission

13. Interpretation? There is a signal of a non-random pattern
Slide source: M. Rathgeber, MERGE consulting, used with permission
There is a signal of a non-random pattern
There is less than 5 % chance that we would see this pattern if something wasn’t going on, i.e. if there wasn’t a real change

14. Plain Language Interpretation?
Slide source: M. Rathgeber, MERGE consulting, used with permission
There is evidence of improvement – the chance we would see a “shift” like this in data if there wasn’t a real change in what we were doing is less than 5%.

15. Two few or too many runs?
1. bring out the table 2. how many points do we have (not on median?) 3. how many runs do we have (cross median +1) 4. what is the upper and lower limit?
Slide source: M. Rathgeber, MERGE consulting, used with permission

16. Two few or too many runs?
1. bring out the table 2. how many points do we have how many runs do we have (cross median +1) what is the upper and lower limit?
Slide source: M. Rathgeber, MERGE consulting, used with permission

17. Two few runs? Plain language interpretation
Slide source: M. Rathgeber, MERGE consulting, used with permission
There is evidence of improvement – our data only crosses the median line twice – three runs. If it was just random variation, we would expect to see more up and down.

18. Two many runs? Plain language interpretation
Slide source: M. Rathgeber, MERGE consulting, used with permission
There is evidence of a non-random pattern. There is a pattern to the way the data rises and falls above and below the median. Something systematically different. Should investigate and maybe plot on separate run charts.

19. Astronomical Data Point?
Slide source: M. Rathgeber, MERGE consulting, used with permission

20. Understanding Variation
Walter Shewhart
(1891 – 1967)
W. Edwards Deming
( )
The Pioneers of Understanding Variation

21. Intended and Unintended Variation
Intended variation is an important part of effective, patient-centered health care.
  Unintended variation is due to changes introduced into healthcare process that are not purposeful, planned or guided.
Walter Shewhart focused his work on this unintended variation. He found that reducing unintended variation in a process usually resulted in improved outcomes and lower costs.
(Berwick 1991)
Slide source: L. Provost, used with permission
Health Care Data Guide, p. 107

22. Shewhart’s Theory of Variation
Common Causes—those causes inherent in the system over time, affect everyone working in the system, and affect all outcomes of the system
Common cause of variation
Chance cause
Stable process
Process in statistical control
Special Causes—those causes not part of the system all the time or do not affect everyone, but arise because of specific circumstances
Special cause of variation
Assignable cause
Unstable process
Process not in statistical control
Could insert “a” game
Health Care Data Guide, p. 108

23. Health Care Data Guide, p. 113
Shewhart Charts
The Shewhart chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes
Slide source: L. Provost, used with permission
(Most common name is a control chart, more descriptive would be learning charts or system performance charts)
Health Care Data Guide, p. 113

24. Control Charts – what features differ from a run chart?

25. Control Charts/Shewhart Charts
upper and lower control limits
to detect special cause variation
Extend limits to predict future performance
Not necessarily ordered by time
advanced application of SPC – is there something different between systems 25

26. Example of Shewhart Chart for Unequal Subgroup Size
Health Care Data Guide, p. 114

27. Adapted from Health Care Data Guide, p. 151 & QI Charts Software

28. Slide source: L. Provost, used with permission

29. Health Care Data Guide, p. 116
Note: A point exactly on the centerline does
not cancel or count towards a shift
Health Care Data Guide, p. 116 29

30. Slide source: L. Provost, used with permission

31. Special cause: point outside the limits
Slide source: L. Provost, used with permission

32. Slide source: L. Provost, used with permission

33. 2 out of 3 consecutive points in outer third of limits or beyond
Special cause
2 out of 3 consecutive points in outer third of limits or beyond
Slide source: L. Provost, used with permission

34. Slide source: L. Provost, used with permission

35. Slide source: L. Provost, used with permission

36. Slide source: L. Provost, used with permission

37. Common Cause
Slide source: L. Provost, used with permission

38. Health Care Data Guide, p. 116
Note: A point exactly on the centerline does
not cancel or count towards a shift
Health Care Data Guide, p. 116 38

39. Case Study #1a

40. Case Study #1b
Percent of cases with urinary tract infection

41. Case Study #1c Percent of cases with urinary tract infection

42. Case Study #1d
Percent of cases with urinary tract infection

43. Case Study #1e
Percent of cases with urinary tract infection

44. Case Study #1f
Percent of cases with urinary tract infection

45. Health Care Data Guide, p. 116
Note: A point exactly on the centerline does
not cancel or count towards a shift
Health Care Data Guide, p. 116 45

46. Case Study #2a
Percent of patients with Death or Serious Morbidity who are >= 65 years of age

47. Case Study #2b
Percent of patients with Death or Serious Morbidity who are >= 65 years of age

48. Case Study #2c
Percent of patients with Death or Serious Morbidity who are >= 65 years of age

49. Case Study #2d
Percent of patients with Death or Serious Morbidity who are >= 65 years of age

50. References BCPSQC Measurement Report
Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP (2009) The Improvement Guide (2nd ed).
Provost L, Murray S (2011) The Health Care Data Guide.
Berwick, Donald M, Controlling Variation in Health Care: A Consultation with Walter Shewhart, Medical Care, December, 1991, Vol. 29, No 12, page
Perla R, Provost L, Murray S (2010) The run chart: a simple analytical tool for learning from variation in healthcare processes, BMJ Qual Saf :
Associates in Process Improvement website