1. How will you know that a change is an improvement?
The importance of measurement

2. Learning Objectives
Create a run chart from data collected during a PDSA Cycle.
Utilize the four rules to determine if a process has improved.
Design a PDSA cycle utilizing the knowledge gained from a previous cycle.

3. PDSA Measurement
Improvement is about making changes to processes & systems, measurement plays a key role in all
improvement efforts.
The purpose of measuring is for learning, not for
judgment or comparison.
Project teams need measures to give them feedback
that the changes they are making are having
the desired impact.

4. Data Documented observations or the results of performing a
measurement process.
Provides opportunities to obtain information & knowledge
through inquiry, analysis or summarization
Strategy is to utilize small samples, just enough to plan the
next sequential test of change.
Run and Shewhart charts are used to determine if changes
yield improvement.

5. Improvement vs. Accountability
Data for accountability is collected to assess performance
Data for improvement is collected to test a hypothesis
Mixing a system designed to collect data for improvement and a
system reporting to external sources for accountability create
chaos with neither purpose satisfied. If staff suspect data being
collected for improvement will be used for accountability, data
may be difficult to obtain.

6. Family of Measures
Health care systems are very complex. Multiple measures are
necessary to evaluate the impact of changes on the many
facets of the system.
Outcome: Focus at the project level & maintained throughout
the life of the project (Sepsis-mortality)
Process: Are the parts in the system performing as planned.
Typically show improvement before the outcome measure does.
(Turn around time of order to administration of antibiotics).
Balancing: Looks at the system from a different dimension,
what happened to the system as we changed our processes.
Aids in detecting unintended consequences (Patient Satisfaction)

7. Run or Shewhart Charts
Because improvements are made over time, in order to facilitate
learning and communication, measures should be displayed on
run charts
Time-ordered charts provide the primary way to assess the
impact of each PDSA cycle.
Better than Before-and –After, feedback is constant & ongoing
Key changes can be annotated on the charts to begin analysis of
the impact of the changes.

8. Did the process improve?
PDSA

9. Seven Steps to Constructing a Run Chart
State the question the run chart will answer
Label the horizontal scale (time, each patient….)
Label the vertical scale-easy to read. Ample room for future data
Plot the data points and connect with the line
Title the graph
Calculate the median (number in the middle of the data set)
Annotate unusual events

10. Interpreting Run Charts
In addition to the overall visual analysis, four rules may be used
to identify nonrandom signals of change on a run chart:
Rule One: A shift- 6 or more points above or below median
Rule Two: A trend-5 or more points going up or down
Rule Three: Too many or too few runs-series of points in a
row on one side of the median
4. Rule Four: Outlier

11. Case Study
A sepsis task force at hospital X hypnotized that by stocking a broad
spectrum antibiotic in the floor pxysis the turn around time from order to
antibiotic administration would decrease. They measured for 5 patients and
created a run chart. Did the process improve?
PDSA

12. PDSA P Placing antibiotic in pxysis will decrease order to
administration TAT for abx
D Run for 5 patients
S Create run chart
What happened with patient # 7?
A Patient #7 abx not ordered STAT.
Corrective action: All sepsis patient abx orders 1st dose now
Next Step: Run PDSA with hypothesis abx in pxysis + 1st dose now

13. Has the process improved ?
PDSA 1
PDSA 2
Note: It can be misleading to place a trend line on a run
Chart if there is no signal of change using the 4 rules

14. Shewhart Chart Another term is control chart
Distinguish between common cause or special cause variation
Include a center line, upper limit and lower limit (+ 3sd)
Enables you to determine process stability, process capability &
to select the appropriate improvement strategy.

15. Common Cause vs. Special Cause Variation
Common Cause= Those causes that are inherent in the process over time, affect everyone working in the system and affect all outcomes of the system.
Special Cause= Those causes that are not part of the system all the time or do not affect everyone, but arise because of specific circumstances.
The distinction between common and special causes of variation is fundamental to developing effective improvement strategies.

16. Interpreting Shewhart Charts
Rule One: A single point outside the control limits
Rule Two: A run of 8 or more points in a row above (or below)
the centerline
Rule Three: Six consecutive points increasing or decreasing
trend.
Rule Four: Two out of three consecutive points near a control
limit.
5. Rule Five: Fifteen consecutive points close to the centerline.

17. Mistakes
Mistake 1: To react to an outcome as if it came from a special
cause, when actually it came from common causes of variation
Mistake 2: To treat an outcome as if it came from common
cause of variation, when it actually it came from a special
cause.

18. Interpreting Control Charts
Manager concludes,
“Tough management gets results!”
Award given
UCL 60 50 40 30
LCL FW
Manager regrets giving award
No more Mr. Nice Guy!
Is the manager responding appropriately to the control chart?

19. Interpreting Control Charts – Case Study 1
The Production Manager for a marketing department had a budget of $198,000 per month for the production of sales materials, proposals, and other marketing collateral. Her budget was based on annual expenses over the previous year. This last month, her budget report showed expenses of $205,000. She circled the number on the report, and sent it out to her direct reports with the note, “Why are our expenses going up? Come to our next staff meeting ready to explain why expenses are up, and how you will control them for the rest of the year.”
Individuals Chart Of Production Expenses
January 1997 To December 1998
a. Should last month’s expenses be of special concern to the Manager? Why?
b. Did the Production Manager take the appropriate type of action?
c. What should she expect her monthly expenses to be?

20. Interpreting Control Charts-Case Study 2
The Call Center keeps track of calls that come in over its telephone line. It wants to use this data to help budget for the new year. If the process is stable, managers can estimate how many calls may be received each day on average. But first they need to know if there are any indications of special causes in the process.
Individuals Chart Of Orders Over Phone Line
January 30 To February 22
a. Does the data indicate the presence of a special cause, or is the variation all the result of common causes?
b. What is the average number of orders they should expect each day?
c. What is the maximum number of orders they should expect to receive each day?

21. Summary The run chart is the fundamental method to evaluate the impact
of changes tested and ultimately the success of improvement
efforts:
Display data to make process performance visible
Determine whether change resulted in improvement
Determine whether you are holding the gain made by the improvement
Create a Shewhart graph by adding probability-based interpretive rules (UCL, mean, LCL) allows us to learn whether our process exhibits a non random signal change.

22. Team Time For one PDSA cycles:
Take your patient specific data & create a run chart
State your hypothesis
Does your data violate any of the rules?
State your conclusion
Design your next PDSA
Share your results with the table next to you
15 minutes

23. Run Chart Examples

24. Phase 1: Nov - Feb Phase 2: Mar Phase 4: May - Current
Baseline Data Collection
Phase 2: Mar
Creation of Sepsis Algorithm
Education Rollout
Creation of Algorithm posters
Phase 3: Apr
Abx placed in Omni cell
Hardwiring Team Discussion of Care Plan
Creation of Order Set
Phase 4: May - Current
Creation of PDSA Tool for Real-time Process Review
n= n= n= n= n= n= n= n=8

25. Super Sirs on Arrival: Door to Antibiotics (Jan – Jun 2012)
Staff Education

26. Super Sirs on Arrival: Door to Antibiotics (Jan – Jun 2012)
Staff Education