1. Understanding Variation in Patient Satisfaction

2. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 2.PPT Variation in Patient Satisfaction Associated With Nursing Purpose: To use appropriate graphical and statistical techniques to identify and quantify the critical factors associated with the variation in overall patient satisfaction. Y = f(X) Patient Satisfaction associated with Nursing =f(??????) Step 1 - Define the Practical Problem Step 2 - Translate to a Statistical Problem Step 3 - Solve the Statistical Problem Step 4 - Translate back to the Practical Problem

3. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 3.PPT Step 1: Define the Practical Problem The hospital has made significant improvements in patient satisfaction over the past few years, but achieving the system goal of 85% excellent ratings has been difficult to achieve and maintain. Anecdotal evidence abounds concerning reasons for variation in scores, but will these anecdotes be supported by the data ?

4. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 4.PPT Step 2: Translate to a Statistical Problem Y – Overall Patient Satisfaction with Nursing Is a function of: X1 – Agency Usage X2 – Overtime Usage X3 – Nursing Management Changes X4 – RN Turnover Rate X5 – Staffing Hours Variance X6 – Skill Mix Percentage X7 – Associate Satisfaction RN

5. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 5.PPT Step 3: Solve the Statistical Problem First, look at aggregate data for 2004: Patient Satisfaction with Nursing 0.650.690.730.770.810.85 95% Confidence Interval for Mu 0.7680.7780.7880.7980.8080.818 95% Confidence Interval for Median Variable: Pt Sat A-Squared: P-Value: Mean StDev Variance Skewness Kurtosis N Minimum 1st Quartile Median 3rd Quartile Maximum 0.770418 0.043367 0.773366 0.512 0.185 0.786605 0.052595 2.77E-03 -7.7E-01 0.383749 43 0.645000 0.759000 0.787000 0.827000 0.870000 0.802791 0.066849 0.813317 Anderson-Darling Normality Test 95% Confidence Interval for Mu 95% Confidence Interval for Sigma 95% Confidence Interval for Median Descriptive Statistics

6. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 6.PPT Step 3: Solve the Statistical Problem (continued) Overall Satisfaction Scores are in statistical control—however, this is not surprising since aggregated data is being used—it hides the true amount of variation.

7. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 7.PPT Step 3: Solve the Statistical Problem (continued) 2004 Overall Agency Usage % by Nursing Unit 0.160.120.080.040.00 95% Confidence Interval for Mu 0.060.050.040.030.02 95% Confidence Interval for Median Variable: Agency 0.021683 0.035100 0.031318 Maximum 3rd Quartile Median 1st Quartile Minimum N Kurtosis Skewness Variance StDev Mean P-Value: A-Squared: 0.054634 0.054106 0.057520 0.159000 0.069000 0.035000 0.005000 0.000000 43 0.384133 0.971670 1.81E-03 4.26E-02 4.44E-02 0.002 1.279 95% Confidence Interval for Median 95% Confidence Interval for Sigma 95% Confidence Interval for Mu Anderson-Darling Normality Test Descriptive Statistics

8. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 8.PPT Step 3: Solve the Statistical Problem (continued) Overall Agency Usage has one unit (7B at University) which falls outside the Upper Control Limit—however, 1 out of 43 units being out of control is no cause for alarm.

9. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 9.PPT Step 3: Solve the Statistical Problem (continued) Correlation between Satisfaction Scores (Y) and Agency Usage (X1) P value =0.011 therefore it can be concluded with > 95% confidence that a statistically significant correlation exists. In fact, 14.6% of the variation in Pt. Sat. is explained by variation in Agency Usage. 0.150.100.050.00 0.85 0.75 0.65 Agency P t S a t S = 0.0491794 R-Sq = 14.6 % R-Sq(adj) = 12.6 % Pt Sat = 0.807609 - 0.472877 Agency 95% CI Regression Regression Plot

10. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 10.PPT Step 3: Solve the Statistical Problem (continued) The following variables did not have a statistically significant correlation to Patient Satisfaction scores using 2004 aggregate data: X 2 – Overtime p-value = 0.121 X 6 – Skill mix p-value = 0.216 X 7 – AFS RN p-value = 0.879

11. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 11.PPT Step 3: Solve the Statistical Problem (continued) 2004 Monthly Satisfaction Scores by Nursing Unit 1 1 T h o m a s 1 3 T h o m a s 2 & 4 J 2 T h o m a s 2 E 2 n d 3 r d 3 S 4 t h 4 W A 5 A 5 B 5 S 5 t h 5 W 6 T h o m a s 6 A 6 B 6 C 6 W 7 A 7 B 7 C 7 W 8 T h o m a s 8 A 8 B 9 A 9 B C V S D E D F B U F - E D L B - G T M E C H O B S C U S N F T C U 0.4 0.5 0.6 0.7 0.8 0.9 1.0 C2 S e r v i c e E x c e l l e n c e Boxplots of Service by C 2 (means are indicated by solid circles)

12. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 12.PPT Step 3: Solve the Statistical Problem (continued) Histogram of Monthly Variation in Satisfaction Scores Conclusion: 72% of the time nursing units fail to meet the 85% satisfaction goal Process Capability for Service Excellence

13. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 13.PPT One-way ANOVA: Service Excellence versus Agency Usage < 5% Analysis of Variance for Service Source DF SS MS F P Agency U 1 0.17457 0.17457 21.88 0.000 Error 481 3.83771 0.00798 Total 482 4.01227 Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev ------+---------+---------+--- ------+ n 164 0.76183 0.09767 (------*------) y 319 0.80197 0.08472 (----*----) ------+---------+---------+---------+ Pooled StDev = 0.08932 0.760 0.780 0.800 0.820 Step 3: Solve the Statistical Problem (continued) Conclusion: Nursing Units that use 5% Agency or less have a statistically significantly higher average satisfaction score than units that use greater than 5% agency.

14. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 14.PPT Step 3: Solve the Statistical Problem (continued) Result of Correlation/Regression using monthly data for 2004 X1 – Agency Usage p value = 0.000 X2 – Overtime Usage p value = 0.000 X3 – Nursing Management Changes X4 – RN Turnover Rate p value = 0.545 X5 – Staffing Hours Variance p value = 0.117 X6 – Skill Mix Percentage p value = 0.006 Conclusion: P values less than 0.05 are statistically significant to a > than 95% Confidence Level. However, while statistically significant these variable explain only a small fraction of the variation in Patient Satisfaction.

15. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 15.PPT The regression equation is Service Excellence = 0.801 - 0.241 Agency - 0.583 OT + 0.0380 Skill Mix + 0.145 Depart Rate RN_1 - 0.00210 HPPD Predictor Coef SE Coef T P Constant 0.80123 0.01828 43.84 0.000 Agency -0.24136 0.08420 -2.87 0.004 OT -0.5833 0.1638 -3.56 0.000 Skill Mi 0.03800 0.02600 1.46 0.145 Depart R 0.14461 0.08675 1.67 0.096 HPPD -0.002103 0.001086 -1.94 0.053 S = 0.08831 R-Sq = 7.3% R-Sq(adj) = 6.3% Best Subsets Regression: Service Excellence versus Agency, OT,... S D H k e P A i p P g l a D e l r n t V c O M a Vars R-Sq R-Sq(adj) y T i R r 1 3.8 3.6 X 1 3.1 2.9 X 2 5.6 5.2 X X 2 4.3 3.9 X X 3 6.3 5.7 X X X 3 6.1 5.5 X X X 4 6.9 6.1 X X X X 4 6.7 6.0 X X X X 5 7.3 6.3 X X X X X Conclusion: Putting all the Factors (X’s) into the model only explains 7.3% of the total variation in satisfaction Step 3: Solve the Statistical Problem (continued)

16. All Rights Reserved, Juran Institute, Inc. Understanding Variation in Patient Satisfaction 16.PPT Step 4: Translate Back to the Practical Problem What actions, if any, do you take knowing this information?