1. Process Improvement in Healthcare: Volunteer Clinic Case Study Nonparametric Statistics ISE 491 Fall 2009 Dr. Joan Burtner Associate Professor, Department of Industrial Engineering and Industrial Management

2. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 2 Case Study Description Volunteer Clinic Utilization Study Methods  Prediction of Future Demand (Forecasting)  Interviews with Key Clinical Personnel  On-site Observations (Time Studies)  Review of Policies and Procedures  Process Mapping  Retrospective Data Analysis

3. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 3 Does Volunteerism Vary By Month?  Factor: Month (3 levels, A B C)  Sample: Random selection of 9 physicians per month  Balanced design (3x9=27 observations)  Outcome: Patient contact hours  Interval level data  With an assumption that the underlying distribution for each month is normal, the appropriate hypothesis test is a one-way ANOVA  Statistical package: Minitab 14 or 15

4. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 4 Raw Data and Minitab Format Demo Number Month 70A 30A 26A 60A 34A 26A 57A 39A 44A 53B 39B 27B 29B 23B 28B 25B 23B 22B 36C 23C 29C 34C 16C 21C 23C 25C 20C MonthA 70 30 26 60 34 26 57 39 44 MonthB 53 39 27 29 23 28 25 23 22 MonthC 36 23 29 34 16 21 23 25 20

5. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 5 One Way ANOVA - Stacked One-way ANOVA: DemoNumber versus DemoMonth Source DF SS MS F P DemoMonth 2 1509 754 5.63 0.010 Error 24 3217 134 Total 26 4726 S = 11.58 R-Sq = 31.92% R-Sq(adj) = 26.25% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev ---+---------+---------+---------+------ A 9 42.89 16.04 (-------*-------) B 9 29.89 10.07 (-------*-------) C 9 25.22 6.59 (-------*-------) ---+---------+---------+---------+------ 20 30 40 50 Pooled StDev = 11.58

6. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 6 Simultaneous Confidence Intervals Tukey 95% Simultaneous Confidence Intervals All Pairwise Comparisons among Levels of DemoMonth Individual confidence level = 98.02% DemoMonth = A subtracted from: DemoMonth Lower Center Upper -+---------+---------+---------+-------- B -26.62 -13.00 0.62 (--------*--------) C -31.29 -17.67 -4.04 (--------*--------) -+---------+---------+---------+-------- -30 -15 0 15 DemoMonth = B subtracted from: DemoMonth Lower Center Upper -+---------+---------+---------+-------- C -18.29 -4.67 8.96 (--------*--------) -+---------+---------+---------+-------- -30 -15 0 15

7. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 7 Initial interpretation Null hypothesis: The mean number of patient contact hours (in the population) is the same for each month Alternate hypothesis: The mean number of patient hours (in the population) differs for at least one month Assumed significance level = 0.05 P-value reported = 0.010 Decision: Reject null hypothesis Conclusion: Physician volunteer hours vary by month

8. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 8 Interpretation of Tukey CIs The means for Month A and B are not significantly different The means for Month A and C are significantly different; Month A mean contact hours is significantly greater than Month C mean contact hours The means for Month B and C are not significantly different

9. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 9 Validation of Assumptions

10. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 10 Does Volunteerism Vary By Month? Part 2  Outcome: Total number of patient contact hours  Factor: Month (3 levels, A B C)  Random selection of 9 physicians per month  Balanced design  Interval level data  No assumption that the underlying distribution for each month is normal  Appropriate analysis is a Kruskal-Wallis or Mood Median Test  Statistical package: Minitab 14 or 15

11. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 11 Mood Median Results Mood Median Test: DemoNumber versus DemoMonth Mood median test for DemoNumber Chi-Square = 4.75 DF = 2 P = 0.093 Individual 95.0% CIs DemoMonth N Median Q3-Q1 ---+---------+---------+---------+--- A 2 7 39.0 30.5 (----------*---------------) B 6 3 27.0 11.0 (---*-------) C 6 3 23.0 11.0 (-*-------) ---+---------+---------+---------+--- 24 36 48 60 Overall median = 28.0

12. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 12 Source: Minitab Help Guide – Mood’s Median Test Stat > Nonparametrics > Mood's Median Test Mood's median test can be used to test the equality of medians from two or more populations and, like the Kruskal-Wallis Test, provides an nonparametric alternative to the one-way analysis of variance. Mood's median test is sometimes called a median test or sign scores test. Mood's median test tests: H0: the population medians are all equal versus H1: the medians are not all equal An assumption of Mood's median test is that the data from each population are independent random samples and the population distributions have the same shape. Mood's median test is robust against outliers and errors in data and is particularly appropriate in the preliminary stages of analysis. Mood's median test is more robust than is the Kruskal-Wallis test against outliers, but is less powerful for data from many distributions, including the normal.Kruskal-Wallis

13. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 13 Kruskal – Wallis Results Kruskal-Wallis Test: DemoNumber versus DemoMonth Kruskal-Wallis Test on DemoNumber DemoMonth N Median Ave Rank Z A 9 39.00 20.1 2.83 B 9 27.00 12.7 -0.59 C 9 23.00 9.2 -2.24 Overall 27 14.0 H = 8.91 DF = 2 P = 0.012 H = 8.95 DF = 2 P = 0.011 (adjusted for ties)

14. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 14 Source: Minitab Help Guide – Kruskal-Wallis You can perform a Kruskal-Wallis test of the equality of medians for two or more populations. This test is a generalization of the procedure used by the Mann- Whitney test and, like Mood's Median test, offers a nonparametric alternative to the one-way analysis of variance. The Kruskal-Wallis hypotheses are:Mann- Whitney H0: the population medians are all equal versus H1: the medians are not all equal An assumption for this test is that the samples from the different populations are independent random samples from continuous distributions, with the distributions having the same shape. The Kruskal-Wallis test is more powerful than Mood's median test for data from many distributions, including data from the normal distribution, but is less robust against outliers.

15. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 15 Modified Design and Analysis The experimenter did not assume that the number of volunteer hours follows a normal distribution The experimenter collected data for the same nine physicians for three different 31-day months Since the design includes three groups (months) blocked by physicians, the appropriate hypothesis test would be the Friedman Response: Load (Hours) Treatment: Month31 Blocks: Physician LoadMonth31Physician 70JanuaryAllen 30JanuaryBrown 26JanuaryCook 60JanuaryDodd 34JanuaryEllis 26JanuaryFrank 57JanuaryGrey 39JanuaryHoward 44JanuaryIngle 53MayAllen 39MayBrown 27MayCook 29MayDodd 23MayEllis 28MayFrank 25MayGrey 23MayHoward 22MayIngle 36JulyAllen 23JulyBrown 29JulyCook 34JulyDodd 16JulyEllis 21JulyFrank 23JulyGrey 25JulyHoward 20JulyIngle

16. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 16 Source: Minitab Help Guide – Friedman Test Stat > Nonparametrics > Friedman Friedman test is a nonparametric analysis of a randomized block experiment, and thus provides an alternative to the Two-way analysis of variance. The hypotheses are:Two-way H0: all treatment effects are zero versus H1: not all treatment effects are zero Randomized block experiments are a generalization of paired experiments, and the Friedman test is a generalization of the paired sign test.

17. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 17 Friedman Test Results Friedman Test: Load versus Month31 blocked by Physician S = 5.56 DF = 2 P = 0.062 Sum Est of Month31 N Median Ranks January 9 41.00 23.0 July 9 23.00 13.0 May 9 25.00 18.0 Grand median = 29.67

18. Fall 2009ISE 491 Dr. Burtner ~ Clinic Case Study Slide 18 Interpretation of Friedman P = 0.062 For an assumed alpha level of 0.05, there is insufficient evidence to reject H0 because the p-value is greater than the alpha level. Therefore we conclude that the data do not support the hypothesis that any of the treatment effects are different from zero. Physician volunteerism, in terms of patient contact hours, does not vary significantly by month.