© 2003 Prentice-Hall, Inc.Chap 11-1 Analysis of Variance & Post-ANOVA ANALYSIS IE 340/440 PROCESS IMPROVEMENT THROUGH PLANNED EXPERIMENTATION Dr. Xueping Li University of Tennessee

© 2003 Prentice-Hall, Inc. Chap 11-2 What If There Are More Than Two Factor Levels? The t-test does not directly apply There are lots of practical situations where there are either more than two levels of interest, or there are several factors of simultaneous interest The analysis of variance (ANOVA) is the appropriate analysis “engine” for these types of experiments – Chapter 3, textbook The ANOVA was developed by Fisher in the early 1920s, and initially applied to agricultural experiments Used extensively today for industrial experiments

© 2003 Prentice-Hall, Inc. Chap 11-3 Figure 3.1 (p. 61) A single-wafer plasma etching tool.

© 2003 Prentice-Hall, Inc. Chap 11-4 Table 3.1 (p. 62) Etch Rate Data (in Å/min) from the Plasma Etching Experiment)

© 2003 Prentice-Hall, Inc. Chap 11-5 The Analysis of Variance (Sec. 3-3, pg. 65) In general, there will be a levels of the factor, or a treatments, and n replicates of the experiment, run in random order…a completely randomized design (CRD) N = an total runs We consider the fixed effects case…the random effects case will be discussed later Objective is to test hypotheses about the equality of the a treatment means

© 2003 Prentice-Hall, Inc. Chap 11-6 Models for the Data There are several ways to write a model for the data:

© 2003 Prentice-Hall, Inc. Chap 11-7 The Analysis of Variance The name “analysis of variance” stems from a partitioning of the total variability in the response variable into components that are consistent with a model for the experiment The basic single-factor ANOVA model is

© 2003 Prentice-Hall, Inc. Chap 11-8 The Analysis of Variance Total variability is measured by the total sum of squares: The basic ANOVA partitioning is:

© 2003 Prentice-Hall, Inc. Chap 11-9 The Analysis of Variance A large value of SS Treatments reflects large differences in treatment means A small value of SS Treatments likely indicates no differences in treatment means Formal statistical hypotheses are:

Chap The Analysis of Variance While sums of squares cannot be directly compared to test the hypothesis of equal means, mean squares can be compared. A mean square is a sum of squares divided by its degrees of freedom: If the treatment means are equal, the treatment and error mean squares will be (theoretically) equal. If treatment means differ, the treatment mean square will be larger than the error mean square.

© 2003 Prentice-Hall, Inc. Chap The Analysis of Variance is Summarized in a Table Computing…see text, pp 70 – 73 The reference distribution for F 0 is the F a-1, a(n-1) distribution Reject the null hypothesis (equal treatment means) if

© 2003 Prentice-Hall, Inc. Chap Features of One-Way ANOVA F Statistic The F Statistic is the Ratio of the Among Estimate of Variance and the Within Estimate of Variance The ratio must always be positive df 1 = a -1 will typically be small df 2 = N - c will typically be large The Ratio Should Be Close to 1 if the Null is True

© 2003 Prentice-Hall, Inc. Chap Features of One-Way ANOVA F Statistic If the Null Hypothesis is False The numerator should be greater than the denominator The ratio should be larger than 1 (continued)

© 2003 Prentice-Hall, Inc. Chap The Reference Distribution:

© 2003 Prentice-Hall, Inc. Chap Table 3.1 (p. 62) Etch Rate Data (in Å/min) from the Plasma Etching Experiment)

© 2003 Prentice-Hall, Inc. Chap Table 3.4 (p. 71) ANOVA for the Plasma Etching Experiment

© 2003 Prentice-Hall, Inc. Chap Table 3.5 (p. 72) Coded Etch Rate Data for Example 3.2 Coding the observations More about manual calculation p.70-71

Chap Graphical View of the Results

© 2003 Prentice-Hall, Inc. Chap Model Adequacy Checking in the ANOVA Text reference, Section 3-4, pg. 76 Checking assumptions is important Normality Constant variance Independence Have we fit the right model? Later we will talk about what to do if some of these assumptions are violated

© 2003 Prentice-Hall, Inc. Chap Model Adequacy Checking in the ANOVA Examination of residuals (see text, Sec. 3-4, pg. 76) Design-Expert generates the residuals Residual plots are very useful Normal probability plot of residuals

© 2003 Prentice-Hall, Inc. Chap Table 3.6 (p. 76) Etch Rate Data and Residuals from Example 3.1 a.

© 2003 Prentice-Hall, Inc. Chap Figure 3.4 (p. 77) Normal probability plot of residuals for Example 3-1.

© 2003 Prentice-Hall, Inc. Chap Figure 3.5 (p. 78) Plot of residuals versus run order or time.

© 2003 Prentice-Hall, Inc. Chap Figure 3.6 (p. 79) Plot of residuals versus fitted values.

© 2003 Prentice-Hall, Inc. Chap Other Important Residual Plots

© 2003 Prentice-Hall, Inc. Chap Post-ANOVA Comparison of Means The analysis of variance tests the hypothesis of equal treatment means Assume that residual analysis is satisfactory If that hypothesis is rejected, we don’t know which specific means are different Determining which specific means differ following an ANOVA is called the multiple comparisons problem There are lots of ways to do this…see text, Section 3-5, pg. 86 We will use pairwise t-tests on means…sometimes called Fisher’s Least Significant Difference (or Fisher’s LSD) Method

© 2003 Prentice-Hall, Inc. Chap Tukey’s Test H0: Mu_i = Mu_j ; H1: Mu_i <> Mu_j T statistic Whether Where f is the DF of MSE a is the number of groups

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure Tells which Population Means are Significantly Different E.g.,  1 =  2   3 2 groups whose means may be significantly different Post Hoc (A Posteriori) Procedure Done after rejection of equal means in ANOVA Pairwise Comparisons Compare absolute mean differences with critical range X f(X)  1 =  2  3

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure: Example 1. Compute absolute mean differences: Machine1 Machine2 Machine Compute critical range: 3. All of the absolute mean differences are greater than the critical range. There is a significant difference between each pair of means at the 5% level of significance.

© 2003 Prentice-Hall, Inc. Chap Fisher’s LSD H0: Mu_i = Mu_j Least Significant Difference Whether Where

© 2003 Prentice-Hall, Inc. Chap Design-Expert Output Treatment Means (Adjusted, If Necessary) EstimatedStandard MeanError MeanStandardt for H0 TreatmentDifferenceDFErrorCoeff=0Prob > |t| 1 vs vs vs < vs vs vs vs vs vs vs <

© 2003 Prentice-Hall, Inc. Chap Figure 3.12 (p. 99) Design-Expert computer output for Example 3-1.

© 2003 Prentice-Hall, Inc. Chap Figure 3.13 (p. 100) Minitab computer output for Example 3-1.

© 2003 Prentice-Hall, Inc. Chap 11-34

© 2003 Prentice-Hall, Inc. Chap Graphical Comparison of Means Text, pg. 89

© 2003 Prentice-Hall, Inc. Chap For the Case of Quantitative Factors, a Regression Model is often Useful Response:Strength ANOVA for Response Surface Cubic Model Analysis of variance table [Partial sum of squares] Sum ofMeanF SourceSquaresDFSquareValue Prob > F Model < A A < A Residual Lack of Fit Pure Error Cor Total CoefficientStandard 95% CI 95% CI FactorEstimateDFErrorLowHighVIF Intercept A-Cotton % A A

Chap The Regression Model Final Equation in Terms of Actual Factors: Strength = * Cotton Weight % * Cotton Weight %^ E- 003 * Cotton Weight %^3 This is an empirical model of the experimental results

© 2003 Prentice-Hall, Inc. Chap Figure 3.7 (p. 83) Plot of residuals versus ŷ ij for Example 3-5.

© 2003 Prentice-Hall, Inc. Chap Table 3.9 (p. 83) Variance-Stabilizing Transformations

© 2003 Prentice-Hall, Inc. Chap Figure 3.8 (p. 84) Plot of log S i versus log for the peak discharge data from Example 3.5.

© 2003 Prentice-Hall, Inc. Chap Figure 3.12 (p. 99) Design-Expert computer output for Example 3-1.

© 2003 Prentice-Hall, Inc. Chap Figure 3.13 (p. 100) Minitab computer output for Example 3-1.

© 2003 Prentice-Hall, Inc. Chap Display on page 103

© 2003 Prentice-Hall, Inc. Chap Example 3-1 p70 EX3-1 p112

© 2003 Prentice-Hall, Inc. Chap 11-45

© 2003 Prentice-Hall, Inc. Chap One-Way ANOVA F Test Example As production manager, you want to see if 3 filling machines have different mean filling times. You assign 15 similarly trained & experienced workers, 5 per machine, to the machines. At the.05 significance level, is there a difference in mean filling times? Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap One-Way ANOVA Example: Scatter Diagram Time in Seconds Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap One-Way ANOVA Example Computations Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Summary Table Source of Variation Degrees of Freedom Sum of Squares Mean Squares (Variance) F Statistic Among (Factor) 3-1= MSA/MSW =25.60 Within (Error) 15-3= Total15-1=

© 2003 Prentice-Hall, Inc. Chap One-Way ANOVA Example Solution F H 0 :  1 =  2 =  3 H 1 : Not All Equal  =.05 df 1 = 2 df 2 = 12 Critical Value(s): Test Statistic: Decision: Conclusion: Reject at  = There is evidence that at least one  i differs from the rest.  = 0.05 F MSA MSW  

© 2003 Prentice-Hall, Inc. Chap Solution in Excel Use Tools | Data Analysis | ANOVA: Single Factor Excel Worksheet that Performs the One-Factor ANOVA of the Example

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure Tells which Population Means are Significantly Different E.g.,  1 =  2   3 2 groups whose means may be significantly different Post Hoc (A Posteriori) Procedure Done after rejection of equal means in ANOVA Pairwise Comparisons Compare absolute mean differences with critical range X f(X)  1 =  2  3

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure: Example 1. Compute absolute mean differences: Machine1 Machine2 Machine Compute critical range: 3. All of the absolute mean differences are greater than the critical range. There is a significant difference between each pair of means at the 5% level of significance.

© 2003 Prentice-Hall, Inc. Chap Solution in PHStat Use PHStat | c-Sample Tests | Tukey-Kramer Procedure … Excel Worksheet that Performs the Tukey- Kramer Procedure for the Previous Example

© 2003 Prentice-Hall, Inc. Chap Levene’s Test for Homogeneity of Variance The Null Hypothesis The c population variances are all equal The Alternative Hypothesis Not all the c population variances are equal

© 2003 Prentice-Hall, Inc. Chap Levene’s Test for Homogeneity of Variance: Procedure 1.For each observation in each group, obtain the absolute value of the difference between each observation and the median of the group. 2.Perform a one-way analysis of variance on these absolute differences.

© 2003 Prentice-Hall, Inc. Chap Levene’s Test for Homogeneity of Variances: Example As production manager, you want to see if 3 filling machines have different variance in filling times. You assign 15 similarly trained & experienced workers, 5 per machine, to the machines. At the.05 significance level, is there a difference in the variance in filling times? Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Levene’s Test: Absolute Difference from the Median

© 2003 Prentice-Hall, Inc. Chap Summary Table

© 2003 Prentice-Hall, Inc. Chap Levene’s Test Example: Solution F H 0 : H 1 : Not All Equal  =.05 df 1 = 2 df 2 = 12 Critical Value(s): Test Statistic: Decision: Conclusion: Do not reject at  = There is no evidence that at least one differs from the rest.  = 0.05

© 2003 Prentice-Hall, Inc. Chap Randomized Blocked Design Items are Divided into Blocks Individual items in different samples are matched, or repeated measurements are taken Reduced within group variation (i.e., remove the effect of block before testing) Response of Each Treatment Group is Obtained Assumptions Same as completely randomized design No interaction effect between treatments and blocks

© 2003 Prentice-Hall, Inc. Chap Randomized Blocked Design (Example)          Factor (Training Method) Factor Levels (Groups) Blocked Experiment Units Dependent Variable (Response) 21 hrs17 hrs31 hrs 27 hrs25 hrs28 hrs 29 hrs20 hrs22 hrs

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design (Partition of Total Variation) Variation Due to Group SSA Variation Among Blocks SSBL Variation Among All Observations SST Commonly referred to as:  Sum of Squares Error  Sum of Squares Unexplained Commonly referred to as:  Sum of Squares Among  Among Groups Variation = + + Variation Due to Random Sampling SSW Commonly referred to as:  Sum of Squares Among Block

© 2003 Prentice-Hall, Inc. Chap Total Variation

© 2003 Prentice-Hall, Inc. Chap Among-Group Variation

© 2003 Prentice-Hall, Inc. Chap Among-Block Variation

© 2003 Prentice-Hall, Inc. Chap Random Error

© 2003 Prentice-Hall, Inc. Chap Randomized Block F Test for Differences in c Means No treatment effect Test Statistic Degrees of Freedom 0  Reject

© 2003 Prentice-Hall, Inc. Chap Summary Table Source of Variation Degrees of Freedom Sum of Squares Mean Squares F Statistic Among Group c – 1SSA MSA = SSA/(c – 1) MSA/ MSE Among Block r – 1SSBL MSBL = SSBL/(r – 1) MSBL/ MSE Error (r – 1)  c – 1) SSE MSE = SSE/[(r – 1)  (c– 1)] Total rc – 1SST

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design: Example As production manager, you want to see if 3 filling machines have different mean filling times. You assign 15 workers with varied experience into 5 groups of 3 based on similarity of their experience, and assigned each group of 3 workers with similar experience to the machines. At the.05 significance level, is there a difference in mean filling times? Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design Example Computation Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design Example: Summary Table Source of Variation Degrees of Freedom Sum of Squares Mean Squares F Statistic Among Group 2 SSA= MSA = / = Among Block 4 SSBL= MSBL = / =.6039 Error  SSE= MSE = Total 14 SST=

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design Example: Solution F H 0 :  1 =  2 =  3 H 1 : Not All Equal  =.05 df 1 = 2 df 2 = 8 Critical Value(s): Test Statistic: Decision: Conclusion: Reject at  = There is evidence that at least one  i differs from the rest.  = 0.05 F MSA MSE  

© 2003 Prentice-Hall, Inc. Chap Randomized Block Design in Excel Tools | Data Analysis | ANOVA: Two Factor Without Replication Example Solution in Excel Spreadsheet

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure Similar to the Tukey-Kramer Procedure for the Completely Randomized Design Case Critical Range

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure: Example 1. Compute absolute mean differences: Machine1 Machine2 Machine Compute critical range: 3. All of the absolute mean differences are greater. There is a significance difference between each pair of means at 5% level of significance.

© 2003 Prentice-Hall, Inc. Chap The Tukey-Kramer Procedure in PHStat PHStat | c-Sample Tests | Tukey-Kramer Procedure … Example in Excel Spreadsheet

© 2003 Prentice-Hall, Inc. Chap Two-Way ANOVA Examines the Effect of: Two factors on the dependent variable E.g., Percent carbonation and line speed on soft drink bottling process Interaction between the different levels of these two factors E.g., Does the effect of one particular percentage of carbonation depend on which level the line speed is set?

© 2003 Prentice-Hall, Inc. Chap Two-Way ANOVA Assumptions Normality Populations are normally distributed Homogeneity of Variance Populations have equal variances Independence of Errors Independent random samples are drawn (continued)

© 2003 Prentice-Hall, Inc. Chap SSE Two-Way ANOVA Total Variation Partitioning Variation Due to Factor A Variation Due to Random Sampling Variation Due to Interaction SSA SSAB SST Variation Due to Factor B SSB Total Variation d.f.= n-1 d.f.= r-1 = + + d.f.= c-1 + d.f.= (r-1)(c-1) d.f.= rc(n’-1)

© 2003 Prentice-Hall, Inc. Chap Two-Way ANOVA Total Variation Partitioning

© 2003 Prentice-Hall, Inc. Chap Total Variation

© 2003 Prentice-Hall, Inc. Chap Factor A Variation Sum of Squares Due to Factor A = the difference among the various levels of factor A and the grand mean

© 2003 Prentice-Hall, Inc. Chap Factor B Variation Sum of Squares Due to Factor B = the difference among the various levels of factor B and the grand mean

© 2003 Prentice-Hall, Inc. Chap Interaction Variation Sum of Squares Due to Interaction between A and B = the effect of the combinations of factor A and factor B

© 2003 Prentice-Hall, Inc. Chap Random Error Sum of Squares Error = the differences among the observations within each cell and the corresponding cell means

© 2003 Prentice-Hall, Inc. Chap Two-Way ANOVA: The F Test Statistic F Test for Factor B Main Effect F Test for Interaction Effect H 0 :  1. =  2. = =  r. H 1 : Not all  i. are equal H 0 :  ij = 0 (for all i and j) H 1 :  ij  0 H 0 :    1 = . 2 = =   c H 1 : Not all . j are equal Reject if F > F U F Test for Factor A Main Effect

© 2003 Prentice-Hall, Inc. Chap Two-Way ANOVA Summary Table Source of Variation Degrees of Freedom Sum of Squares Mean Squares F Statistic Factor A (Row) r – 1SSA MSA = SSA/(r – 1) MSA/ MSE Factor B (Column) c – 1SSB MSB = SSB/(c – 1) MSB/ MSE AB (Interaction) (r – 1)(c – 1)SSAB MSAB = SSAB/ [(r – 1)(c – 1)] MSAB/ MSE Error r  c  n ’ – 1) SSE MSE = SSE/[r  c  n ’ – 1)] Total r  c  n ’ – 1 SST

© 2003 Prentice-Hall, Inc. Chap Features of Two-Way ANOVA F Test Degrees of Freedom Always Add Up rcn’-1=rc(n’-1)+(c-1)+(r-1)+(c-1)(r-1) Total=Error+Column+Row+Interaction The Denominator of the F Test is Always the Same but the Numerator is Different The Sums of Squares Always Add Up Total=Error+Column+Row+Interaction

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Rank Test for c Medians Extension of Wilcoxon Rank Sum Test Tests the equality of more than 2 (c) population medians Distribution-Free Test Procedure Used to Analyze Completely Randomized Experimental Designs Use  2 Distribution to Approximate if Each Sample Group Size n j > 5 df = c – 1

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Rank Test Assumptions Independent random samples are drawn Continuous dependent variable Data may be ranked both within and among samples Populations have same variability Populations have same shape Robust with Regard to Last 2 Conditions Use F test in completely randomized designs and when the more stringent assumptions hold

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Rank Test Procedure Obtain Ranks In event of tie, each of the tied values gets their average rank Add the Ranks for Data from Each of the c Groups Square to obtain T j 2

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Rank Test Procedure Compute Test Statistic # of observation in j –th sample H may be approximated by chi-square distribution with df = c –1 when each n j >5 (continued)

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Rank Test Procedure Critical Value for a Given  Upper tail Decision Rule Reject H 0 : M 1 = M 2 = = M c if test statistic H > Otherwise, do not reject H 0 (continued)

© 2003 Prentice-Hall, Inc. Chap Machine1 Machine2 Machine Kruskal-Wallis Rank Test: Example As production manager, you want to see if 3 filling machines have different median filling times. You assign 15 similarly trained & experienced workers, 5 per machine, to the machines. At the.05 significance level, is there a difference in median filling times?

© 2003 Prentice-Hall, Inc. Chap Machine1 Machine2 Machine Example Solution: Step 1 Obtaining a Ranking Raw DataRanks Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Example Solution: Step 2 Test Statistic Computation

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Test Example Solution H 0 : M 1 = M 2 = M 3 H 1 : Not all equal  =.05 df = c - 1 = = 2 Critical Value(s): Reject at Test Statistic: Decision: Conclusion: There is evidence that population medians are not all equal.  =.05  =.05. H = 11.58

© 2003 Prentice-Hall, Inc. Chap Kruskal-Wallis Test in PHStat PHStat | c-Sample Tests | Kruskal-Wallis Rank Sum Test … Example Solution in Excel Spreadsheet

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test for Differences in c Medians Tests the equality of more than 2 (c) population medians Distribution-Free Test Procedure Used to Analyze Randomized Block Experimental Designs Use  2 Distribution to Approximate if the Number of Blocks r > 5 df = c – 1

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test Assumptions The r blocks are independent The random variable is continuous The data constitute at least an ordinal scale of measurement No interaction between the r blocks and the c treatment levels The c populations have the same variability The c populations have the same shape

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test: Procedure  Replace the c observations by their ranks in each of the r blocks; assign average rank for ties  Test statistic:  R.j 2 is the square of the rank total for group j  F R can be approximated by a chi-square distribution with (c –1) degrees of freedom  The rejection region is in the right tail

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test: Example As production manager, you want to see if 3 filling machines have different median filling times. You assign 15 workers with varied experience into 5 groups of 3 based on similarity of their experience, and assigned each group of 3 workers with similar experience to the machines. At the.05 significance level, is there a difference in median filling times? Machine1 Machine2 Machine

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test: Computation Table

© 2003 Prentice-Hall, Inc. Chap Friedman Rank Test Example Solution H 0 : M 1 = M 2 = M 3 H 1 : Not all equal  =.05 df = c - 1 = = 2 Critical Value: Reject at Test Statistic: Decision: Conclusion: There is evidence that population medians are not all equal.  =.05 F R = 8.4

© 2003 Prentice-Hall, Inc. Chap Chapter Summary Described the Completely Randomized Design: One-Way Analysis of Variance ANOVA Assumptions F Test for Difference in c Means The Tukey-Kramer Procedure Levene’s Test for Homogeneity of Variance Discussed the Randomized Block Design F Test for the Difference in c Means The Tukey Procedure

© 2003 Prentice-Hall, Inc. Chap Chapter Summary Described the Factorial Design: Two-Way Analysis of Variance Examine effects of factors and interaction Discussed Kruskal-Wallis Rank Test for Differences in c Medians Illustrated Friedman Rank Test for Differences in c Medians (continued)