1. Design and Analysis of Experiments
Dr. Tai-Yue Wang
Department of Industrial and Information Management
National Cheng Kung University
Tainan, TAIWAN, ROC
This is a basic course blah, blah, blah…

2. Analysis of Variance Dr. Tai-Yue Wang
Department of Industrial and Information Management
National Cheng Kung University
Tainan, TAIWAN, ROC
This is a basic course blah, blah, blah…

3. Outline(1/2) Example The ANOVA Analysis of Fixed effects Model
Model adequacy Checking
Practical Interpretation of results
Determining Sample Size

4. Outline (2/2) Discovering Dispersion Effects
Nonparametric Methods in the ANOVA

5. What If There Are More Than Two Factor Levels?
The t-test does not directly apply
There are lots of practical situations where there are either more than two levels of interest, or there are several factors of simultaneous interest
The ANalysis Of VAriance (ANOVA) is the appropriate analysis “engine” for these types of experiments
The ANOVA was developed by Fisher in the early 1920s, and initially applied to agricultural experiments
Used extensively today for industrial experiments

6. An Example(1/6)

7. An Example(2/6)
An engineer is interested in investigating the relationship between the RF power setting and the etch rate for this tool. The objective of an experiment like this is to model the relationship between etch rate and RF power, and to specify the power setting that will give a desired target etch rate.
The response variable is etch rate.

8. An Example(3/6)
She is interested in a particular gas (C2F6) and gap (0.80 cm), and wants to test four levels of RF power: 160W, 180W, 200W, and 220W. She decided to test five wafers at each level of RF power.
The experimenter chooses 4 levels of RF power 160W, 180W, 200W, and 220W
The experiment is replicated 5 times – runs made in random order

9. An Example --Data

10. An Example – Data Plot Data: Etch-Rate.mtw
Graph -> Boxplot, Scatterplot

11. An Example--Questions
Does changing the power change the mean etch rate?
Is there an optimum level for power?
We would like to have an objective way to answer these questions
The t-test really doesn’t apply here – more than two factor levels

12. The Analysis of Variance
In general, there will be a levels of the factor, or a treatments, and n replicates of the experiment, run in random order…a completely randomized design (CRD)
N = an total runs
We consider the fixed effects case…the random effects case will be discussed later
Objective is to test hypotheses about the equality of the a treatment means

13. The Analysis of Variance

14. The Analysis of Variance
The name “analysis of variance” stems from a partitioning of the total variability in the response variable into components that are consistent with a model for the experiment

15. The Analysis of Variance
The basic single-factor ANOVA model is

16. Models for the Data
There are several ways to write a model for the data
Mean model
Also known as one-way or single-factor ANOVA

17. Models for the Data Fixed or random factor?
The a treatments could have been specifically chosen by the experimenter. In this case, the results may apply only to the levels considered in the analysis. fixed effect models

18. Models for the Data
The a treatments could be a random sample from a larger population of treatments. In this case, we should be able to extend the conclusion to all treatments in the population.
 random effect models

19. Analysis of the Fixed Effects Model
Recall the single-factor ANOVA for the fixed effect model
Define

20. Analysis of the Fixed Effects Model
Hypothesis 

21. Analysis of the Fixed Effects Model
Thus, the equivalent Hypothesis

22. Analysis of the Fixed Effects Model-Decomposition
Total variability is measured by the total sum of squares:
The basic ANOVA partitioning is:

23. Analysis of the Fixed Effects Model-Decomposition
In detail
(=0)

24. Analysis of the Fixed Effects Model-Decomposition
Thus
SST
SSTreatments
SSE

25. Analysis of the Fixed Effects Model-Decomposition
A large value of SSTreatments reflects large differences in treatment means
A small value of SSTreatments likely indicates no differences in treatment means
Formal statistical hypotheses are:

26. Analysis of the Fixed Effects Model-Decomposition
For SSE
Recall

27. Analysis of the Fixed Effects Model-Decomposition
Combine a sample variances
The above formula is a pooled estimate of the common variance with each a treatment.

28. Analysis of the Fixed Effects Model-Mean Squares
Define
and df df

29. Analysis of the Fixed Effects Model-Mean Squares
By mathematics,
That is, MSE estimates σ2.
If there are no differences in treatments means, MSTreatments also estimates σ2.

30. Analysis of the Fixed Effects Model-Statistical Analysis
Cochran’s Theorem
Let Zi be NID(0,1) for i=1,2,…,v and
then Q1, q2,…,Qs are independent chi-square random variables withv1, v2, …,vs degrees of freedom, respectively, If and only if

31. Analysis of the Fixed Effects Model-Statistical Analysis
Cochran’s Theorem implies that
are independently distributed chi-square random variables
Thus, if the null hypothesis is true, the ratio
is distributed as F with a-1 and N-a degrees of freedom.

32. Analysis of the Fixed Effects Model-Statistical Analysis
Cochran’s Theorem implies that
Are independently distributed chi-square random variables
Thus, if the null hypothesis is true, the ratio
is distributed as F with a-1 and N-a degrees of freedom.

33. Analysis of the Fixed Effects Models-- Summary Table
The reference distribution for F0 is the
Fa-1, a(n-1) distribution
Reject the null hypothesis (equal treatment means) if

34. Analysis of the Fixed Effects Models-- Example
Recall the example of etch rate (Etch-Rate.mtw),
Hypothesis

35. Analysis of the Fixed Effects Models-- Example
ANOVA table

36. Analysis of the Fixed Effects Models-- Example
Rejection region

37. Analysis of the Fixed Effects Models-- Example
P-value
P-value

38. Analysis of the Fixed Effects Models-- Example
Minitab
Data: Etch-Rate.mtw
Stat -> ANOVA -> One-Way
Graphs: Residual plots -> Check four-in-one

39. Analysis of the Fixed Effects Models-- Example
Minitab
One-way ANOVA: Etch Rate versus Power
Method
Null hypothesis All means are equal
Alternative hypothesis At least one mean is different
Significance level α = 0.05
Equal variances were assumed for the analysis.
Factor Information
Factor Levels Values
Power , 180, 200, 220
Analysis of Variance
Source DF Adj SS Adj MS F-Value P-Value
Power
Error
Total
Model Summary
S R-sq R-sq(adj) R-sq(pred)
% % %
Means
Power N Mean StDev % CI
(533.88, )
(570.08, )
(608.08, )
(689.68, )
Pooled StDev =

40. Analysis of the Fixed Effects Model-Statistical Analysis
Coding the observations will not change the results
Without the assumption of randomization, the ANOVA F test can be viewed as approximation to the randomization test.

41. Analysis of the Fixed Effects Model- Estimation of the model parameters
Reasonable estimates of the overall mean and the treatment effects for the single-factor model
are given by

42. Analysis of the Fixed Effects Model- Estimation of the model parameters
Confidence interval for μi
Confidence interval for μi - μj

43. Analysis of the Fixed Effects Model- Unbalanced data
For the unbalanced data

44. A little (very little) humor…

45. Model Adequacy Checking
Assumptions on the model
Errors are normally distributed and independently distributed with mean zero and constant but unknown variances σ2
Define residual
where is an estimate of yij

46. Model Adequacy Checking --Normality
Normal probability plot

47. Model Adequacy Checking --Normality
Four-in-one

48. Model Adequacy Checking --Plot of residuals in time sequence
Residuals vs run order

49. Model Adequacy Checking --Residuals vs fitted

50. Model Adequacy Checking --Residuals vs fitted
Defects
Horn shape
Moon type
Test for equal variances
Bartlett’s test

51. Model Adequacy Checking --Residuals vs fitted
Test for equal variances
Bartlett’s test
Stat ->ANOVA -> Test for Equal Variances

52. Model Adequacy Checking --Residuals vs fitted
Test for equal variances
Bartlett’s test
Test for Equal Variances: Etch Rate versus Power
Method
Null hypothesis All variances are equal
Alternative hypothesis At least one variance is different
Significance level α = 0.05
95% Bonferroni Confidence Intervals for Standard Deviations
Power N StDev CI
( , )
( , )
( , )
( , )
Individual confidence level = 98.75%
Tests
Test
Method Statistic P-Value
Multiple comparisons —
Levene

53. Model Adequacy Checking --Residuals vs fitted
Test for equal variances
Bartlett’s test

54. Model Adequacy Checking --Residuals vs fitted
Variance-stabilizing transformation
Deal with non-constant variance
If observations follows Poisson distribution
 square root transformation
If observations follows Lognormal distribution
 Logarithmic transformation

55. Model Adequacy Checking --Residuals vs fitted
Variance-stabilizing transformation
If observations are binominal data
Arcsine transformation
Other transformation  check the relationship among observations and mean.

56. Practical Interpretation of Results – Regression Model
The one-way ANOVA model
is a regression model and is similar to
Stat -> Regression -> regression
-> fit regression model

57. Practical Interpretation of Results – Regression Model
Computer Results
Regression Analysis: Etch Rate versus Power
Analysis of Variance
Source DF Adj SS Adj MS F-Value P-Value
Regression
Power
Error
Lack-of-Fit
Pure Error
Total
Model Summary
S R-sq R-sq(adj) R-sq(pred)
% % %
Coefficients
Term Coef SE Coef T-Value P-Value VIF
Constant
Power
Regression Equation
Etch Rate = Power
Fits and Diagnostics for Unusual Observations
Obs Etch Rate Fit Resid Std Resid R
R Large residual

58. Practical Interpretation of Results – Regression Model
Computer Results

59. Practical Interpretation of Results – Regression Model
Redo regression by using power and power*power as continuous factors

60. The Regression Model

61. Practical Interpretation of Results – Regression Model

62. Practical Interpretation of Results – Comparison of Means
The analysis of variance tests the hypothesis of equal treatment means
Assume that residual analysis is satisfactory
If that hypothesis is rejected, we don’t know which specific means are different
Determining which specific means differ following an ANOVA is called the multiple comparisons problem

63. Practical Interpretation of Results – Comparison of Means
There are lots of ways to do this
We will use pairwised t-tests on means…sometimes called Fisher’s Least Significant Difference (or Fisher’s LSD) Method

64. Practical Interpretation of Results – Comparison of Means
Fisher Pairwise Comparisons
Grouping Information Using the Fisher LSD Method and 95% Confidence
Power N Mean Grouping A B C D
Means that do not share a letter are significantly different.

65. Practical Interpretation of Results – Comparison of Means

66. Practical Interpretation of Results – Graphical Comparison of Means

67. Practical Interpretation of Results – Contrasts
A linear combination of parameters
So the hypothesis becomes

68. Practical Interpretation of Results – Contrasts
Examples

69. Practical Interpretation of Results – Contrasts
Testing
t-test
Contrast average
Contrast Variance
Test statistic

70. Practical Interpretation of Results – Contrasts
Testing
F-test
Test statistic

71. Practical Interpretation of Results – Contrasts
Testing
F-test
Reject hypothesis if

72. Practical Interpretation of Results – Contrasts
Confidence interval

73. Practical Interpretation of Results – Contrasts
Standardized contrast
Standardize the contrasts when more than one contrast is of interest

74. Practical Interpretation of Results – Contrasts
Unequal sample size
Contrast
t-statistic

75. Practical Interpretation of Results – Contrasts
Unequal sample size
Contrast sum of squares

76. Practical Interpretation of Results – Orthogonal Contrasts
Define two contrasts with coefficients {ci} and {di} are orthogonal contrasts if
Unbalanced design

77. Practical Interpretation of Results – Orthogonal Contrasts
Why use orthogonal contrasts ?
 For a treatments, the set of a-1 orthogonal contrasts partition the sum of squares due to treatments into a-1 independent single-degree-of freedom
tests performed on orthogonal contrasts are independent.

78. Practical Interpretation of Results – Orthogonal Contrasts
example

79. Practical Interpretation of Results – Orthogonal Contrasts
Example for contrast

80. Practical Interpretation of Results – Scheffe’s method for comparing all contrasts
Comparing any and all possible contrasts between treatment means
Suppose that a set of m contrasts in the treatment means
of interest have been determined.
The corresponding contrast in the treatment averages is

81. Practical Interpretation of Results – Scheffe’s method for comparing all contrasts
The standard error of this contrast is
The critical value against which should be compared is
If |Cu|>Sα,u , the hypothesis that contrast Γu equals zero is rejected.

82. Practical Interpretation of Results – Scheffe’s method for comparing all contrasts
The simultaneous confidence intervals with type I error α

83. Practical Interpretation of Results – example for Scheffe’s method
Contrast of interest
The numerical values of these contrasts are

84. Practical Interpretation of Results – example for Scheffe’s method
Standard error
One percent critical values are
|C1|>S0.01,1 and |C1|>S0.01,1 , both contrast hypotheses should be rejected.

85. Practical Interpretation of Results – Minitab example
Orthogonal contrasts:
Contrast 1:
1= 2
Contrast 2:
1+ 2= 3+ 4
Contrast 3:
3= 4

86. Practical Interpretation of Results – Minitab example
Coding “power” into 3 orthogonal contrasts in Minitab:
contrast1 contrast2 contrast3
1 1 0
Data Code numeric to numeric

87. Practical Interpretation of Results – example for Scheffe’s method

88. Practical Interpretation of Results – example for Scheffe’s method
Stat Regressionregressionfit regression
Choose contrast1, 2, 3 as factors
Do not choose “power” as factor because “power” has been separated by contrast 1, 2 and 3.

89. Practical Interpretation of Results – example for Scheffe’s method
Regression Analysis: Etch Rate versus conytrast1, contrast2, contrast3 Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Regression conytrast contrast contrast Error Total Model Summary S R-sq R-sq(adj) R-sq(pred) % 91.22% 88.45%

90. Practical Interpretation of Results –comparing pairs of treatment means
Tukey’s test
Fisher’s Least significant Difference (LSD) method
Hsu’s Methods

91. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
One-way ANOVA: Etch Rate versus Power
Source DF SS MS F P
Power
Error
Total
S = R-Sq = 92.61% R-Sq(adj) = 91.22%
Individual 95% CIs For Mean Based on Pooled StDev
Level N Mean StDev
(--*---)
(--*---)
(--*---)
(--*---)
Pooled StDev = 18.27

92. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
Grouping Information Using Tukey Method
Power N Mean Grouping A B C D
Means that do not share a letter are significantly different.
Tukey 95% Simultaneous Confidence Intervals
All Pairwise Comparisons among Levels of Power
Individual confidence level = 98.87%
Power = 160 subtracted from:
Power Lower Center Upper
(---*--)
(--*---)
(---*--)

93. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
Power = 180 subtracted from:
Power Lower Center Upper
(---*--)
(--*--)
Power = 200 subtracted from:
Power Lower Center Upper
(--*--)

94. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
Hsu's MCB (Multiple Comparisons with the Best)
Family error rate = 0.05
Critical value = 2.23
Intervals for level mean minus largest of other level means
Level Lower Center Upper
(---* )
(--* )
(--* )
( *--)

95. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
Grouping Information Using Fisher Method
Power N Mean Grouping A B C D
Means that do not share a letter are significantly different.
Fisher 95% Individual Confidence Intervals
All Pairwise Comparisons among Levels of Power
Simultaneous confidence level = 81.11%
Power = 160 subtracted from:
Power Lower Center Upper
(--*-)
(-*--)
(--*-)

96. Practical Interpretation of Results –comparing pairs of treatment means—Computer output
Power = 180 subtracted from:
Power Lower Center Upper
(--*-)
(-*-)
Power = 200 subtracted from:
Power Lower Center Upper
(-*--)

97. Practical Interpretation of Results –comparing treatment means with a control
Donntt’s method
Control– the one to be compared
Totally a-1 comparisons
Dunnett's comparisons with a control
Family error rate = 0.05
Individual error rate =
Critical value = 2.59
Control = level (220) of Power
Intervals for treatment mean minus control mean
Level Lower Center Upper
( * )
( * )
( * )

98. Determining Sample size -- Minitab
Stat  Power and sample size
One-way ANOVA
Number of level ->4
Sample size ->
Max. difference  75
Power value  0.9
SD  25
One-way ANOVA
Alpha = Assumed standard deviation = 25
Factors: 1 Number of levels: 4
Maximum Sample Target
Difference Size Power Actual Power
The sample size is for each level.

99. Dispersion Effects ANOVA for location effects
Different factor level affects variability
dispersion effects
Example
Average and standard deviation are measured for a response variable.

100. Dispersion Effects ANOVA found no location effects
Transform the standard deviation to
Ratio
Obser.
Control
Algorithm 1 2 3 4 5 6

101. Dispersion Effects ANOVA found dispersion effects
One-way ANOVA: y=ln(s) versus Algorithm
Source DF SS MS F P
Algorithm
Error
Total
S = R-Sq = 76.71% R-Sq(adj) = 73.22%

102. Regression and ANOVA Regression Analysis: Etch Rate versus Power
The regression equation is
Etch Rate = Power
Predictor Coef SE Coef T P
Constant
Power
S = R-Sq = 88.4% R-Sq(adj) = 87.8%
Analysis of Variance
Source DF SS MS F P
Regression
Residual Error
Total
Unusual Observations
Obs Power Etch Rate Fit SE Fit Residual St Resid R
R denotes an observation with a large standardized residual.

103. Nonparametric methods in the ANOVA
When normality is invalid
Use Kruskal-Wallis test
Rank observation yij in ascending order
Replace each observation by its rank, Rij
In case tie, assign average rank to them
test statistic

104. Nonparametric methods in the ANOVA
Kruskal-Wallis test
test statistic
where

105. Nonparametric methods in the ANOVA
Kruskal-Wallis test If
the null hypothesis is rejected.

106. Nonparametric methods in the ANOVA
Kruskal-Wallis Test: Etch Rate versus Power
Kruskal-Wallis Test on Etch Rate
Power N Median Ave Rank Z
Overall
H = DF = 3 P = 0.001
H = DF = 3 P = (adjusted for ties)