1. Edme section 11 CHEN4860 2-Factorial Example All Slides in this presentation are copyrighted by StatEase, Inc. and used by permission

2. Edme section 12  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT) 1.Mark Anderson and Pat Whitcomb (2000), DOE Simplified, Productivity Inc., chapter 3. 2.Douglas Montgomery (2006), Design and Analysis of Experiments, 6 th edition, John Wiley, sections 6.1 – 6.3. Two-Level Full Factorials

3. Edme section 13 Agenda Transition  DOE – Process and design construction Introduce the process for designing factorial experiments and motivate their use.  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

4. Edme section 14 Process Noise Factors “z” Controllable Factors “x” Responses “y” DOE (Design of Experiments) is: “A systematic series of tests, in which purposeful changes are made to input factors, so that you may identify causes for significant changes in the output responses.” Design of Experiments

5. Edme section 15 Expend no more than 25% of budget on the 1st cycle. Conjecture Design Experiment Analysis Iterative Experimentation

6. Edme section 16 DOE Process (1 of 2) 1.Identify opportunity and define objective. 2.State objective in terms of measurable responses. a.Define the change (  y) that is important to detect for each response. b.Estimate experimental error (  ) for each response. c.Use the signal to noise ratio (  y/  ) to estimate power. 3.Select the input factors to study. (Remember that the factor levels chosen determine the size of  y.)

7. Edme section 17 DOE Process (2 of 2) 4.Select a design and:  Evaluate aliases (details in section 4).  Evaluate power (details in section 2).  Examine the design layout to ensure all the factor combinations are safe to run and are likely to result in meaningful information (no disasters). We will begin using and flesh out the DOE Process in the next section.

8. Edme section 18 Process Noise Factors “z” Controllable Factors “x” Responses “y” Let’s brainstorm. What process might you experiment on for best payback? How will you measure the response? What factors can you control? Write it down. Design of Experiments

9. Edme section 19 Jacob Bernoulli (1654-1705) The ‘Father of Uncertainty’ “Even the most stupid of men, by some instinct of nature, by himself and without any instruction, is convinced that the more observations have been made, the less danger there is of wandering from one’s goal.” Central Limit Theorem Compare Averages NOT Individuals

10. Edme section 110  As the sample size (n) becomes large, the distribution of averages becomes approximately normal.  The variance of the averages is smaller than the variance of the individuals by a factor of n.  (sigma) symbolizes true standard deviation  The mean of the distribution of averages is the same as the mean of distribution of individuals.  (mu) symbolizes true population mean The CLT applies regardless of the distribution of the individuals. Central Limit Theorem Compare Averages NOT Individuals

11. Edme section 111 Individuals are uniform; averages tending toward normal! Example:"snakeyes" [1/1] is the only way to get an average of one. Central Limit Theorem Illustration using Dice 1 1 1 1 3 3 2 2 1 5 2 4 3 3 4 2 5 1 2 6 3 5 4 4 5 3 6 2 4 6 5 5 6 4 6 6 1 2 2 1 6 5 6 5 3 6 4 5 5 4 6 3 1 4 2 3 3 2 4 1 _ _ _ _ _ _ _ _ _ _ _ _ 1 2 45 6 3 Averages of Two Dice 1 2 45 6 3

12. Edme section 112  As the sample size (n) becomes large, the distribution of averages becomes approximately normal.  The variance of the averages is smaller than the variance of the individuals by a factor of n.  The mean of the distribution of averages is the same as the mean of distribution of individuals. Central Limit Theorem Uniform Distribution

13. Edme section 113  Want to estimate factor effects well; this implies estimating effects from averages. Refer to the slides on the Central Limit Theorem.  Want to obtain the most information in the fewest number of runs.  Want to estimate each factor effect independent of the existence of other factor effects.  Want to keep it simple. Motivation for Factorial Design

14. Edme section 114 Run all high/low combinations of 2 (or more) factors Use statistics to identify the critical factors 2 2 Full Factorial What could be simpler? Two-Level Full Factorial Design

15. Edme section 115 StdABCABACBCABC 1–––+++–y1y1 2+––––++y2y2 3–+––+–+y3y3 4++–+–––y4y4 5––++––+y5y5 6+–+–+––y6y6 7–++––+–y7y7 8+++++++y8y8 Design Construction 2 3 Full Factorial

16. Edme section 116 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn) Learn benefits and basics of two-level factorial design by working through a simple example.  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

17. Edme section 117 Two Level Factorial Design As Easy As Popping Corn! Kitchen scientists* conducted a 2 3 factorial experiment on microwave popcorn. The factors are: A.Brand of popcorn B.Time in microwave C.Power setting A panel of neighborhood kids rated taste from one to ten and weighed the un-popped kernels (UPKs). *For full report, see Mark and Hank Andersons' Applying DOE to Microwave Popcorn, PI Quality 7/93, p30.

18. Edme section 118 Two Level Factorial Design As Easy As Popping Corn! * Average scores multiplied by 10 to make the calculations easier. ABCR1R1 R2R2 RunBrandTimePowerTasteUPKsStd Ordexpenseminutespercentrating*oz.Ord 1Costly475 3.52 2Cheap675711.63 3Cheap4100810.75 4Costly675801.24 5Costly4100770.76 6Costly6100320.38 7Cheap6100420.57 8Cheap475743.11

19. Edme section 119 Two Level Factorial Design As Easy As Popping Corn! Factors shown in coded values ABCR1R1 R2R2 RunBrandTimePowerTasteUPKsStd Ordexpenseminutespercentratingoz.Ord 1+––753.52 2–+–711.63 3––+810.75 4++–801.24 5+–+770.76 6+++320.38 7–++420.57 8–––743.11

20. Edme section 120 R 1 - Popcorn Taste Average A-Effect 75 – 74 = + 1 80 – 71 = + 9 77 – 81 = – 4 32 – 42 = – 10 There are four comparisons of factor A (Brand), where levels of factors B and C (time and power) are the same:

21. Edme section 121 R 1 - Popcorn Taste Average A-Effect

22. Edme section 122 R 1 - Popcorn Taste Analysis Matrix in Standard Order  I for the intercept, i.e., average response.  A, B and C for main effects (ME's). These columns define the runs.  Remainder for factor interactions (FI's) Three 2FI's and One 3FI. Std. OrderIABCABACBCABC Taste rating 1+–––+++–74 2++––––++75 3+–+––+–+71 4+++–+–––80 5+––++––+81 6++–+–+––77 7+–++––+–42 8++++++++32

23. Edme section 123 Popcorn Taste Compute the effect of C and BC Std.Taste OrderABCABACBCABCrating 1–––+++–74 2+––––++75 3–+––+–+71 4++–+–––80 5––++––+81 6+–+–+––77 7–++––+–42 8+++++++32 yy -20.50.5-6-3.5

24. Edme section 124 Sparsity of Effects Principle Do you expect all effects to be significant? Two types of effects: Vital Few: About 20 % of ME's and 2 FI's will be significant. Trivial Many: The remainder result from random variation.

25. Edme section 125 Estimating Noise How are the “trivial many” effects distributed?  Hint: Since the effects are based on averages you can apply the Central Limit Theorem.  Hint: Since the trivial effects estimate noise they should be centered on zero. How are the “vital few” effects distributed?  No idea! Except that they are too large to be part of the error distribution.

26. Edme section 126 Half Normal Probability Paper Sorting the vital few from the trivial many. Significant effects (the vital few) are outliers. They are too big to be explained by noise. They’re "keepers"! Negligible effects (the trivial many) will be N(0,  ), so they fall near zero on straight line. These are used to estimate error.

27. Edme section 127 Half Normal Probability Paper Sorting the vital few from the trivial many. Significant effects: The model terms! Negligible effects: The error estimate!

28. Edme section 128 iPiPi |  y| ID 17.140.5AB 221.43|–1.0|A 335.71|–3.5|ABC 450.00|–6.0|AC 5 678.57|–20.5|B 7 1.Sort absolute value of effects into ascending order, “i”. Enter C & BC effects. 2.Compute P i s for effects. Enter P i s for i = 5 & 7. 3.Label the effects. Enter labels for C & BC effects. Half Normal Probability Paper (taste) Sorting the vital few from the trivial many.

29. Edme section 129 Half Normal Probability Paper (taste) Sorting the vital few from the trivial many.

30. Edme section 130 Analysis of Variance (taste) Sorting the vital few from the trivial many. Compute Sum of Squares for C and BC: iPiPi |  y| SSID 17.140.5 AB 221.43|–1.0|2.0A 335.71|–3.5|24.5ABC 450.00|–6.0|72.0AC 564.29|–17.0|C 678.57|–20.5|840.5B 792.86|–21.5|BC

31. Edme section 131 Analysis of Variance (taste) Sorting the vital few from the trivial many. 1.Add SS for significant effects: B, C & BC. Call these vital few the “Model”. 2.Add SS for negligible effects: A, AB, AC & ABC. Call these trivial many the “Residual”.

32. Edme section 132

33. Edme section 133

34. Edme section 134

35. Edme section 135 Analysis of Variance (taste) Sorting the vital few from the trivial many. F-value = 31.5 0.001 < p-value < 0.005

36. Edme section 136 Analysis of Variance (taste) Sorting the vital few from the trivial many Null Hypothesis: There are no effects, that is: H 0 :  A =  B =…=  ABC = 0 F-value: If the null hypothesis is true (all effects are zero) then the calculated F-value is  1. As the model effects (  B,  C and  BC ) become large the calculated F-value becomes >> 1. p-value: The probability of obtaining the observed F-value or higher when the null hypothesis is true.

37. Edme section 137 Popcorn Taste BC Interaction BCTaste ––747574.5 +– –+ ++

38. Edme section 138 Popcorn Taste BC Interaction BCTaste ––747574.5 +–718075.5 –+817779.0 ++423237.0

39. Edme section 139 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer Learn to extract more information from the data.  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

40. Edme section 140 Popcorn via Computer! Use Design-Expert to build and analyze the popcorn DOE: Std ord A: Brand expense B: Time minutes C: Power percent R 1 : Taste rating R 2 : UPKs oz. 1Cheap4.075.074.03.1 2Costly4.075.0 3.5 3Cheap6.075.071.01.6 4Costly6.075.080.01.2 5Cheap4.0100.081.00.7 6Costly4.0100.077.00.7 7Cheap6.0100.042.00.5 8Costly6.0100.032.00.3

41. Edme section 141 Popcorn Analysis via Computer! Instructor led (page 1 of 2) 1.File, New Design. 2.Build a design for 3 factors, 8 runs. 3.Enter factors: 4.Enter responses. Leave delta and sigma blank to skip power calculations. Power will be introduced in section 2!

42. Edme section 142 Popcorn Analysis via Computer! Instructor led (page 2 of 2) 5.Right-click on Std column header and choose “Sort by Standard Order”. 6.Type in response data (from previous page) for Taste and UPKs. 7.Analyze Taste.  Taste will be instructor led; you will analyze the UPKs on your own. 8.Save this design.

43. Edme section 143 Popcorn Analysis – Taste Effects Button - View, Effects List TermStdized EffectSumSqr% Contribution RequireIntercept ErrorA-Brand-120.0819001 ErrorB-Time-20.5840.534.4185 ErrorC-Power-1757823.6691 ErrorAB0.50.50.020475 ErrorAC-6722.9484 ErrorBC-21.5924.537.8583 ErrorABC-3.524.51.00328 Lenth's ME33.8783 Lenth's SME81.0775

44. Edme section 144 Popcorn Analysis – Taste Effects - View, Half Normal Plot of Effects

45. Edme section 145 Popcorn Analysis – Taste Effects - View, Pareto Chart of “t” Effects

46. Edme section 146 Popcorn Analysis – Taste ANOVA button Analysis of variance table [Partial sum of squares] Sum ofMeanF SourceSquaresdfSquareValueProb > F Model2343.003781.0031.560.0030 B-Time840.501840.5033.960.0043 C-Power578.001578.0023.350.0084 BC924.501924.5037.350.0036 Residual99.00424.75 Cor Total2442.007

47. Edme section 147 Popcorn Analysis – Taste ANOVA (summary statistics) Std. Dev.4.97R-Squared0.9595 Mean66.50Adj R-Squared0.9291 C.V. %7.48Pred R-Squared0.8378 PRESS396.00Adeq Precision11.939

48. Edme section 148 Popcorn Analysis – Taste ANOVA Coefficient Estimates CoefficientStandard95% CI95% CI FactorEstimateDFErrorLowHighVIF Intercept66.5011.7661.6271.38 B-Time-10.2511.76-15.13-5.371.00 C-Power-8.5011.76-13.38-3.621.00 BC-10.7511.76-15.63-5.871.00 Coefficient Estimate: One-half of the factorial effect (in coded units)

49. Edme section 149 Final Equation in Terms of Coded Factors: Taste = +66.50 -10.25*B -8.50*C -10.75*B*C StdBCPred y 1−−74.50 2−− 3+−75.50 4+− 5−+79.00 6−+ 7++37.00 8++ Popcorn Analysis – Taste Predictive Equation (Coded)

50. Edme section 150 Final Equation in Terms of Actual Factors: Taste = -199.00 +65.00*Time +3.62*Power -0.86*Time*Power Popcorn Analysis – Taste Predictive Equation (Actual) StdBCPred y 14 min75%74.50 24 min75%74.50 36 min75%75.50 46 min75%75.50 54 min100%79.00 64 min100%79.00 76 min100%37.00 86 min100%37.00

51. Edme section 151 Popcorn Analysis – Taste Predictive Equations For process understanding, use coded values: 1.Regression coefficients tell us how the response changes relative to the intercept. The intercept in coded values is in the center of our design. 2.Units of measure are normalized (removed) by coding. Coefficients measure half the change from –1 to +1 for all factors. Actual Factors: Taste = -199.00 +65.00*Time +3.62*Power -0.86*Time*Power Coded Factors: Taste = +66.50 -10.25*B -8.50*C -10.75*B*C

52. Edme section 152 Independent N(0,   ) Factorial Design Residual Analysis

53. Edme section 153 Popcorn Analysis – Taste Diagnostic Case Statistics Diagnostics → Influence → Report Diagnostics Case Statistics InternallyExternallyInfluence on StdActualPredictedStudentizedStudentizedFitted ValueCook'sRun OrderValueValueResidualLeverageResidualResidualDFFITSDistanceOrder 174.0074.50-0.500.500-0.142-0.123-0.1230.0058 275.0074.500.500.5000.1420.1230.1230.0051 371.0075.50-4.500.500-1.279-1.441-1.4410.4092 480.0075.504.500.5001.2791.4411.4410.4094 581.0079.002.000.5000.5690.5140.5140.0813 677.0079.00-2.000.500-0.569-0.514-0.5140.0815 742.0037.005.000.5001.4211.7501.7500.5057 832.0037.00-5.000.500-1.421-1.750-1.7500.5056 See “Diagnostics Report – Formulas & Definitions” in your “Handbook for Experimenters”.

54. Edme section 154 Factorial Design ANOVA Assumptions Additive treatment effects Factorial: An interaction model will adequately represent response behavior. Independence of errors Knowing the residual from one experiment gives no information about the residual from the next. Studentized residuals N(0,  2 ): Normally distributed Mean of zero Constant variance,  2 =1 Check assumptions by plotting studentized residuals! Model F-test Lack-of-Fit Box-Cox plot S Residuals versus Run Order Normal Plot of S Residuals S Residuals versus Predicted

55. Edme section 155 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions

56. Edme section 156 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions

57. Edme section 157 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions Details in section 3

58. Edme section 158 Popcorn Analysis – Taste Influence

59. Edme section 159 Popcorn Analysis – Taste Influence

60. Edme section 160 Popcorn Analysis – Taste Influence

61. Edme section 161 Popcorn Analysis – Taste Influence ToolDescriptionWIIFM* Internally Studentized Res. Residual divided by the estimated standard deviation of that residual Normality, constant  2 Externally Studentized Res. Residual divided by the estimated std dev of that residual, without the i th case Outlier detection Cook’s DistanceChange in joint confidence ellipsoid (regression) with and without a run Influence DF Fits (difference in fits) Change in predictions with and without a run; the influence a run has on the predictions Influence DF Betas (difference in betas) Change in each model coefficient (beta) with and without a run Influence

62. Edme section 162 Popcorn Analysis – Taste Model Graphs - Factor “B” Effect Plot Don’t make one factor plot of factors involved in an interaction!

63. Edme section 163 Popcorn Analysis – Taste Model Graphs – View, Interaction Plot (BC)

64. Edme section 164 Popcorn Analysis – Taste Model Graphs – View, Contour Plot and 3D Surface (BC)

65. Edme section 165 Popcorn Analysis – Taste BC Interaction Plot Comparison C: Power 4.004.505.005.506.00 Interaction B: Time Taste 30.0 44.0 58.0 72.0 86.0 C- C+

66. Edme section 166 Popcorn Analysis – UPKs Your Turn! 1.Analyze UPKs: Use the “Factorial Analysis Guide” in your “Handbook for Experimenters” – page 2-1. 2.Pick the time and power settings that maximize popcorn taste while minimizing UPKs.

67. Edme section 167 Choose factor levels to try to simultaneously satisfy all requirements. Balance desired levels of each response against overall performance. Popcorn Revisited!

68. Edme section 168 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization Learn to use numerical search tools to find factor settings to optimize tradeoffs among multiple responses.  Advantage over one-factor-at-a-time (OFAT)

69. Edme section 169 1.Go to the Numerical Optimization node and set the goal for Taste to “maximize” with a lower limit of “60” and an upper limit of “90”. 2.Set the goal for UPKs to “minimize” with a lower limit of “0” and an upper limit of “2”. Popcorn Optimization

70. Edme section 170 3.Click on the “Solutions” button: Solutions #Brand*TimePowerTasteUPKsDesirability 1Costly4.00100.0079.00.700.642 Selected 2Cheap4.00100.0079.00.700.642 3Cheap6.0075.0075.51.400.394 4Costly6.0075.0075.51.400.394 *Has no effect on optimization results. Take a look at the “Ramps” view for a nice summary. Popcorn Optimization

71. Edme section 171 4.Click on the “Graphs” button and by right clicking on the factors tool pallet choose “B:Time” as the X1-axis and “C:Power” as the X2-axis: Popcorn Optimization

72. Edme section 172 5.Choose “Contour” and “3D Surface” from the “View” menu : Popcorn Optimization

73. Edme section 173 Popcorn Optimization To learn more about optimization: Read Derringer’s article from Quality Progress: www.statease.com/pubs/derringer.pdf Attend the “RSM” workshop on response surface methodology!

74. Edme section 174 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT) Summarize the benefits factorial design has over one-factor-at-a-time experimentation.

75. Edme section 175 Traditional Approach to DOE One Factor at a Time (OFAT) “There aren't any interactions." “I'll investigate that factor next.” “It's too early to use statistical methods.” “A statistical experiment would be too large.” “My data are too variable to use statistics.” “We'll worry about the statistics after we've run the experiment.” “Lets just vary one thing at a time so we don't get confused.”

76. Edme section 176 Relative Efficiency Factorial versus OFAT A B Relative efficiency = 6/4 = 1.5 Relative efficiency = 16/8= 2.0

77. Edme section 177 2 k Factorial Design Advantages  What could be simpler?  Minimal runs required. Can run fractions if 5 or more factors.  Have hidden replication.  Wider inductive basis than OFAT experiments.  Show interactions. Key to Success - Extremely important!  Easy to analyze. Do by hand if you want.  Interpretation is not too difficult. Graphs make it easy.  Can be applied sequentially.  Form base for more complex designs. Second order response surface design.