Edme section 11 CHEN Factorial Example All Slides in this presentation are copyrighted by StatEase, Inc. and used by permission

Edme section 12  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT) 1.Mark Anderson and Pat Whitcomb (2000), DOE Simplified, Productivity Inc., chapter 3. 2.Douglas Montgomery (2006), Design and Analysis of Experiments, 6 th edition, John Wiley, sections 6.1 – 6.3. Two-Level Full Factorials

Edme section 13 Agenda Transition  DOE – Process and design construction Introduce the process for designing factorial experiments and motivate their use.  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

Edme section 14 Process Noise Factors “z” Controllable Factors “x” Responses “y” DOE (Design of Experiments) is: “A systematic series of tests, in which purposeful changes are made to input factors, so that you may identify causes for significant changes in the output responses.” Design of Experiments

Edme section 15 Expend no more than 25% of budget on the 1st cycle. Conjecture Design Experiment Analysis Iterative Experimentation

Edme section 16 DOE Process (1 of 2) 1.Identify opportunity and define objective. 2.State objective in terms of measurable responses. a.Define the change (  y) that is important to detect for each response. b.Estimate experimental error (  ) for each response. c.Use the signal to noise ratio (  y/  ) to estimate power. 3.Select the input factors to study. (Remember that the factor levels chosen determine the size of  y.)

Edme section 17 DOE Process (2 of 2) 4.Select a design and:  Evaluate aliases (details in section 4).  Evaluate power (details in section 2).  Examine the design layout to ensure all the factor combinations are safe to run and are likely to result in meaningful information (no disasters). We will begin using and flesh out the DOE Process in the next section.

Edme section 18 Process Noise Factors “z” Controllable Factors “x” Responses “y” Let’s brainstorm. What process might you experiment on for best payback? How will you measure the response? What factors can you control? Write it down. Design of Experiments

Edme section 19 Jacob Bernoulli ( ) The ‘Father of Uncertainty’ “Even the most stupid of men, by some instinct of nature, by himself and without any instruction, is convinced that the more observations have been made, the less danger there is of wandering from one’s goal.” Central Limit Theorem Compare Averages NOT Individuals

Edme section 110  As the sample size (n) becomes large, the distribution of averages becomes approximately normal.  The variance of the averages is smaller than the variance of the individuals by a factor of n.  (sigma) symbolizes true standard deviation  The mean of the distribution of averages is the same as the mean of distribution of individuals.  (mu) symbolizes true population mean The CLT applies regardless of the distribution of the individuals. Central Limit Theorem Compare Averages NOT Individuals

Edme section 111 Individuals are uniform; averages tending toward normal! Example:"snakeyes" [1/1] is the only way to get an average of one. Central Limit Theorem Illustration using Dice _ _ _ _ _ _ _ _ _ _ _ _ Averages of Two Dice

Edme section 112  As the sample size (n) becomes large, the distribution of averages becomes approximately normal.  The variance of the averages is smaller than the variance of the individuals by a factor of n.  The mean of the distribution of averages is the same as the mean of distribution of individuals. Central Limit Theorem Uniform Distribution

Edme section 113  Want to estimate factor effects well; this implies estimating effects from averages. Refer to the slides on the Central Limit Theorem.  Want to obtain the most information in the fewest number of runs.  Want to estimate each factor effect independent of the existence of other factor effects.  Want to keep it simple. Motivation for Factorial Design

Edme section 114 Run all high/low combinations of 2 (or more) factors Use statistics to identify the critical factors 2 2 Full Factorial What could be simpler? Two-Level Full Factorial Design

Edme section 115 StdABCABACBCABC 1–––+++–y1y1 2+––––++y2y2 3–+––+–+y3y3 4++–+–––y4y4 5––++––+y5y5 6+–+–+––y6y6 7–++––+–y7y y8y8 Design Construction 2 3 Full Factorial

Edme section 116 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn) Learn benefits and basics of two-level factorial design by working through a simple example.  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

Edme section 117 Two Level Factorial Design As Easy As Popping Corn! Kitchen scientists* conducted a 2 3 factorial experiment on microwave popcorn. The factors are: A.Brand of popcorn B.Time in microwave C.Power setting A panel of neighborhood kids rated taste from one to ten and weighed the un-popped kernels (UPKs). *For full report, see Mark and Hank Andersons' Applying DOE to Microwave Popcorn, PI Quality 7/93, p30.

Edme section 118 Two Level Factorial Design As Easy As Popping Corn! * Average scores multiplied by 10 to make the calculations easier. ABCR1R1 R2R2 RunBrandTimePowerTasteUPKsStd Ordexpenseminutespercentrating*oz.Ord 1Costly Cheap Cheap Costly Costly Costly Cheap Cheap

Edme section 119 Two Level Factorial Design As Easy As Popping Corn! Factors shown in coded values ABCR1R1 R2R2 RunBrandTimePowerTasteUPKsStd Ordexpenseminutespercentratingoz.Ord 1+–– –+– –– – – – –––743.11

Edme section 120 R 1 - Popcorn Taste Average A-Effect 75 – 74 = – 71 = – 81 = – 4 32 – 42 = – 10 There are four comparisons of factor A (Brand), where levels of factors B and C (time and power) are the same:

Edme section 121 R 1 - Popcorn Taste Average A-Effect

Edme section 122 R 1 - Popcorn Taste Analysis Matrix in Standard Order  I for the intercept, i.e., average response.  A, B and C for main effects (ME's). These columns define the runs.  Remainder for factor interactions (FI's) Three 2FI's and One 3FI. Std. OrderIABCABACBCABC Taste rating 1+–––+++–74 2++–––– –+––+– –+–––80 5+––++–– –+–+––77 7+–++––+–

Edme section 123 Popcorn Taste Compute the effect of C and BC Std.Taste OrderABCABACBCABCrating 1–––+++–74 2+––––++75 3–+––+– –+–––80 5––++––+81 6+–+–+––77 7–++––+– yy

Edme section 124 Sparsity of Effects Principle Do you expect all effects to be significant? Two types of effects: Vital Few: About 20 % of ME's and 2 FI's will be significant. Trivial Many: The remainder result from random variation.

Edme section 125 Estimating Noise How are the “trivial many” effects distributed?  Hint: Since the effects are based on averages you can apply the Central Limit Theorem.  Hint: Since the trivial effects estimate noise they should be centered on zero. How are the “vital few” effects distributed?  No idea! Except that they are too large to be part of the error distribution.

Edme section 126 Half Normal Probability Paper Sorting the vital few from the trivial many. Significant effects (the vital few) are outliers. They are too big to be explained by noise. They’re "keepers"! Negligible effects (the trivial many) will be N(0,  ), so they fall near zero on straight line. These are used to estimate error.

Edme section 127 Half Normal Probability Paper Sorting the vital few from the trivial many. Significant effects: The model terms! Negligible effects: The error estimate!

Edme section 128 iPiPi |  y| ID AB |–1.0|A |–3.5|ABC |–6.0|AC |–20.5|B 7 1.Sort absolute value of effects into ascending order, “i”. Enter C & BC effects. 2.Compute P i s for effects. Enter P i s for i = 5 & 7. 3.Label the effects. Enter labels for C & BC effects. Half Normal Probability Paper (taste) Sorting the vital few from the trivial many.

Edme section 129 Half Normal Probability Paper (taste) Sorting the vital few from the trivial many.

Edme section 130 Analysis of Variance (taste) Sorting the vital few from the trivial many. Compute Sum of Squares for C and BC: iPiPi |  y| SSID AB |–1.0|2.0A |–3.5|24.5ABC |–6.0|72.0AC |–17.0|C |–20.5|840.5B |–21.5|BC

Edme section 131 Analysis of Variance (taste) Sorting the vital few from the trivial many. 1.Add SS for significant effects: B, C & BC. Call these vital few the “Model”. 2.Add SS for negligible effects: A, AB, AC & ABC. Call these trivial many the “Residual”.

Edme section 132

Edme section 133

Edme section 134

Edme section 135 Analysis of Variance (taste) Sorting the vital few from the trivial many. F-value = < p-value < 0.005

Edme section 136 Analysis of Variance (taste) Sorting the vital few from the trivial many Null Hypothesis: There are no effects, that is: H 0 :  A =  B =…=  ABC = 0 F-value: If the null hypothesis is true (all effects are zero) then the calculated F-value is  1. As the model effects (  B,  C and  BC ) become large the calculated F-value becomes >> 1. p-value: The probability of obtaining the observed F-value or higher when the null hypothesis is true.

Edme section 137 Popcorn Taste BC Interaction BCTaste –– – –+ ++

Edme section 138 Popcorn Taste BC Interaction BCTaste –– – –

Edme section 139 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer Learn to extract more information from the data.  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT)

Edme section 140 Popcorn via Computer! Use Design-Expert to build and analyze the popcorn DOE: Std ord A: Brand expense B: Time minutes C: Power percent R 1 : Taste rating R 2 : UPKs oz. 1Cheap Costly Cheap Costly Cheap Costly Cheap Costly

Edme section 141 Popcorn Analysis via Computer! Instructor led (page 1 of 2) 1.File, New Design. 2.Build a design for 3 factors, 8 runs. 3.Enter factors: 4.Enter responses. Leave delta and sigma blank to skip power calculations. Power will be introduced in section 2!

Edme section 142 Popcorn Analysis via Computer! Instructor led (page 2 of 2) 5.Right-click on Std column header and choose “Sort by Standard Order”. 6.Type in response data (from previous page) for Taste and UPKs. 7.Analyze Taste.  Taste will be instructor led; you will analyze the UPKs on your own. 8.Save this design.

Edme section 143 Popcorn Analysis – Taste Effects Button - View, Effects List TermStdized EffectSumSqr% Contribution RequireIntercept ErrorA-Brand ErrorB-Time ErrorC-Power ErrorAB ErrorAC ErrorBC ErrorABC Lenth's ME Lenth's SME

Edme section 144 Popcorn Analysis – Taste Effects - View, Half Normal Plot of Effects

Edme section 145 Popcorn Analysis – Taste Effects - View, Pareto Chart of “t” Effects

Edme section 146 Popcorn Analysis – Taste ANOVA button Analysis of variance table [Partial sum of squares] Sum ofMeanF SourceSquaresdfSquareValueProb > F Model B-Time C-Power BC Residual Cor Total

Edme section 147 Popcorn Analysis – Taste ANOVA (summary statistics) Std. Dev.4.97R-Squared Mean66.50Adj R-Squared C.V. %7.48Pred R-Squared PRESS396.00Adeq Precision11.939

Edme section 148 Popcorn Analysis – Taste ANOVA Coefficient Estimates CoefficientStandard95% CI95% CI FactorEstimateDFErrorLowHighVIF Intercept B-Time C-Power BC Coefficient Estimate: One-half of the factorial effect (in coded units)

Edme section 149 Final Equation in Terms of Coded Factors: Taste = *B -8.50*C *B*C StdBCPred y 1−− −− 3+− − 5− − Popcorn Analysis – Taste Predictive Equation (Coded)

Edme section 150 Final Equation in Terms of Actual Factors: Taste = *Time +3.62*Power -0.86*Time*Power Popcorn Analysis – Taste Predictive Equation (Actual) StdBCPred y 14 min75% min75% min75% min75% min100% min100% min100% min100%37.00

Edme section 151 Popcorn Analysis – Taste Predictive Equations For process understanding, use coded values: 1.Regression coefficients tell us how the response changes relative to the intercept. The intercept in coded values is in the center of our design. 2.Units of measure are normalized (removed) by coding. Coefficients measure half the change from –1 to +1 for all factors. Actual Factors: Taste = *Time +3.62*Power -0.86*Time*Power Coded Factors: Taste = *B -8.50*C *B*C

Edme section 152 Independent N(0,   ) Factorial Design Residual Analysis

Edme section 153 Popcorn Analysis – Taste Diagnostic Case Statistics Diagnostics → Influence → Report Diagnostics Case Statistics InternallyExternallyInfluence on StdActualPredictedStudentizedStudentizedFitted ValueCook'sRun OrderValueValueResidualLeverageResidualResidualDFFITSDistanceOrder See “Diagnostics Report – Formulas & Definitions” in your “Handbook for Experimenters”.

Edme section 154 Factorial Design ANOVA Assumptions Additive treatment effects Factorial: An interaction model will adequately represent response behavior. Independence of errors Knowing the residual from one experiment gives no information about the residual from the next. Studentized residuals N(0,  2 ): Normally distributed Mean of zero Constant variance,  2 =1 Check assumptions by plotting studentized residuals! Model F-test Lack-of-Fit Box-Cox plot S Residuals versus Run Order Normal Plot of S Residuals S Residuals versus Predicted

Edme section 155 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions

Edme section 156 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions

Edme section 157 Popcorn Analysis – Taste Diagnostics - ANOVA Assumptions Details in section 3

Edme section 158 Popcorn Analysis – Taste Influence

Edme section 159 Popcorn Analysis – Taste Influence

Edme section 160 Popcorn Analysis – Taste Influence

Edme section 161 Popcorn Analysis – Taste Influence ToolDescriptionWIIFM* Internally Studentized Res. Residual divided by the estimated standard deviation of that residual Normality, constant  2 Externally Studentized Res. Residual divided by the estimated std dev of that residual, without the i th case Outlier detection Cook’s DistanceChange in joint confidence ellipsoid (regression) with and without a run Influence DF Fits (difference in fits) Change in predictions with and without a run; the influence a run has on the predictions Influence DF Betas (difference in betas) Change in each model coefficient (beta) with and without a run Influence

Edme section 162 Popcorn Analysis – Taste Model Graphs - Factor “B” Effect Plot Don’t make one factor plot of factors involved in an interaction!

Edme section 163 Popcorn Analysis – Taste Model Graphs – View, Interaction Plot (BC)

Edme section 164 Popcorn Analysis – Taste Model Graphs – View, Contour Plot and 3D Surface (BC)

Edme section 165 Popcorn Analysis – Taste BC Interaction Plot Comparison C: Power Interaction B: Time Taste C- C+

Edme section 166 Popcorn Analysis – UPKs Your Turn! 1.Analyze UPKs: Use the “Factorial Analysis Guide” in your “Handbook for Experimenters” – page Pick the time and power settings that maximize popcorn taste while minimizing UPKs.

Edme section 167 Choose factor levels to try to simultaneously satisfy all requirements. Balance desired levels of each response against overall performance. Popcorn Revisited!

Edme section 168 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization Learn to use numerical search tools to find factor settings to optimize tradeoffs among multiple responses.  Advantage over one-factor-at-a-time (OFAT)

Edme section Go to the Numerical Optimization node and set the goal for Taste to “maximize” with a lower limit of “60” and an upper limit of “90”. 2.Set the goal for UPKs to “minimize” with a lower limit of “0” and an upper limit of “2”. Popcorn Optimization

Edme section Click on the “Solutions” button: Solutions #Brand*TimePowerTasteUPKsDesirability 1Costly Selected 2Cheap Cheap Costly *Has no effect on optimization results. Take a look at the “Ramps” view for a nice summary. Popcorn Optimization

Edme section Click on the “Graphs” button and by right clicking on the factors tool pallet choose “B:Time” as the X1-axis and “C:Power” as the X2-axis: Popcorn Optimization

Edme section Choose “Contour” and “3D Surface” from the “View” menu : Popcorn Optimization

Edme section 173 Popcorn Optimization To learn more about optimization: Read Derringer’s article from Quality Progress: Attend the “RSM” workshop on response surface methodology!

Edme section 174 Agenda Transition  DOE – Process and design construction  Step-by-step analysis (popcorn)  Popcorn analysis via computer  Multiple response optimization  Advantage over one-factor-at-a-time (OFAT) Summarize the benefits factorial design has over one-factor-at-a-time experimentation.

Edme section 175 Traditional Approach to DOE One Factor at a Time (OFAT) “There aren't any interactions." “I'll investigate that factor next.” “It's too early to use statistical methods.” “A statistical experiment would be too large.” “My data are too variable to use statistics.” “We'll worry about the statistics after we've run the experiment.” “Lets just vary one thing at a time so we don't get confused.”

Edme section 176 Relative Efficiency Factorial versus OFAT A B Relative efficiency = 6/4 = 1.5 Relative efficiency = 16/8= 2.0

Edme section k Factorial Design Advantages  What could be simpler?  Minimal runs required. Can run fractions if 5 or more factors.  Have hidden replication.  Wider inductive basis than OFAT experiments.  Show interactions. Key to Success - Extremely important!  Easy to analyze. Do by hand if you want.  Interpretation is not too difficult. Graphs make it easy.  Can be applied sequentially.  Form base for more complex designs. Second order response surface design.