The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs Developed by Don Edwards, John Grego and James Lynch Center for Reliability and Quality Sciences Department of Statistics University of South Carolina

Part I. Full Factorial Designs 2 4 Designs 2 4 Designs –Introduction –Analysis Tools –Example –Violin Exercise 2 k Designs 2 k Designs

2 4 Designs U-Do-It - Violin Exercise

2 4 Designs U-Do-It - Violin Exercise How to Play the Violin in 176 Easy Steps 1,2 A very scientifically-inclined violinist was interested in determining what factors affect the loudness of her instrument. She believed these might include: A very scientifically-inclined violinist was interested in determining what factors affect the loudness of her instrument. She believed these might include: – A: Pressure (Lo,Hi) – B: Bow placement (near,far) – C: Bow Angle (Lo,Hi) – D: Bow Speed (Lo,Hi) The precise definition of factor levels is not shown, but they were very rigidly defined and controlled in the experiment. The precise definition of factor levels is not shown, but they were very rigidly defined and controlled in the experiment. Eleven replicates of the full 2 4 were performed, in completely randomized order. Analyze the data! 1 176=11x16 2 Data courtesy of Carla Padgett Eleven replicates of the full 2 4 were performed, in completely randomized order. Analyze the data! 1 176=11x16 2 Data courtesy of Carla Padgett

2 4 Designs U-Do-It - Violin Exercise Report Form Responses are Averages of 11 Independent Replicates Responses are Averages of 11 Independent Replicates –All 176 trials were randomly ordered Analyze and Interpret the Data Analyze and Interpret the Data Responses are Averages of 11 Independent Replicates Responses are Averages of 11 Independent Replicates –All 176 trials were randomly ordered Analyze and Interpret the Data Analyze and Interpret the Data

2 4 Designs U-Do-It Solution - Violin Exercise Signs Table

U-Do-It Exercise U-Do-It Solution - Violin Exercise Cube Plot Factors Factors –A: Pressure (Lo,Hi) –B: Bow Placement (near,far) –C: Bow Angle (Lo,Hi) –D: Bow Speed (Lo,Hi) Factors Factors –A: Pressure (Lo,Hi) –B: Bow Placement (near,far) –C: Bow Angle (Lo,Hi) –D: Bow Speed (Lo,Hi)

2 4 Designs U-Do-It Solution - Violin Exercise Effects Normal Probability Plot Factors Factors –A: Pressure (Lo,Hi) –B: Bow Placement (near,far) –C: Bow Angle (Lo,Hi) –D: Bow Speed (Lo,Hi) Factors Factors –A: Pressure (Lo,Hi) –B: Bow Placement (near,far) –C: Bow Angle (Lo,Hi) –D: Bow Speed (Lo,Hi)

2 4 Designs U-Do-It Solution - Violin Exercise Interpretation The interaction between A and B is so weak that it is probably ignorable and will not be included initially. This simplifies the analysis since, when there are no interactions, the observed changes in the response will be the sum of the individual changes in the main effects, i.e, the main effects are additive. The interaction between A and B is so weak that it is probably ignorable and will not be included initially. This simplifies the analysis since, when there are no interactions, the observed changes in the response will be the sum of the individual changes in the main effects, i.e, the main effects are additive.

2 4 Designs U-Do-It Solution - Violin Exercise Interpretation When the AB interaction is ignored, we expect When the AB interaction is ignored, we expect –A loudness increase of 3.3 decibels when increasing bow speed from Lo to Hi. –A loudness increase of about 5 decibels when changing the bow placement from “near” to “far”. –A loudness increase of 4.8 decibels when changing pressure from Lo to Hi. –The loudness seems unaffected by the angle factor; this “non-effect” is in itself interesting and useful.

U-Do-It Exercise U-Do-It Solution Violin Exercise: Including the AB Interaction We now include the AB interaction for comparison purposes. Since the interaction is so weak, it does not appreciably change the analysis We now include the AB interaction for comparison purposes. Since the interaction is so weak, it does not appreciably change the analysis

U-Do-It Exercise U-Do-It Solution - Violin Exercise AB Interaction Table

U-Do-It Exercise U-Do-It Solution - Violin Exercise AB Interaction Table/Plot

2 4 Designs U-Do-It Solution - Violin Exercise Interpretation If We Include the AB Interaction, We Expect If We Include the AB Interaction, We Expect –Loudness to increase 3.3 when bowing speed, D, increases from Lo to Hi. –Since the lines in the AB interaction are nearly parallel, the effect of the interaction is weak. This is reflected in our estimates of the EMR.

2 4 Designs U-Do-It Solution - Violin Exercise EMR Let us calculate the EMR if we want the response to be the quietest. Let us calculate the EMR if we want the response to be the quietest. If We Don’t Include the AB Interaction, If We Don’t Include the AB Interaction, –Set A, B and D at their Lo setting, -1. –EMR = ( )/2 = If We Include the AB Interaction, If We Include the AB Interaction, –Set D at its Lo setting, -1. The AB Interaction Table and Plot show that A and B still should be set Lo, -1. Note that when A and B are both -1, AB is +1. –EMR = ( )/2 +(-1.3)/2 =68.93

2 k Designs Introduction Suppose the effects of k factors, each having two levels, are to be investigated. Suppose the effects of k factors, each having two levels, are to be investigated. How many runs (recipes) will there be with no replication? How many runs (recipes) will there be with no replication? –2 k runs How may effects are you estimating? How may effects are you estimating? –There will be 2 k -1 columns in the Signs Table –Each column will be estimating an Effect  k main effects, A, B, C,...  k(k-1)/2 two-way interactions, AB, AC, AD,...  k(k-1)(k-2)/3! three-way interactions ...  k (k-1)-way interactions  one k-way interaction

2 k Designs Analysis Tools Signs Table to Estimate Effects Signs Table to Estimate Effects –2 k -1 columns of signs; first k estimate the k main effects and remaining 2 k -k -1 estimate interactions 2 k - 1 Effects Normal Probability Plots to Determine Statistically Significant Effects 2 k - 1 Effects Normal Probability Plots to Determine Statistically Significant Effects Interaction Tables/Plots to Analyze Two- Way Interactions Interaction Tables/Plots to Analyze Two- Way Interactions EMR Computed as Before EMR Computed as Before

2 k Designs Concluding Comments Know How to Design, Analyze and Interpret Full Factorial Two-Level Designs Know How to Design, Analyze and Interpret Full Factorial Two-Level Designs This means that This means that –The design is orthogonal –The run order is totally randomized

2 k Designs Orthogonality (Hard to Explain) If a Design is Orthogonal, Each Factor’s “Effect” can be Estimated Without Interference From the Others... (Hard to Explain) If a Design is Orthogonal, Each Factor’s “Effect” can be Estimated Without Interference From the Others...

2 k Designs Orthogonality - Checking Orthogonality 1. Use the -1 and 1 Design Matrix. 2. Pick Any Pair of Columns 3. Create a New Column by Multiplying These Two, Row by Row. 4. Sum the New Column; If the Sum is Zero, the Two Columns/Factors Are Orthogonal. 5. If Every Pair of Columns is Orthogonal, the Design is Orthogonal.

2 k Designs Randomization It is Highly Recommended That the Trials be Carried Out in a Randomized Order!!! It is Highly Recommended That the Trials be Carried Out in a Randomized Order!!!

2 k Designs Randomization devices Slips of Paper in a Bowl Slips of Paper in a Bowl Multi-Sided Die Multi-Sided Die Coin Flips Coin Flips Table of Random Digits Table of Random Digits Pseudo-Random Numbers on a Computer Pseudo-Random Numbers on a Computer

2 k Designs Randomization - Why randomize order? It’s MAE’s fault... It’s MAE’s fault... M=A + E (M easured response)= (A ctual effect of factor combination) + (E verything else-”random error”)

2 k Designs Randomization - Beware the convenient sample! Randomize Run Order to Protect Against the Unknown Factors Which are not Either Randomize Run Order to Protect Against the Unknown Factors Which are not Either – varied as experimental factors, or – fixed as background effects. Try Hard to Determine What These Unknown Factors Are! Try Hard to Determine What These Unknown Factors Are!

2 k Designs Randomization - Instructions for Operators Having Randomized the Run Order, Present the Operator With Easy-to-Follow Instructions. Having Randomized the Run Order, Present the Operator With Easy-to-Follow Instructions. Tell Him/Her Not to "Help" by Rearranging the Order for Convenience! Tell Him/Her Not to "Help" by Rearranging the Order for Convenience!

2 k Designs Randomization - Partial Randomization In Certain Situations It May Not Be Possible to Totally Randomize All the Runs In Certain Situations It May Not Be Possible to Totally Randomize All the Runs –e.g., it may be too costly to completely randomize the temperatures of a series of ovens while one may be able to totally randomize the other factor levels This Leads to Blocks of Runs Within Which The Factor Settings Can Be Totally Randomized This Leads to Blocks of Runs Within Which The Factor Settings Can Be Totally Randomized –The Analysis of Blocked Designs Will Be Discussed in a Later Module Remember An Important Goal of a DOE is to Get Good Data Remember An Important Goal of a DOE is to Get Good Data –Randomization Protects Us From Background Sources of Variation Of Which We May Not Be Aware –Blocking Allows Us to Include Known But Hard to Control Sources So That We Estimate Their Effect. We Can Then Remove Their Effect and Analyze the other Factor Effects