The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs Developed by Don Edwards, John Grego and James Lynch Center for Reliability and Quality Sciences Department of Statistics University of South Carolina

Part I.3 The Essentials of 2-Cubed Designs Methodology Methodology –Cube Plots – Estimating Main Effects – Estimating Interactions (Interaction Tables and Graphs)

Part I.3 The Essentials of 2-Cubed Designs Statistical Significance: When is an Effect “Real”? Statistical Significance: When is an Effect “Real”? An Example With Interactions An Example With Interactions A U-Do-It Case Study A U-Do-It Case Study Replication Replication Rope Pull Exercise Rope Pull Exercise

Statistical Significance When is an Effect “Real”? (As Opposed to Being “Due to Error”) Introduction The Effects (Main and Interactions) We Compute are Really Estimates of the “True Effects” (Remember MAE). The Effects (Main and Interactions) We Compute are Really Estimates of the “True Effects” (Remember MAE). All the True Effects are Probably Nonzero, but Some are Very Small - It is More Correct to Ask If an Effect is “Distinguishable from Error” or “Indistinguishable from Error”. All the True Effects are Probably Nonzero, but Some are Very Small - It is More Correct to Ask If an Effect is “Distinguishable from Error” or “Indistinguishable from Error”.

Statistical Significance When is an Effect “Real”? (As Opposed to Being “Due to Error”) Introduction We will Discuss Tools to Help in This Decision We will Discuss Tools to Help in This Decision –Normal Probability Plots of Estimated Effects –Replication –ANOVA

Statistical Significance When is an Effect “Real”? Normal Probability Plots of Estimated Effects What if all the true effects were zero, so that estimated effects represented only random error? What if all the true effects were zero, so that estimated effects represented only random error?

Statistical Significance When is an Effect “Real”? Normal Probability Plots - Background In 1959, Cuthbert Daniel Found a Way to Plot the Estimated Effects so that Effects Due to Random Error Fall (Roughly) on a Straight Line in the Plot In 1959, Cuthbert Daniel Found a Way to Plot the Estimated Effects so that Effects Due to Random Error Fall (Roughly) on a Straight Line in the Plot To Construct a Normal Probability Plot of the Effects To Construct a Normal Probability Plot of the Effects –1. Order the Estimated Effects from Smallest to Largest (Minus Signs Count: -1 is Less Than 2, For Example). –2. Plot the Points (E i,P i ), i = 1,..,m on Normal Probability Paper, Where m = Number of Effects, E i is the i th Smallest Effect (Put the E’s on the Horizontal Axis), and P i = 100(i-0.5)/m. –3. Normal Probability Paper is on the next Slide for m = 7. To Use This Paper To Use This Paper –Scale the x-axis (Horizontal Axis) to Cover the Range of the Effects –Plot the Smallest Value on Line 1, the Next Smallest on Line 2, etc.

Statistical Significance When is an Effect “Real”? Normal Probability Plots - Seven Effects Paper

Statistical Significance When is an Effect “Real”? Normal Probability Plots - Interpretation If There are Enough Effects Plotted, and Some are Due to Random Error, These Will Lie Approximately on a Straight Line Centered at 0. Sketch in the Line. If There are Enough Effects Plotted, and Some are Due to Random Error, These Will Lie Approximately on a Straight Line Centered at 0. Sketch in the Line. Identify Any Effects That Fall off the Line to the Upper Right and Lower Left. These Effects are Probably Not Due to Noise; They are “Distinguishable from Error”. Identify Any Effects That Fall off the Line to the Upper Right and Lower Left. These Effects are Probably Not Due to Noise; They are “Distinguishable from Error”.

Statistical Significance When is an Effect “Real”? Normal Probability Plots - Example 2

Methodology Example 3 - PC Response Time Objective Reduce Company’s PC Response Time Objective Reduce Company’s PC Response Time Factors Factors –A: Cache (Two Levels Lo, Hi) –B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM –C: Line (Lo - 56K modem, Hi - LAN)

Methodology Example 3 - PC Response Time Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN) Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN)

Methodology Example 3 - PC Response Time Cube Plot Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN) Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN)

Methodology Example 3 - PC Response Time Estimating the Effects - Signs Tables

Methodology Example 3 - PC Response Time Effects Normal Probability Plot

Methodology Interaction Tables and Graphs Tools for Aiding Interpretation of SIGNIFICANT Two-Way Interactions Tools for Aiding Interpretation of SIGNIFICANT Two-Way Interactions At the Right is a Blank AB Interaction Table At the Right is a Blank AB Interaction Table In the Table, 1 Corresponds to the Lo Level and 2 to the Hi Level In the Table, 1 Corresponds to the Lo Level and 2 to the Hi Level Tools for Aiding Interpretation of SIGNIFICANT Two-Way Interactions Tools for Aiding Interpretation of SIGNIFICANT Two-Way Interactions At the Right is a Blank AB Interaction Table At the Right is a Blank AB Interaction Table In the Table, 1 Corresponds to the Lo Level and 2 to the Hi Level In the Table, 1 Corresponds to the Lo Level and 2 to the Hi Level B: 1 A: A -1 B -1 C -1 A -1 B 1 C -1 A -1 B -1 C 1 A -1 B 1 C 1 A -1 B -1 A -1 B 1 1 A 1 B -1 C -1 A 1 B 1 C -1 A 1 B -1 C 1 A1B1C1A1B1C1 A 1 B -1 A1B1A1B1

Methodology Example 3 - PC Response Time AC Interaction Table C: Line 1 A: Cache A -1 C -1 = 45.4A -1 C 1 = A 1 C -1 = 21.5A -1 C 1 = 16.3

Methodology Interaction Tables and Graphs Interaction Plots - Construction 1. For a Given Pair of Factors (Say A and B) Find the Average Response at Each of Their Four Level Combinations. 1. For a Given Pair of Factors (Say A and B) Find the Average Response at Each of Their Four Level Combinations. 2. Plot These with Response on the Vertical Axis, Using One of the Factor’s Levels (Say B) on the Horizontal Axis. Connect and Label the Averages with the Same Level of the Other Factor (A). 2. Plot These with Response on the Vertical Axis, Using One of the Factor’s Levels (Say B) on the Horizontal Axis. Connect and Label the Averages with the Same Level of the Other Factor (A).

Methodology Interaction Tables and Graphs Interaction Plots - Interpretation 1. If the Lines are Roughly Parallel, There is No Strong Interaction. 1. If the Lines are Roughly Parallel, There is No Strong Interaction. 2. If There is Interaction, the Plot Shows Clearly the Effect of a Factor at Each of the Levels of the Other Factor. 2. If There is Interaction, the Plot Shows Clearly the Effect of a Factor at Each of the Levels of the Other Factor. 3. Maximizing and Minimizing Combinations of the Factors are Easily Identified on the Plot and in the Table. 3. Maximizing and Minimizing Combinations of the Factors are Easily Identified on the Plot and in the Table.

Methodology Example 3 AC Interaction Table and Graph Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN) Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN) C: Line 1 A: Cache A -1 C -1 = 45.4A -1 C 1 = A 1 C -1 = 21.5A -1 C 1 = 16.3

Methodology Example 3 - AC Interaction Graph Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN) Response: PC Response Time Response: PC Response Time Factors Factors –A: Cache (Two Levels Lo,Hi) –B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) –C: Line (Lo - 56K modem, Hi - LAN)

Methodology Example 3 - AC Interaction Interpretation Noise Factors versus Control Factors To minimize the response, choose B Hi and C Hi. When C is Hi, the effect of A is negligible. To minimize the response, choose B Hi and C Hi. When C is Hi, the effect of A is negligible. Refer back to the cube plot—the (B Hi, C Hi) edge had the two lowest readings. Our analysis shows that this was not due to a BC interaction, but to a significant B main effect and the particular form of the significant AC interaction. Refer back to the cube plot—the (B Hi, C Hi) edge had the two lowest readings. Our analysis shows that this was not due to a BC interaction, but to a significant B main effect and the particular form of the significant AC interaction. o Response: PC Response Time o Factors – A: Cache (Two Levels Lo,Hi) – B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) – C: Line (Lo - 56K modem, Hi - LAN) o Response: PC Response Time o Factors – A: Cache (Two Levels Lo,Hi) – B: Machine (Lo - 200MHz, 64 MB RAM, Hi - 400MHz, 1GB RAM) – C: Line (Lo - 56K modem, Hi - LAN)

Methodology Example 3 - Estimating the Mean Response: A = +1, B = -1, C = +1 Estimated Mean Response (EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B) +(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 Estimated Mean Response (EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B) +(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 For A = +1, B = -1, C = +1, EMR = [(+1)(-11.9)+(-1)(-16.2)+(+1)(- 17.2)+(1)(12)]/2 = 24.4 For A = +1, B = -1, C = +1, EMR = [(+1)(-11.9)+(-1)(-16.2)+(+1)(- 17.2)+(1)(12)]/2 = 24.4 Notice that for A = +1 and C = +1, [(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 = [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = = A 2 C 2 – y; so EMR= =16.35 Notice that for A = +1 and C = +1, [(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 = [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = = A 2 C 2 – y; so EMR= =16.35 Estimated Mean Response (EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B) +(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 Estimated Mean Response (EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B) +(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 For A = +1, B = -1, C = +1, EMR = [(+1)(-11.9)+(-1)(-16.2)+(+1)(- 17.2)+(1)(12)]/2 = 24.4 For A = +1, B = -1, C = +1, EMR = [(+1)(-11.9)+(-1)(-16.2)+(+1)(- 17.2)+(1)(12)]/2 = 24.4 Notice that for A = +1 and C = +1, [(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 = [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = = A 2 C 2 – y; so EMR= =16.35 Notice that for A = +1 and C = +1, [(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 = [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = = A 2 C 2 – y; so EMR= =16.35 o For Calculating EMR Include: –Significant Main Effects –Significant Interactions, and All Their Lower Order Interactions and Main Effects o For Calculating EMR Include: –Significant Main Effects –Significant Interactions, and All Their Lower Order Interactions and Main Effects

Methodology U-Do-It : Example 3 - Estimate the Response A = +1, B = +1, C = +1 and A = +1, B = +1, C = -1