1. II.2 Four Factors in Eight Runs Introduction Introduction Confounding Confounding –Confounding/Aliasing –Alias Structure Examples and Exercises Examples and Exercises A Demonstration of the Effects of Confounding A Demonstration of the Effects of Confounding

2. II.2 Four Factors in Eight Runs: Introduction Figure 2 - 2 3 Design Signs Table and Four Factors in Eight Runs Design Matrix o Let’s Compare – 2 3 Design Signs Table – Four Factors in Eight Runs Design Matrix o Let’s Compare – 2 3 Design Signs Table – Four Factors in Eight Runs Design Matrix

3. II.2 Four Factors in Eight Runs: Introduction Exercise - Four Factors in Eight Runs Signs Table To compute estimates, create columns for a signs table by multiplying columns as before; some are done for you. To compute estimates, create columns for a signs table by multiplying columns as before; some are done for you.

4. II.2 Four Factors in Eight Runs: Introduction Exercise Solution - Four Factors in Eight Runs Signs Table The completed signs table is below The completed signs table is below

5. II.2 Four Factors in Eight Runs: Introduction Exercise - Four Factors in Eight Runs Signs Table Solution By plan the column for D = column for ABC, so we say that for this design "D=ABC." Also, we can see from above By plan the column for D = column for ABC, so we say that for this design "D=ABC." Also, we can see from above A = BCDA = BCD B = ACDB = ACD C = ABDC = ABD AB = CDAB = CD AC = BDAC = BD BC = ADBC = AD I = ABCDI = ABCD Where "I" is a column of ones. Where "I" is a column of ones.

6. II.2 Four Factors in Eight Runs: Confounding If we use the signs table to estimate D, what we really get is an estimate of D + ABC. (Exactly the same estimate we’d get if we had done a full 2 4 Design, computed D and ABC and added them.) If we use the signs table to estimate D, what we really get is an estimate of D + ABC. (Exactly the same estimate we’d get if we had done a full 2 4 Design, computed D and ABC and added them.) The two effects are “stuck” together; hence, we say they are confounded with each other (on purpose here). The two effects are “stuck” together; hence, we say they are confounded with each other (on purpose here). Similarly, in this design, Similarly, in this design, A is confounded with BCDA is confounded with BCD B is confounded with ACDB is confounded with ACD AB is confounded with CDAB is confounded with CD ETC!ETC!

7. II.2 Four Factors in Eight Runs: Confounding To Illustrate To Illustrate –We want to know if method 1 is better than method 2 for a task. Ann does method 1, Dan does method 2. If Ann’s results are better, is it because method 1 is better than method 2? Or, is Ann better than Dan? Or, is it both? The factor worker is confounded with the factor method. We can’t separate their effects. Confounding can sometimes be a very dumb thing to do (but not always). Confounding can sometimes be a very dumb thing to do (but not always).

8. II.2 Four Factors in Eight Runs: Confounding When we get the data and compute “D”, the result is really an estimate of D + ABC. When we get the data and compute “D”, the result is really an estimate of D + ABC. So, (another new word coming - duck!) “D” is a false name for the estimate - an alias. When two effects are confounded, we say they are aliases of each other. So, (another new word coming - duck!) “D” is a false name for the estimate - an alias. When two effects are confounded, we say they are aliases of each other. The Alias Structure (also called the confounding structure) of the design is this table you’ve already seen (rearranged here): The Alias Structure (also called the confounding structure) of the design is this table you’ve already seen (rearranged here): I = ABCD I = ABCD A = BCD A = BCD B = ACD B = ACD C = ABD C = ABD D = ABC D = ABC AB = CD AB = CD AC = BD AC = BD BC = AD BC = AD

9. II.2 Four Factors in Eight Runs: An Example Revisit Examples 2 and 4 of Part I Response y: Throughput (KB/sec) Response y: Throughput (KB/sec) The Original Experiment was a 2 4 Design (16 Runs) The Original Experiment was a 2 4 Design (16 Runs) –Four Factors: A, B, C, D, performance tuning parameters such as  number of buffers  size of unix inode tables for file handling –Two Levels In Example 2 an 8 Run Design with only Three Factors was Considered for Illustrative Purposes. The Numbers were Rounded Off for Ease of Calculation In Example 2 an 8 Run Design with only Three Factors was Considered for Illustrative Purposes. The Numbers were Rounded Off for Ease of Calculation –Original Data Was In Tenths and Involved Four Factors –The Estimate of the Three-way Interaction ABC was also Estimating the Effect of D. (D and ABC are confounded/aliased.)

10. II.2 Four Factors in Eight Runs: An Example Revisit Examples 2 and 4 of Part I ABCD = I determines runs in half fraction ABCD = I determines runs in half fraction D = ABC for these runs (Complementary half fraction is determined by ABCD = -I, or D = -ABC, for these runs) D = ABC for these runs (Complementary half fraction is determined by ABCD = -I, or D = -ABC, for these runs)

11. II.2 Four Factors in Eight Runs: An Example Design Matrix

12. II.2 Four Factors in Eight Runs: An Example Signs Table Use Eight Run Signs Table to Estimate Effects Use Eight Run Signs Table to Estimate Effects Factor D is Assigned to the Last Column, ABC Factor D is Assigned to the Last Column, ABC Use Alias Structure to Determine What These Quantities Are Estimating Use Alias Structure to Determine What These Quantities Are Estimating

13. II.2 Four Factors in Eight Runs: An Example Normal Probability Plot Effect A+BCD is Statistically Significant Effect A+BCD is Statistically Significant

14. II.2 Four Factors in Eight Runs: An Example Interpretation Response y: Throughput (KB/sec) Response y: Throughput (KB/sec) Assuming BCD is negligible, you should choose A Hi (A = +) to maximize y Caution: ASSUME Assuming BCD is negligible, you should choose A Hi (A = +) to maximize y Caution: ASSUME

15. II.2 Four Factors in Eight Runs U-Do-It Exercise: Violin Example For the Violin Data, Pretend That a Half-fraction of the Full 2 4 Was Run. For your convenience, the violin data and signs table is on the next slide as well as an eight run signs table with the aliasing structure that determines the half-fraction For the Violin Data, Pretend That a Half-fraction of the Full 2 4 Was Run. For your convenience, the violin data and signs table is on the next slide as well as an eight run signs table with the aliasing structure that determines the half-fraction –Find the Levels of Factors A, B, C and D that Would Have Been Run –Pick out the observed y’s for these runs. Enter these into an eight-run response table and compute the observed effects. –Compare these effects to those which were computed from the full 2 4

16. II.2 Four Factors in Eight Runs U-Do-It Exercise: Violin Example - Signs Tables

17. II.2 Four Factors in Eight Runs U-Do-It Exercise: Violin Solution - Completed Design Matrix The recommended runs used for the half-fraction would assign D to column 7 of an eight run signs table The recommended runs used for the half-fraction would assign D to column 7 of an eight run signs table The completed eight run design matrix (with runs rearranged to standard order) is shown below The completed eight run design matrix (with runs rearranged to standard order) is shown below The completed signs table is shown on the next page The completed signs table is shown on the next page

18. II.2 Four Factors in Eight Runs U-Do-It Exercise: Violin Solution - Completed Signs Tables The completed signs table is below The completed signs table is below The responses that go in standard order on A, B, C in the half-fraction are runs 1, 10, 11, 4, 13, 6, 7, and 16 (in standard order) The responses that go in standard order on A, B, C in the half-fraction are runs 1, 10, 11, 4, 13, 6, 7, and 16 (in standard order) o The half-fraction we used corresponds to those runs in the sixteen run experiment when ABCD = I

19. II.2 Four Factors in Eight Runs U-Do-It Exercise: Violin Solution Table Comparing Estimated Effects A table comparing estimated effect: A table comparing estimated effect: There is strong agreement between the two results. With the half-fraction, we would have come to essentially the same conclusions as the full 2 4, with half the data (and half the work.) There is strong agreement between the two results. With the half-fraction, we would have come to essentially the same conclusions as the full 2 4, with half the data (and half the work.)

20. II.2 Four Factors in Eight Runs Some Notation 2 4-1 is Shorthand for a Half Fraction of the 2 4 Design (Four Factors in Eight Runs) 2 4-1 is Shorthand for a Half Fraction of the 2 4 Design (Four Factors in Eight Runs) The 4 stands for four factorsThe 4 stands for four factors 2 4-1 = (2 4 )(2 -1 ) = half of the 2 4 experiment 2 4-1 = (2 4 )(2 -1 ) = half of the 2 4 experiment 2 k-p is Shorthand for k Factors in 2 k-p Runs 2 k-p is Shorthand for k Factors in 2 k-p Runs k stands for number of factorsk stands for number of factors The 2 -p stands for the fractionation of the 2 k experiment (p=1 for a half fraction, p=2 for a quarter fraction, etc)The 2 -p stands for the fractionation of the 2 k experiment (p=1 for a half fraction, p=2 for a quarter fraction, etc)