1. CPE 619 Two-Factor Full Factorial Design With Replications Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama in Huntsville http://www.ece.uah.edu/~milenka http://www.ece.uah.edu/~lacasa

2. 2 Overview Model Computation of Effects Estimating Experimental Errors Allocation of Variation ANOVA Table and F-Test Confidence Intervals For Effects

3. 3 Model Replications allow separating out the interactions from experimental errors Model: With r replications Where

4. 4 Model (cont ’ d) The effects are computed so that their sum is zero: The interactions are computed so that their row as well as column sums are zero: The errors in each experiment add up to zero:

5. 5 Computation of Effects Averaging the observations in each cell: Similarly,  Use cell means to compute row and column effects

6. 6 Example 22.1: Code Size

7. 7 Example 22.1: Log Transformation

8. 8 Example 22.1: Computation of Effects An average workload on an average processor requires a code size of 10 3.94 (8710 instructions) Proc. W requires 10 0.23 (=1.69) less code than avg processor Processor X requires 10 0.02 (=1.05) less than an average processor … The ratio of code sizes of an average workload on processor W and X is 10 0.21 (= 1.62).

9. 9 Example 22.1: Interactions Check: The row as well column sums of interactions are zero Interpretation: Workload I on processor W requires 0.02 less log code size than an average workload on processor W or equivalently 0.02 less log code size than I on an average processor

10. 10 Computation of Errors Estimated Response: Error in the kth replication: Example 22.2: Cell mean for (1,1) = 3.8427 Errors in the observations in this cell are: 3.8455-3.8427 = 0.0028 3.8191-3.8427 = -0.0236, and 3.8634-3.8427 = 0.0208 Check: Sum of the three errors is zero

11. 11 Allocation of Variation Interactions explain less than 5% of variation  may be ignored

12. 12 Analysis of Variance Degrees of freedoms:

13. 13 ANOVA for Two Factors w Replications

14. 14 Example 22.4: Code Size Study All three effects are statistically significant at a significance level of 0.10

15. 15 Confidence Intervals For Effects Use t values at ab(r-1) degrees of freedom for confidence intervals

16. 16 Example 22.5: Code Size Study From ANOVA table: s e =0.03. The standard deviation of processor effects: The error degrees of freedom: ab(r-1) = 40  use Normal tables For 90% confidence, z 0.95 = 1.645 90% confidence interval for the effect of processor W is:  1 ¨ t s  1 = -0.2304 ¨ 1.645 £ 0.0060 = -0.2304 ¨ 0.00987 = (-0.2406, -0.2203) The effect is significant

17. 17 Example 22.5: Conf. Intervals (cont ’ d) The intervals are very narrow.

18. 18 Example 22.5: CI for Interactions

19. 19 Example 22.5: Visual Tests No visible trend. Approximately linear ) normality is valid

20. 20 Summary Replications allow interactions to be estimated SSE has ab(r-1) degrees of freedom Need to conduct F-tests for MSA/MSE, MSB/MSE, MSAB/MSE

21. CPE 619 General Full Factorial Designs With k Factors Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama in Huntsville http://www.ece.uah.edu/~milenka http://www.ece.uah.edu/~lacasa

22. 22 Overview Model Analysis of a General Design Informal Methods Observation Method Ranking Method Range Method

23. 23 General Full Factorial Designs With k Factors Model: k factors ) 2 k -1 effects k main effects two factor interactions, three factor interactions, and so on. Example: 3 factors A, B, C:

24. 24 Model Parameters Analysis: Similar to that with two factors The sums of squares, degrees of freedom, and F-test also extend as expected

25. 25 Case Study 23.1: Paging Process Total 81 experiments

26. 26 Case Study 23.1 (cont ’ d) Total Number of Page Swaps y max /y min = 23134/32 = 723  log transformation

27. 27 Case Study 23.1 (cont ’ d) Transformed Data For the Paging Study

28. 28 Case Study 23.1 (cont ’ d) Effects: Also Six two-factor interactions, Four three-factor interactions, and One four-factor interaction.

29. 29 Case Study 23.1: ANOVA Table

30. 30 Case Study 23.1: Simplified model Most interactions except DM are small. Where,

31. 31 Case Study 23.1: Simplified Model (cont ’ d) Interactions Between Deck Arrangement and Memory Pages

32. 32 Case Study 23.1: Error Computation

33. 33 Case Study 23.1: Visual Test Almost a straight line Outlier was verified

34. 34 Case Study 23.1: Final Model Standard Error = Stdv of sample mean = Stdv of Error

35. 35 Observation Method To find the best combination Example: Scheduler Design Three Classes of Jobs: Word processing Interactive data processing Background data processing Five Factors 2 5-1 design

36. 36 Example 23.1: Measured Throughputs

37. 37 Example 23.1: Conclusions To get high throughput for word processing jobs: 1.There should not be any preemption (A=-1) 2.The time slice should be large (B=1) 3.The fairness should be on (E=1) 4.The settings for queue assignment and re-queueing do not matter

38. 38 Ranking Method Sort the experiments.

39. 39 Example 23.2: Conclusions 1.A=-1 (no preemption) is good for word processing jobs and also that A=1 is bad 2.B=1 (large time slice) is good for such jobs. No strong negative comment can be made about B=-1 3. Given a choice C should be chosen at 1, that is, there should be two queues 4.The effect of E is not clear 5.If top rows chosen, then E=1 is a good choice

40. 40 Range Method Range = Maximum-Minimum Factors with large range are important Memory size is the most influential factor Problem program, deck arrangement, and replacement algorithm are next in order

41. 41 Summary A general k factor design can have k main effects, two factor interactions, three factor interactions, and so on. Information Methods: Observation: Find the highest or lowest response Ranking: Sort all responses Range: Largest - smallest average response