1. No-Confounding Designs1 Alternatives to Resolution IV Screening Designs in 16 Runs Douglas C. Montgomery Regents’ Professor of Industrial Engineering & Statistics ASU Foundation Professor of Engineering Arizona State University doug.montgomery@asu.edu

2. No-Confounding Designs2 Alternatives to Resolution IV Screening Designs in 16 Runs This is joint work with Bradley Jones, SAS Institute Jones, B. and Montgomery, D.C. (2010), “Alternatives to Resolution IV Screening Designs in 16 Runs”, International Journal of Experimental Design and Process Optimisation, Vol. 1, No. 4, pp. 285-295.

3. No-Confounding Designs3 Resolution IV Screening Designs in 16 Runs These are designs for 6, 7, and 8 factors Very widely used The generators for the standard designs are: –For six factors, E = ABC and F = BCD; –for seven factors, E = ABC, F = BCD, and G = ACD; –for eight factors, E = BCD, F =ACD, G = ABC, and H = ABD.

4. No-Confounding Designs4 Alias Relationships: Six Factors

5. No-Confounding Designs5 Alias Relationships: Seven and Eight Factors

6. No-Confounding Designs6 Alias Relationships In each design, there are seven alias chains involving only two—factor interactions These are regular fractions These are the minimum aberration fractions Because two-factor interactions are completely confounded experimenters often experience ambiguity in interpreting results –Resolve with “process knowledge’ –Additional experimentation

7. No-Confounding Designs7 Example: Montgomery (2012), Design and Analysis of Experiments, 8E This experiment was conducted to study the effects of six factors on the thickness of photoresist coating applied to a silicon wafer. The design factors are A = spin speed, B = acceleration, C = volume of resist applied, D = spin time, E = resist viscosity, and F = exhaust rate.

8. No-Confounding Designs8 Interpretation: A, B, C, and E are probably real effects AB and CE are aliases Either some process knowledge or additional experimentation is required to complete the interpretation

9. No-Confounding Designs9 Additional Experimentation: Fold-Over Reverse the signs in column A, add another 16 runs to the original design This single-factor fold over allows all the two-factor interactions involving the factor whose signs are switched to be separated:

10. No-Confounding Designs10 Results:

11. No-Confounding Designs11 Other “Design Augmentation” Tricks Partial fold-over, requiring only eight additional runs Optimal augmentation (using the D-optimality criterion) But what if additional experimentation isn’t an option?

12. No-Confounding Designs12 An Alternative to Additional Experimentation While strong two-factor interactions may be less likely than strong main effects, there are many more interactions than main effects in screening situations. It is not unusual to find that between 15 and 30% of the effects in a screening design are active As a result, the likelihood of at least one significant interaction effect is high. There is often substantial institutional reluctance to commit additional time and material (and sometimes it’s impossible). Consequently, experimenters would like to avoid the need for a follow-up study. We show can lower the risk of analytical ambiguity by using a specific orthogonal but non-regular fractional factorial design. The proposed designs for six, seven, and eight factor studies in 16 runs have no complete confounding of pairs of two-factor interactions.

13. No-Confounding Designs13 Background Plackett and Burman (1946) introduced non-regular orthogonal designs for sample sizes that are a multiple of four but not powers of two. Hall (1961) identified five non-isomorphic orthogonal designs for 15 factors in 16 runs. Our proposed six through eight factor designs are projections of the Hall designs created by selecting specific sets of columns. Box and Hunter (1961) introduced the regular fractional factorial designs that became the standard tools for factor screening. Sun, Li and Ye (2002) catalogued all the non-isomorphic projections of the Hall designs. Li, Lin and Ye (2003) used this catalog to identify the best designs to use in case there is a need for a fold-over. For each of these designs they provide the columns to use for folding and the resulting resolution of the combined design. Loeppky, Sitter and Tang (2007) also used this catalog to identify the best designs to use assuming that a small number of factors are active and the experimenter wished to fit a model including the active main effects and all two-factor interactions involving factors having active main effects.

14. No-Confounding Designs14 Metrics for Comparing Screening Designs Consider the model y = X β + e where X contains columns for the intercept, main effects and all two-factor interactions, β is the vector of model parameters, and ε is the usual error vector. For six through eight factors in 16 runs, the matrix, X, has more columns than rows. Thus, it is not of full rank and the usual least squares estimate for β does not exist because the matrix X’X is singular. With respect to this model, every 16 run design is supersaturated. Booth and Cox (1962) introduced the E(s 2 ) criterion as a diagnostic measure for comparing supersaturated designs: Minimizing the E(s2) criterion is equivalent to minimizing the sum of squared off-diagonal elements of the correlation matrix of X.

15. No-Confounding Designs15 The Correlation Matrix for the Regular 2 6-2 Resolution IV Fractional Factorial Design The correlation is zero between all main effects and two-factor interactions (because the design is resolution IV) and the correlation is +1 between every two-factor interaction and at least one other two- factor interaction. If another member of the same design family had been used at least one of the generators would have been used with a negative sign in design construction and some of the entries of the correlation matrix would have been -1.

16. No-Confounding Designs16 Alias Matrix The model we fit The true model In a regular design, all entries in A are either 0 or ±1. In a non-regular design, some entries will be 0 < |a ij | < 1. A few non-regular designs have no ±1 entries. The trace of is a measure of the total bias in a design.

17. No-Confounding Designs17 Number of Factors Number of Designs 627 755 880 Number of 16-Run Orthogonal Non- isomorphic Designs

18. No-Confounding Designs18 RunABCDEF 1111111 211 3 11 4 11 5111 1 611 1 1 7 1 1 8 11 91 111 101 1 11111 1 121 1 1311 141 111 1511 11 161 1 Recommended 16-Run Six-Factor No- Confounding Design

19. No-Confounding Designs19 Correlation Matrix (a) Regular 2 6-2 Fractional Factorial (b) the No-Confounding Design

20. No-Confounding Designs20 Recommended 16-Run Seven Factor No-Confounding Design RunABCDEFG 11111111 2111 311 11 411 11 51 11 1 61 1 1 1 71 1 1 81 11 9 11111 1011 1 111 1 11 121 1 13 11 14 1 111 15 11 1 16 1

21. No-Confounding Designs21 Correlation Matrix (a) Regular 2 7-3 Fractional Factorial (b) the No-Confounding Design

22. No-Confounding Designs22 Recommended 16-Run Eight Factor No-Confounding Design RunABCDEFGH 111111111 21111 311 11 411 11 51 1 1 1 61 1 1 1 71 11 1 81 1 11 9 1111111 1011 1 111 1 1 121 1 1 13 11 1 14 1 11 15 111 16 1 11

23. No-Confounding Designs23 Correlation Matrix (a) Regular 2 8-4 Fractional Factorial (b) the No-Confounding Design

24. No-Confounding Designs24 Design Comparison on Metrics N Factors Design Confounded Effect Pairs E(s 2 )Trace(AA’) 6Recommended07.316 Resolution IV910.970 7Recommended010.166 Resolution IV2114.200 8Recommended012.8010.5 Resolution IV4217.070

25. No-Confounding Designs25 Example Revisited The No-Confounding Design for the Photoresist Experiment RunABCDEF Thick ness 1 1111114494 2 11 4592 3 11 4357 4 114489 5 1111 4513 6 111 14483 7 1 14288 8 11 4448 9 1111 4691 10 1 14671 11 111 14219 12 1 1 4271 13 11 4530 14 1 1114632 15 11 114337 16 1 1 4391

26. No-Confounding Designs26 LockEnteredParameterEstimatenDFSS"F Ratio""Prob>F" XXIntercept4462.812 5 100.0001 XA85.3125177634.3753.9762.46e-5 XB-77.6825164368.7644.7535.43e-5 XC-34.1875242735.8414.8560.00101 D0131.198570.0200.89184 XE21.5625231474.3410.9410.00304 F012024.0451.4740.25562 A*B01395.85180.2550.6259 A*C01476.17810.3080.59234 A*D023601.7491.3360.31571 A*E01119.46610.0750.78986 A*F024961.2832.1060.18413 B*C0160.915110.0380.84923 B*D02938.88090.2790.76337 B*E013677.9313.0920.11254 B*F022044.1190.6630.54164 C*D021655.2640.5200.61321 XC*E54.8125124035.2816.7110.00219 C*F022072.4970.6730.53667 D*E0279.650540.0220.97803 D*F000.. E*F025511.2752.4850.14476

27. No-Confounding Designs27 If I only knew then what I know now…

28. No-Confounding Designs28 A personal experience with DOX: Wine-making. Original vineyard property purchased in 1983. Objectives were to begin as a grape supplier to other winemakers, then develop a winemaking process. Big problem: none of the partners were winemakers (how do you make a small fortune in wine…”). However, some partners knew the power of designed experiments. “Wine-making is a chemical process…I didn’t know how to make polymer, either, when I took my first job as a chemical engineer.”

29. No-Confounding Designs29 There are many factors involved. One experiment per year is all that is feasible. Focus on Pinot Noir (Burgundy). Factors considered for one year (1985):

30. No-Confounding Designs30 The experimental design for 1985 is a fractional factorial:

31. No-Confounding Designs31 Results for 1985: [AD] = AD + CF + BH + EG

32. No-Confounding Designs32 Some of the results are interesting and useful, such as 1.toasting the barrel a little more seems like a good idea, and 2.it doesn’t seem to matter much where the oak comes from. Some results are surprising such as no temperature effect! There is an interaction: [AD] = AD + CF + BH + EG Which effect(s) are real? CF and EG are more intuitive than AD, but we really don’t have any “process knowledge” How would we normally resolve this? 1.Fold-over (be careful)? 2.Partial fold-over? 3.Would the recommended non-regular designs have been better?

33. No-Confounding Designs33 How did we do? First commercial release, the1990 Pinot Noir, won a gold medal at the American Wine Competition 1991 release won a silver medal 1992 release won gold a medal, sixth best wine overall (of 2000 entries), best Oregon Pinot Noir Consistently ranked by The Wine Spectator as among the best Pinot Noir available, 91-94 rating Consistently highly rated in the International Pinot Noir Competition

34. No-Confounding Designs34

35. No-Confounding Designs35 Other Work What’s the power of these designs? How many two-factor interactions can we detect? Projections? Analysis methods? What about the resolution III case (9-15 factors in 16 runs)?

36. No-Confounding Designs36 References Booth, K.H.V., Cox, D.R., (1962). “Some systematic supersaturated designs.” Technometrics 4, 489–495. Box, G. E. P. and Hunter, J. S. (1961). “The 2 k-p Fractional Factorial Designs.” Technometrics 3, pp.449–458. Bursztyn, D. and Steinberg, D. (2006). “Comparison of designs for computer experiments” Journal of Statistical Planning and Inference 136, pp. 1103-1119. Hall, M. Jr. (1961). Hadamard matrix of order 16. Jet Propulsion Laboratory Research Summary, 1, pp. 21–26. Jones, B. and Montgomery, D.C. (2010), “Alternatives to Resolution IV Screening Designs in 16 Runs”, International Journal of Experimental Design and Process Optimisation, Vol. 1, No. 4, pp. 285-295. Li, W., Lin, D.K.J., Ye, K. (2003) “Optimal Foldover Plans for Two-Level Non-regular Orthogonal Designs” Technometrics 45, pp.347–351. Plackett, R. L. and Burman, J. P. (1946). “The Design of Optimum Multifactor Experiments.” Biometrika 33, pp. 305–325. Loeppky, J. L., Sitter, R. R., and Tang, B.(2007) “Nonregular Designs With Desirable Projection Properties.” Technometrics 49, pp.454–466. Montgomery, D.C. (2009). Design and Analysis of Experiments 7th Edition. Wiley Hoboken, New Jersey. Sun, D. X., Li, W., and Ye, K. Q. (2002). “An Algorithm for Sequentially Constructing Non- Isomorphic Orthogonal Designs and Its Applications.” Technical Report SUNYSB-AMS-02-13, State University of New York at Stony Brook, Dept. of Applied Mathematics and Statistics.