1. GD OE QPRC Tempe - June 2002 Tools for Designing and Analyzing Experiments Russell Barton Management Science and Information Systems Smeal College of Business Administration The Pennsylvania State University Some

2. GD OE QPRC Tempe - June 2002 Experiment Design and the Scientific Process

3. GD OE QPRC Tempe - June 2002 Five Steps in the Design of an Experiment 1.Define the goals. 2.Identify and classify (dependent, independent, intermediate, nuisance) variables. 3.Choose a probability model: hypothesize mathematical form of relations between independent and dependent variables. 4.Choose an experiment design. 5.Validate the design.

4. GD OE QPRC Tempe - June 2002 1: Goal Hierarchy Plots

5. GD OE QPRC Tempe - June 2002 2: Identifying and Classifying via Cause-Effect

6. GD OE QPRC Tempe - June 2002 2: Identifying and Classifying via Andrews Andrews Diagram for Spam

7. GD OE QPRC Tempe - June 2002 2: Identifying and Classifying via IDEF0

8. GD OE QPRC Tempe - June 2002 2: Identifying and Classifying via IDEF0

9. GD OE QPRC Tempe - June 2002 Five Steps in the Design of an Experiment 1.Define the goals. 2.Identify and classify (dependent, independent, intermediate, nuisance) variables. 3.Choose a probability model: hypothesize mathematical form of relations between independent and dependent variables. 4.Choose an experiment design. 5.Validate the design.

10. GD OE QPRC Tempe - June 2002 3: Choosing a Model gas mileage = nominal + 2 x tirepress - 3 x speed y =  0 +  1 x 1 +  2 x 2 + … +  a-priori main effects plots

11. GD OE QPRC Tempe - June 2002 3: Choosing a Model gas mileage = nominal + 2 x tirepress - 3 x speed + 2 x (tirepress x speed) y =  0 +  1 x 1 +  2 x 2 +  3 x 1 x 2 + … +  a-priori interaction plots

12. GD OE QPRC Tempe - June 2002 4: Choosing a Design - Factorial Designs Factorial Designs: grid designs. A A B A B C A B C A B C D

13. GD OE QPRC Tempe - June 2002 4: Choosing a Design - Factorial Designs Factorial Designs: easy to determine most complex model: 1.'power' terms up to one less than the number of levels, and 2.all possible cross-products of different variables. x 1 x 1 2 x 2 x 3 x 1 x 2 x 1 2 x 2 x 1 x 3 x 1 2 x 3 x 2 x 3 x 1 x 2 x 3 x 1 2 x 2 x 3 x1x1 x2x2 x3x3

14. GD OE QPRC Tempe - June 2002 4: Choosing a Design - Factorial Designs Problem: factorial designs overemphasize interactions. Solution: use a fractional factorial. Two difficulties: 1.which terms to pair/confound? (statistician's dogma, a priori plots), and 2.which design will do this? (defining relations, projections, geometric patterns)

15. GD OE QPRC Tempe - June 2002 4: Choosing a Design - Projections Key Concept: Effect Sparsity x1x1 x3x3 x2x2 x3x3 x1x1 x2x2 x1x1 x2x2 x3x3

16. GD OE QPRC Tempe - June 2002 4: Choosing a Design - Projections Geometric Patterns: max distance, regularity Two representation styles x1x1 x2x2 x3x3 x1x1 x2x2 x 3 = hi x 3 = lo

17. GD OE QPRC Tempe - June 2002 4: Fries/Hunter (1980) Minimum Aberration Designs A B C D E G F= { I = ABCF = BCDG = ADFG

18. GD OE QPRC Tempe - June 2002 4: Fries/Hunter (1980) Minimum Aberration Designs I = ABCF = -ADEG = -BCDEFG A B C D E G F= {

19. GD OE QPRC Tempe - June 2002 4: Fries/Hunter (1980) Minimum Aberration Designs I = ABCDF = ABCEG = DEFG

20. GD OE QPRC Tempe - June 2002 4: Graphical Construction Rules 1.Use high-order confounding patterns. 2.Check projections. 3.Maximize minimum distance. 4.Points uniformly distributed in space. 5.Decompose complicated designs into geometric components. 6.Place three-or higher level factors on inner (larger) geometric figure. 7.Use icons for three-level hierarchies or three- level variables.

21. GD OE QPRC Tempe - June 2002 Graphical Analysis Pearson (1934): brick strength

22. GD OE QPRC Tempe - June 2002 Graphical Analysis Response-Scaled Run Plot: brick strength

23. GD OE QPRC Tempe - June 2002 Graphical Analysis Snee (1985): product color

24. GD OE QPRC Tempe - June 2002 Graphical Analysis Response-Scaled Run Plot: product color

25. GD OE QPRC Tempe - June 2002 Graphical Analysis Model-Free Interpretation: don't run all x's at high level

26. GD OE QPRC Tempe - June 2002 Non-Graphical Analysis Q: Where is the insight? Estimated Model Coefficients ______________________________________________________________________ Parameter Standard T for H0: Variable DF Estimate Error Parameter=0 Prob > |T| INTERCEPT 1 8.054951 0.11733721 68.648 0.0001 XA 1 -0.341090 0.11733721 -2.907 0.0050 XB 1 -0.706990 0.11733721 -6.025 0.0001 XC 1 -0.266921 0.11733721 -2.275 0.0263 XAXB 1 -0.418560 0.11733721 -3.567 0.0007 XAXC 1 -0.521779 0.11733721 -4.447 0.0001 XBXC 1 -0.302551 0.11733721 -2.578 0.0122 XAXBXC 1 -0.557293 0.11733721 -4.749 0.0001

27. GD OE QPRC Tempe - June 2002 22 100 23 30 75 16 30 143 34 71 19 27 141 25 23 100 26 39 AGE SCNRT pH RPM Block-4 0 < Current < 30 30 < Current < 60 60 < Current < 90 90 < Current < 120 120 < Current Key: Graphical Analysis: Acid Mine Drainage

28. GD OE QPRC Tempe - June 2002 Graphical Analysis Robust Design of Spot Welding Current and Cycle Time

29. GD OE QPRC Tempe - June 2002 Graphical Analysis Robust Design of Back End Burn-in Process (Rosen, Geist, Finke, Nanda, WSC’01)

30. GD OE QPRC Tempe - June 2002 GDOE: Tools for DOE Advantages: Easy to remember and conduct each of the five steps of design. Easy to communicate the results of each step. Can be used to construct and understand designs. The same graphical frame can be used to present results. Sometimes, the model-free interpretation is better. Disadvantages: Difficult for more than 8 variables, and for more than 2-3 levels. Verify design properties mathematically - errors possible. Bottom Line: essential, but not exclusive.

31. GD OE QPRC Tempe - June 2002 GDOE: Tools for DOE Acknowledgments: George Box Stu Hunter David Coleman U.S. Army M.S.I. (Anil Nerode) Doug Montgomery Many statisticians (Andrews, Box, Cox, Cornell, Hahn, Ishikawa, DeBaun, Snee, Tufte, Tukey, Wegman, …) Reference: Graphical Methods for the Design of Experiments, Russell R. Barton, Springer-Verlag LNS 143, 1999.