GD OE QPRC Tempe - June 2002 Tools for Designing and Analyzing Experiments Russell Barton Management Science and Information Systems Smeal College of Business Administration The Pennsylvania State University Some

GD OE QPRC Tempe - June 2002 Experiment Design and the Scientific Process

GD OE QPRC Tempe - June 2002 Five Steps in the Design of an Experiment 1.Define the goals. 2.Identify and classify (dependent, independent, intermediate, nuisance) variables. 3.Choose a probability model: hypothesize mathematical form of relations between independent and dependent variables. 4.Choose an experiment design. 5.Validate the design.

GD OE QPRC Tempe - June : Goal Hierarchy Plots

GD OE QPRC Tempe - June : Identifying and Classifying via Cause-Effect

GD OE QPRC Tempe - June : Identifying and Classifying via Andrews Andrews Diagram for Spam

GD OE QPRC Tempe - June : Identifying and Classifying via IDEF0

GD OE QPRC Tempe - June : Identifying and Classifying via IDEF0

GD OE QPRC Tempe - June 2002 Five Steps in the Design of an Experiment 1.Define the goals. 2.Identify and classify (dependent, independent, intermediate, nuisance) variables. 3.Choose a probability model: hypothesize mathematical form of relations between independent and dependent variables. 4.Choose an experiment design. 5.Validate the design.

GD OE QPRC Tempe - June : Choosing a Model gas mileage = nominal + 2 x tirepress - 3 x speed y =  0 +  1 x 1 +  2 x 2 + … +  a-priori main effects plots

GD OE QPRC Tempe - June : Choosing a Model gas mileage = nominal + 2 x tirepress - 3 x speed + 2 x (tirepress x speed) y =  0 +  1 x 1 +  2 x 2 +  3 x 1 x 2 + … +  a-priori interaction plots

GD OE QPRC Tempe - June : Choosing a Design - Factorial Designs Factorial Designs: grid designs. A A B A B C A B C A B C D

GD OE QPRC Tempe - June : Choosing a Design - Factorial Designs Factorial Designs: easy to determine most complex model: 1.'power' terms up to one less than the number of levels, and 2.all possible cross-products of different variables. x 1 x 1 2 x 2 x 3 x 1 x 2 x 1 2 x 2 x 1 x 3 x 1 2 x 3 x 2 x 3 x 1 x 2 x 3 x 1 2 x 2 x 3 x1x1 x2x2 x3x3

GD OE QPRC Tempe - June : Choosing a Design - Factorial Designs Problem: factorial designs overemphasize interactions. Solution: use a fractional factorial. Two difficulties: 1.which terms to pair/confound? (statistician's dogma, a priori plots), and 2.which design will do this? (defining relations, projections, geometric patterns)

GD OE QPRC Tempe - June : Choosing a Design - Projections Key Concept: Effect Sparsity x1x1 x3x3 x2x2 x3x3 x1x1 x2x2 x1x1 x2x2 x3x3

GD OE QPRC Tempe - June : Choosing a Design - Projections Geometric Patterns: max distance, regularity Two representation styles x1x1 x2x2 x3x3 x1x1 x2x2 x 3 = hi x 3 = lo

GD OE QPRC Tempe - June : Fries/Hunter (1980) Minimum Aberration Designs A B C D E G F= { I = ABCF = BCDG = ADFG

GD OE QPRC Tempe - June : Fries/Hunter (1980) Minimum Aberration Designs I = ABCF = -ADEG = -BCDEFG A B C D E G F= {

GD OE QPRC Tempe - June : Fries/Hunter (1980) Minimum Aberration Designs I = ABCDF = ABCEG = DEFG

GD OE QPRC Tempe - June : Graphical Construction Rules 1.Use high-order confounding patterns. 2.Check projections. 3.Maximize minimum distance. 4.Points uniformly distributed in space. 5.Decompose complicated designs into geometric components. 6.Place three-or higher level factors on inner (larger) geometric figure. 7.Use icons for three-level hierarchies or three- level variables.

GD OE QPRC Tempe - June 2002 Graphical Analysis Pearson (1934): brick strength

GD OE QPRC Tempe - June 2002 Graphical Analysis Response-Scaled Run Plot: brick strength

GD OE QPRC Tempe - June 2002 Graphical Analysis Snee (1985): product color

GD OE QPRC Tempe - June 2002 Graphical Analysis Response-Scaled Run Plot: product color

GD OE QPRC Tempe - June 2002 Graphical Analysis Model-Free Interpretation: don't run all x's at high level

GD OE QPRC Tempe - June 2002 Non-Graphical Analysis Q: Where is the insight? Estimated Model Coefficients ______________________________________________________________________ Parameter Standard T for H0: Variable DF Estimate Error Parameter=0 Prob > |T| INTERCEPT XA XB XC XAXB XAXC XBXC XAXBXC

GD OE QPRC Tempe - June AGE SCNRT pH RPM Block-4 0 < Current < < Current < < Current < < Current < < Current Key: Graphical Analysis: Acid Mine Drainage

GD OE QPRC Tempe - June 2002 Graphical Analysis Robust Design of Spot Welding Current and Cycle Time

GD OE QPRC Tempe - June 2002 Graphical Analysis Robust Design of Back End Burn-in Process (Rosen, Geist, Finke, Nanda, WSC’01)

GD OE QPRC Tempe - June 2002 GDOE: Tools for DOE Advantages: Easy to remember and conduct each of the five steps of design. Easy to communicate the results of each step. Can be used to construct and understand designs. The same graphical frame can be used to present results. Sometimes, the model-free interpretation is better. Disadvantages: Difficult for more than 8 variables, and for more than 2-3 levels. Verify design properties mathematically - errors possible. Bottom Line: essential, but not exclusive.

GD OE QPRC Tempe - June 2002 GDOE: Tools for DOE Acknowledgments: George Box Stu Hunter David Coleman U.S. Army M.S.I. (Anil Nerode) Doug Montgomery Many statisticians (Andrews, Box, Cox, Cornell, Hahn, Ishikawa, DeBaun, Snee, Tufte, Tukey, Wegman, …) Reference: Graphical Methods for the Design of Experiments, Russell R. Barton, Springer-Verlag LNS 143, 1999.