Chapter 22 Design of Experiments

Experiments Experiment: A study in which certain independent variables are manipulated, their effect on one or more dependent variables is determined, and the levels (values) of these independent variables are assigned at random to the experimental units in study. Trial and Error Experiments Lack direction and focus Waste valuable resources May never reach optimal solution

Design of Experiments (DOE) Design of Experiment: A method of experimenting with the complex interactions among parameters in a process or product. It provides a layout of the different factors and the values at which those factors are to be tested. It is useful to identify the effects of hidden variables

Design of Experiments (DOE) Terminology Factor: A factor is an input variable that can be manipulated during the experiment (e.g., temperature) Level: A level is one of the settings available for a factor (e.g., level 1 = 120F, level 2 = 150F) Qualitative factors: levels vary by category Quantitative factors: levels vary by numerical degree Fixed levels: levels are set at specified values Random levels: levels are randomly selected from all possible values

Design of Experiments (DOE) Terminology Response variable: The variable(s) of interest used to describe the reaction of a process to variations in control variables (factors) Effect: The effect is the change exhibited by he response variable when the factor level is changed Main Effect Interactions: Two or more factors that together produce a result different than what the result of their separate effects would be

Design of Experiments (DOE) Terminology Experimental Unit: The entity to which a specific treatment combination is applied (e.g. PC board, silicon wafer) Blocking: A schedule for conducting treatment combinations in an experimental study such that any effects on the experimental results due to a known change (e.g. raw materials, operators, machines, etc.) become concentrated in the levels of the blocking variable (e.g. shift)

Design of Experiments (DOE) Terminology Treatment: The specific combination of levels for each factor used for a particular run Run: An experimental trail, the application of one treatment Replicate: Repeat of the treatment condition Repetition: Multiple runs of a particular treatment condition

Design of Experiments (DOE) Terminology Noise Factor: An uncontrollable, but measurable, source of variation in the function characteristics of a process or a product Confounding factor: A factor’s effect on the dependent variable cannot be distinguished from the effect of the independent variable. X and Y are confounded when there is a third variable Z that influences both X and Y; such a variable is then called a confounder of X and Y. Alias: When the estimate of an effect also includes the influence of one or more other effects, the effects are said to be aliased. For example, if the estimate of effect D in a four factor experiment actually estimates (D + ABC), then the main effect D is aliased with the 3-way interaction ABC.

Types of Experiments Trial-error methods -- Introduce a change and see what happens Running special lots -- Produced under controlled conditions Pilot runs – Set up to produce a desired effect One-factor at a time experiments – Vary 1 factor and keep all other factors constant Planned comparisons of two methods – Background variables considered in plan Experiments with 2~4 factors – Study separate effects and interactions Experiments with 5~20 factors – Screening studies Comprehensive experimental plan with many phases – Modeling, multiple factor levels, optimization

How does DOE help to improve a process? Screening Designs -- to identify the “vital few” process factors, large number of factors at two levels Characterization Design – to study effects of a small number of factors, full factorial models Optimization Design -- to study complicated effects and interactions of one or two number of factors, full factorial models

Steps in DOE Define Design Statement of Problem Choice of Response (dependent variable). Selection of factors (independent variables) to be varied. Choices of levels of these factors Qualitative or Quantitative Fixed or random Design Number of observation to be taken Order of experimentation. Method of randomization to be used. Mathematical model to describe the experiment. Hypothesis to be test.

Steps in DOE Analysis Data collection and processing. Computation of test statistics and preparation of graphics. Interpretation of results.

Treatment Combination (1,1) Treatment Combination (1,2) Complete Factorial Experiment Design   Factor B Factor A Level 1 Level 2 Level 3 Treatment Combination (1,1) Obs 1 Obs 2 Obs 3 Treatment Combination (1,2)

Nested Experiment Design Factor A A-level 1 A-level 2 A-level 3 Factor B B-level 1 B-level 2 B-level 3 B-level 4 B-level 5 B-level 6  

Completely Randomized Single-Factor Experiments An observation, Yij, for the ith observation and jth treatment may contain three parts Yij = m + tj + eij m: a common effect for the whole experiment, a fixed parameter tj: the effect of the jth treatment, tj = mj - m eij: random error present in the ith observation and jth treatment. NID (0, 2) where 2 is the common variance within all treatments

Completely Randomized Single-Factor Experiments Fixed Model t1, t2, .... , tj, ..... , tk are considered to be fixed parameters H0: t1 = t2 = ........ = tk = 0 H1: At least one of the treatment effect  0

Completely Randomized Single-Factor Experiments Random Model t1, t2, .... , tj, ..... , tk are considered to be random variables tj’s are NID(0, 2) H0: 2= 0 H1: At least one of the treatment effect  0

Completely Randomized Single-Factor Experiments Example 1 We are interested in knowing if there is a difference of the resistance to abrasion of 4 different fabrics, A, B, C, D. A single factor randomized experiment is run. The four treatments (levels) are A, B, C, and D. (Fixed Effect)

Example 1 H0: t1 = t2 = ........ = tk = 0 H1: At least one of the treatment effect  0 Treatment Obs. A B C D 1 1.93 2.55 2.40 2.33 2 2.38 2.72 2.68 3 2.20 2.75 2.31 2.28 4 2.25 2.70

Example 1 One-way ANOVA Source DF SS MS F P Treatment 3 .5201 .1734 8.53 0.0026 Residual Error 12 .2438 .0203 Total 15 .7639 S = 0.1425 R-Sq = 68.09% R-Sq(adj) = 60.11% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev ---------+---------+---------+---------+ A 4 2.1900 0.1892 (------*-----) B 4 2.6800 0.0891 (-----*-----) C 4 2.4175 0.1823 (------*-----) D 4 2.3150 0.0656 (------*-----) ---------+---------+---------+---------+ 2.25 2.50 2.75 3.00 Pooled StDev = 0.1425

Example 2 H0: t1 = t2 = ........ = tk = 0 H1: At least one of the treatment effect  0 Output Obs. A B C 1 4 2 -3 8 3 5 -2 7 -1 6

Example 2 Source DF SS MS F P Machine 2 124.13 62.07 25.86 0.000 Error 12 28.80 2.40 Total 14 152.93 S = 1.549 R-Sq = 81.17% R-Sq(adj) = 78.03% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev --------+---------+---------+---------+- A 5 6.000 1.581 (----*----) B 5 1.800 1.483 (----*----) C 5 -1.000 1.581 (----*----) --------+---------+---------+---------+- 0.0 3.0 6.0 9.0 Pooled StDev = 1.549

ANOVA Rationale Factor Treatment (level #1) Treatment (level #2)   Factor Treatment (level #1) Treatment (level #2) Treatment (level #j) Treatment (level #k) Obs. #1 Y11 Y12 Y1j Y1k Obs. #2 Y21 Y22 Y2j Y2k Obs. #3 Y31 Y32 Y3j Y3k Obs. #i Yi1 Yi2 Yij Yik m1 m22 mj mk

ANOVA Rationale Yij = m + tj + eij Yij = m + (mj – m) + (Yij – mj)   Yij = m + (mj – m) + (Yij – mj) Yij - m = (mj – m) + (Yij – mj)

SStotal = SStreatment +SSerror ANOVA Rationale SStotal = SSbetween+SSwithin SStotal = SStreatment +SSerror

ANOVA Rationale

Single Factor ANOVA Test Source df SS MS F p-value Treatmt. k-1 SStreatment MStreatment P(F(1, 2)  f) Error N-k SSerror MSerror   Totals N-1 SStotal

After ANOVA, there is a difference. What do we do next? Identify the treatment(s) which make the difference. Compare the means

Two-way ANOVA High Pressure Med Pressure Low Pressure High Temp 39 32 18 30 31 20 35 28 21 43 25 29 26 Med Temp 38 10 22 15 36 Low Temp 24 37 27

Two-way ANOVA Two-way ANOVA: Photoresist versus Pressure, Temp Source DF SS MS F P Pressure 2 1034.84 517.422 20.15 0.000 Temp 2 23.51 11.756 0.46 0.636 Interaction 4 139.56 34.889 1.36 0.268 Error 36 924.40 25.678 Total 44 2122.31 S = 5.067 R-Sq = 56.44% R-Sq(adj) = 46.76%

Two-way ANOVA Individual 95% CIs For Mean Based on Pooled StDev Pressure Mean ---------+---------+---------+---------+ H 34.2667 (-----*----) L 23.0667 (----*----) M 25.6000 (----*-----) ---------+---------+---------+---------+ 25.0 30.0 35.0 40.0 Temp Mean --------+---------+---------+---------+- H 28.6667 (------------*-------------) L 27.1333 (-------------*------------) M 27.1333 (-------------*------------) --------+---------+---------+---------+- 26.0 28.0 30.0 32.0

Factorial Experiments Full Factorial design consists of all possible combinations of all selected levels of the factors to be investigated

Example Interested to find the effect of both Temperature and Altitude on the Current Flow in IC. Two temperature settings: 25oC and 55oC. Two altitude settings: 0K ft. and 3K ft. A

One-factor-at-a-time Experiment Holding one variable constant while changing the other. At 0K ft. (Altitude) Temperature 25oC 55oC Current 210 mA 240 mA Observed: Temperature  , Current  by 30 mA By chance? True increase? How do we make sure?

One-factor-at-a-time Experiment (Cont.) Repeat the experiment again! At 0K ft. (Altitude) Temperature 25oC 55oC Current (I) 210 mA 240 mA Current (II) 205 mA 230 mA Temperature  , Current  by (30+25)/2 = 27.5 mA How about the current under altitude?

One-factor-at-a-time Experiment (Cont.) At 25oC (Temperature) Altitude 0K ft 3K ft Current (I) 210 mA 180 mA Current (II) 205 mA 185 mA Altitude  , Current  by (30+20)/2 = 25 mA At 55oC, would current increase /decrease as Altitude increase? Run two more experiments under 55oC. 8 experiments total!!

Factorial Arrangement 25oC 55oC 0K ft 210 mA 240 mA 3K ft 180 mA 200 mA No Interaction!

Factorial Arrangement Factorial Fit: Current versus Altitude, Temp Estimated Effects and Coefficients for Current (coded units) Term Effect Coef Constant 207.50 Altitude -35.00 -17.50 Temp 25.00 12.50 Altitude*Temp -5.00 -2.50 Analysis of Variance for Current (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 1850.00 1850.00 925.00 * * 2-Way Interactions 1 25.00 25.00 25.00 * * Residual Error 0 * * * Total 3 1875.00

Factorial Arrangement 25oC 55oC 0K ft 210 mA 240 mA 3K ft 180 mA 160 mA Interaction!

Factorial Arrangement Analysis of Variance for Current (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 3050.0 3050.0 1525.0 * * 2-Way Interactions 1 625.0 625.0 625.0 * * Residual Error 0 * * * Total 3 3675.0 Estimated Coefficients for Current using data in uncoded units Term Coef Constant 185.000 Altitude 3.88889 Temp 1.00000 Altitude*Temp -0.555556

Factorial Arrangement More efficient: less experiments, more information. Temperature effect Altitude effect Interaction   Better identify the optimal setting

One-factor-at-a-time Experiment

Factorial Arrangement

Factorial Experiment (An Example) Subject: Vacuum Tube Interested to find the effect of Exhaust index ( in seconds) Pump heater Voltage (in volts)   On the pressure inside a tube ( m-6 Hg) Index: 60, 90, 150 Voltage: 127, 220

Factorial Experiment (An Example) Pump Pressure Exhaust Index Voltage 60 90 150 127 48 28 7 58 33 15 220 62 14 9 54 10 6

Factorial Experiment (An Example) One-way ANOVA One-way ANOVA: Pressure versus Group Source DF SS MS F P Group 5 4987.7 997.5 43.06 0.000 Error 6 139.0 23.2 Total 11 5126.7 S = 4.813 R-Sq = 97.29% R-Sq(adj) = 95.03% Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev +---------+---------+---------+--------- 1 2 53.000 7.071 (----*---) 2 2 58.000 5.657 (---*---) 3 2 30.500 3.536 (---*---) 4 2 12.000 2.828 (---*---) 5 2 11.000 5.657 (----*---) 6 2 7.500 2.121 (---*---) +---------+---------+---------+--------- 0 20 40 60 Pooled StDev = 4.813

Factorial Experiment (An Example) Two-way ANOVA Two-way ANOVA: Pressure versus Exhaust, Voltage Source DF SS MS F P Exhaust 2 4608.17 2304.08 99.46 0.000 Voltage 1 96.33 96.33 4.16 0.088 Interaction 2 283.17 141.58 6.11 0.036 Error 6 139.00 23.17 Total 11 5126.67 S = 4.813 R-Sq = 97.29% R-Sq(adj) = 95.03% Individual 95% CIs For Mean Based on Pooled StDev Exhaust Mean --------+---------+---------+---------+- 60 55.50 (---*---) 90 21.25 (---*---) 150 9.25 (---*---) --------+---------+---------+---------+- 15 30 45 60 Voltage Mean -------+---------+---------+---------+-- 127 31.5000 (-----------*-----------) 220 25.8333 (-----------*-----------) -------+---------+---------+---------+-- 24.0 28.0 32.0 36.0

Factorial Experiment (An Example) General Linear Model: Pressure versus Exhaust, Voltage Factor Type Levels Values Exhaust fixed 3 60, 90, 150 Voltage fixed 2 127, 220 Analysis of Variance for Pressure, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Exhaust 2 4608.17 4608.17 2304.08 99.46 0.000 Voltage 1 96.33 96.33 96.33 4.16 0.088 Exhaust*Voltage 2 283.17 283.17 141.58 6.11 0.036 Error 6 139.00 139.00 23.17 Total 11 5126.67 S = 4.81318 R-Sq = 97.29% R-Sq(adj) = 95.03%

Factorial Experiment (An Example) A significant interaction means that the effect of exhaust index on the vacuum tube pressure at one voltage is different from its effect at the other voltage.

Factorial Experiment (An Example) One-way ANOVA

Factorial Experiment (An Example) Two-way ANOVA

Factorial Experiments (2 Factors) Yijk =  + Ai + Bj + ABij + k(ij)   H01: A1 = A2 = ........ = Aa = 0 H02: B1 = B2 = ........ = Bb = 0 H03: AB11 = AB12 = ........ = ABab = 0

2x2 Factorial Experiments (2 Factors, 2 Levels each) Yijk =  + Ai + Bj + ABij + k(ij) i=1,2 j=1,2 k=1, 2, …, n

Example Interested to find the effect of both Temperature and Altitude on the Current Flow in IC. Two temperature settings: 25oC and 55oC. Two altitude settings: 0K ft. and 3K ft. A

Factorial Arrangement 25oC 55oC 0K ft 210 mA 240 mA 3K ft 180 mA 200 mA Notations for Treatment Combinations Factor A Factor B Low (0) High (1) A0B01 A1B0a A0B1b A1B1ab

Normal Order 22 Factorial Experiments: 1, a, b, ab. 1, a, b, ab, c, ac, bc, abc. 24 Factorial Experiments: 1, a, b, ab, c, ac, bc, abc, d, ad, bd, abd, cd, acd, bcd, abcd.

Effect of a Factor Factor A Factor B Low (0) High (1) T1 T2 T3 T4 EA ={(T2- T1)+ (T4- T3)}/2 =(1/2){ -T1+T2-T3+T4 } CA= -T1+T2-T3+T4 EB ={(T3- T1)+ (T4- T2)}/2 =(1/2){ -T1-T2+T3+T4 } CB= -T1-T2+T3+T4 EA B={(T4- T3)- (T2- T1)}/2 =(1/2){ T1-T2-T3+T4 } CAB= T1-T2-T3+T4

Orthogonal Table Linear Contrast Treatment A B AB Total (1) -1 +1 T1 a

Orthogonal Table (22 Factorial)

23 Factorial Experiments (3 Factors, 2 Levels each) Yijkl =  + Ai + Bj + ABij + Ck+ ACik+BCjk+ABCijk+l(ijk) i=1, 2 j=1, 2 k=1, 2 l=1, 2, …, n

Notations for Treatment Combinations Factor C (0) Factor A Factor B Low (0) High (1) A0B0C01 A1B0C0a A0B1C0b A1B1C0ab Factor C (1) Factor A Factor B Low (0) High (1) A0B0C1c A1B0C1ac A0B1C1bc A1B1C1abc

Orthogonal Table Linear Contrast Trmt A B C AB AC BC ABC Total (1) -1 +1 T1 a T2 b T3 ab T4 c T5 ac T6 bc T7 abc T8

Orthogonal Table (23 Factorial)

2f Factorial Experiments (f Factors, 2 Levels each)

Fraction Factorial Experiments Study only a fraction or subset of all the possible combinations To reduce the total number of experiments to a practical level Different designs exist

Fraction Factorial Experiments Plakett-Burman Screening Design Basic 8 run Plakett-Burman Design A B C D E F G 1 - 2 + 3 4 5 6 7 8

Fraction Factorial Experiments Plakett-Burman Screening Design Basic 12 run Plakett-Burman Design A B C D E F G H I J K 1 - 2 + 3 4 5 6 7 8 9 10 11 12

Fraction Factorial Experiments Taguchi Design B C D E F G 1 - 2 + 3 4 5 6 7 8