1. Stat 470-11 Stat 470-1 Introduction to the Design of Experiments Instructor: Derek Bingham, Office: West Hall 451 Contact Information: Email: dbingham@umich.edu Phone: (734) 763-9294 Office hours: West Hall 443 Tuesday & Thursday, 12:30-2:00; others by appointment Text: Experiments: Parameter Design and Optimization by Wu and Hamada

2. Stat 470-12 Stat 470 – Overview/Syllabus Coverage: Review of linear regression; most of Chapters 1-4, additional topics as needed, and Chapter 9 if time permits Course notes will be available on the web by 12:00 day of class…otherwise as handouts in class (www.stat.lsa.umich.edu/~dbingham/Stat470) Term Project –design, conduct, analyze, report on an experiment –Will be given out later

3. Stat 470-13 Computing –SPSS –Any other package you like

4. Stat 470-14 What is Experimental Design? Experiments performed in almost all fields of study Experiment is conducted to learn something about a process or system Designed experiment is a series of tests (experiments) where changes are made in the inputs to observe and identify the impact on the output Better understanding of how the factors impact the system allows the experimenter predict future values or optimize the process

5. Stat 470-15 Can consider a process as: Inputs  Process  Output The input variables (usually called variables or factors) will be denoted will be denoted x 1, x 2,…, x p. The output variable (often called the response variable) will be denoted will be denoted by y.

6. Stat 470-16 Example (Tomato Fertilizer) Experiment was conducted by a horticulturist Has 2 types of fertilizer available for tomato production (A and B) Objective: Is one fertilizer better than the other – higher yield, on average? Has 11 tomato plots Experiment Procedure - specify fertilizer amounts each fertilizer; decide upon number of pots to receive each fertilizer; randomly assign fertilizer to pots Response: yield – pounds of tomatoes

7. Stat 470-17 Some Definitions Factor: variable whose influence upon a response variable is being studied in the experiment Factor Level: numerical values or settings for a factor Treatment or level combination: set of values for all factors in a trial Experimental unit: object, to which a treatment is applied Trial: application of a treatment to an experimental unit Replicates: repetitions of a trial Randomization: using a chance mechanism to assign treatments to experimental units

8. Stat 470-18 What is an Experiment Design? Suppose you are going to conduct an experiment with 8 factors Suppose each factor has only to possible settings How many possible treatments are there? Suppose you have enough resources for 32 trials. Which treatments are you going to perform? Design: specifies the treatments, replication, randomization, and conduct of the experiment

9. Stat 470-19 Types of Experiments Treatment Comparisons: Purpose is to compare several treatments of a factor (have 3 diets and would like to see if they are different in terms of effectiveness) Variable Screening: Have a large number of factors, but only a few are important. Experiment should identify the important few. (we will focus on these!) Response Surface Exploration: After important factors have been identified, their impact on the system is explored to optimize response

10. Stat 470-110 Types of Experiments System Optimization: Often interested in determining the optimum conditions (e.g., Experimenters often wish to maximize the yield of a process or minimize defects) System Robustness: Often wish to optimize a system and also reduce the impact of uncontrollable (noise) factors. (e.g., would like a fridge to cool to a set temperature…but the fridge must work in Florida, Alaska and Michigan!)

11. Stat 470-111 Systematic Approach to Experimentation State the objective of the study Choose the response variable…should correspond to the purpose of the study Choose factors and levels Choose experiment design (purpose of this course) Perform the experiment Analyze data (design should be selected to meet objective and so analysis is efficient and easy) Draw conclusions

12. Stat 470-112 Observation vs. Experimentation Data collection is not experimentation By observation, you can learn that lightning can cause fires By experimentation, you can learn that friction between certain materials can cause fires By more experimentation, we have learned how to make fire-starting reliable, cheap, easy, … By experimentation, we learn more and we learn faster

13. Stat 470-113 The Need for Experimentation A doctor has impression that persons he gives Medicine A to recover more quickly than persons he gives Medicine B to. But, apparent difference could be due to: –luck, random variation, small sample sizes, … –bias in choice of medicine for patients –physical differences in people receiving A vs. B age, weight, sex, prior health, …. Clinical trials (experiments that control extraneous sources of variation and bias) are required to get a scientific assessment of Medicine A vs. Medicine B

14. Stat 470-114 Three Principles Replication – each treatment is applied to experimental units that are representative of the population of interest –independent repetition of a trial –provides a measure of “noise,” meaning: experimental error -- the variability of experimental units which receive the same treatment –experimental error is the yardstick against which we compare different treatments –increasing number of replicates decreases variance of treatment effects and increases the power to detect significant differences –Replication provides a measure of experimental “noise” and the means for controlling the level of that noise. (More replication means less noise in averages.)

15. Stat 470-115 Warning! Sometimes what looks like replication is not! Repeat measurements on one experimental unit is not replication Measurements on multiple samples from one experimental unit is not replication Example: two cake recipes. –Make one batch by recipe A; one batch by recipe B –Bake 12 cupcakes from each batch; measure fluffiness –The experimental unit is a batch; there has been no replication of either recipe A or recipe B there is no valid comparison of recipe A to recipe B; apparent difference could be random batch differences

16. Stat 470-116 Principle 2 - Randomization 2. Randomization -- use of a chance mechanism (e.g., random number generator) to assign treatments to experimental units or to the sequence of experiments –provides protection against unknown lurking variables –help justify the assumption of “independence” that will underlie many analyses

17. Stat 470-117 Principle 3 - Blocking 3. Blocking -- run groups of treatments on homogenous units (block) to reduce variability of effect estimates and have more fair comparisons –Example: To compare 4 varieties of corn, an experimenter could consider blocks of land of various soil types and terrain, subdivide each block into plots, and randomly assign the 4 varieties to plots in a block. –Blocking: controls variability due to soil types and terrain and allows varieties to be compared within blocks broadens the scope of conclusions, e.g., by including variety of soil types and terrain in the experiment

18. Stat 470-118 Case Study: Reliability of Wire Bonding on Integrated Circuits Process monitoring: –Sample ICs selected, pull- tested Available data: –pull strengths from 1000s of pull tests Success criterion: –pull strength > 2.5g Planned analysis: –fit a distribution to all the data –estimate reliability

19. Stat 470-119 Designed Experiment to the Rescue Experimental Design: –3 bonding operators –3 bonding machines –3 pull-test operators –2 IC packages per combination –48 wires per IC package –All combinations = 2,592 observations (!) Note: Unusually large experiment, but feasible in this case – many defective IC’s available and processing time is short

20. Stat 470-120 Case Study (cont.) Analysis Findings: –No appreciable difference among bonding machines –Large and independent effects of bonding and pull-test operators. NOT GOOD! Ave. Pull Strength - each the ave. of 288 observations.(grams) Pull Test Operator A B C Bond. A 8.46.37.0 Op B 9.06.87.6(noise std dev = 1.5 grams) C 7.15.35.8 Further conjectures and experiments led to improved consistency of manufacturing and testing techniques

21. Stat 470-121 Case Study -- Messages People and procedures can have more of an influence on quality than machines. Think about possible sources of variability Use designed experiments to control and evaluate these sources of variability

22. Stat 470-122 Assignment Review t-tests (2-sample and paired) Review Linear regression Review ANOVA These will be the fundamental analysis tools for this course.