1. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 1 of 34 Chapter 1 Section 5 The Design of Experiments

2. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 2 of 34 Chapter 1 – Section 5 ●Learning objectives  Define designed experiment  Understand the steps in designing an experiment  Understand the completely randomized design  Understand the matched-pairs design 1 2 3 4

3. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 3 of 34 Chapter 1 – Section 5 ●Learning objectives  Define designed experiment  Understand the steps in designing an experiment  Understand the completely randomized design  Understand the matched-pairs design 1 2 3 4

4. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 4 of 34 Chapter 1 – Section 5 ●Data can be collected in two main ways  Through sample surveys  Through designed experiments ●Sample surveys lead to observational studies ●Designed experiments enable researchers to control variables, leading to additional conclusions

5. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 5 of 34 Chapter 1 – Section 5 ●A designed experiment is a controlled study ●The purpose of designed experiments is to control as many factors as possible to isolate the effects of a particular factor ●Designed experiments must be carefully set up to achieve their purposes

6. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 6 of 34 Chapter 1 – Section 5 ●Some variables in a designed experiment are controlled, those are the explanatory variables ●These variables are also sometimes called the factors ●Factors  Are part of a controlled environment  Has values that can be changed by the researcher  Are considered as possible causes

7. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 7 of 34 Chapter 1 – Section 5 ●Examples of factors are  The dosage of a drug in a medical experiment  The type of teaching method in an education experiment  One drug by itself compared to that drug used in conjunction with another

8. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 8 of 34 Chapter 1 – Section 5 ●The designed experiment analyzes the affects of the factors on the response variable ●Response variables  Are not part of a controlled environment  Has values that are measured by the researcher  Measure the effects

9. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 9 of 34 Chapter 1 – Section 5 ●Examples of response variables are  The blood pressures of the patients  The test scores for a class  The sizes of a cancerous tumor for patients

10. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 10 of 34 Chapter 1 – Section 5 ●A treatment is a combination of the values of the factors ●Examples of treatments  Giving one medication to one group of patients and a different medication to another  Using one type of fertilizer on a set of plots of corn and a different type of fertilizer on a different set of plots  Playing country music to one group of mice and rap music to another

11. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 11 of 34 Chapter 1 – Section 5 ●The treatment is applied to experimental units (people, plants, materials, other objects, …) ●When the experimental units are people, we refer to them as subjects ●Subjects in an experiment correspond to individuals in a survey

12. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 12 of 34 Chapter 1 – Section 5 ●An example of a designed experiment is to determine whether a new drug, Drug N, is more effective at treating high blood pressure than the existing drug, Drug E ●Patients with high blood pressure are given either Drug N or Drug E ●The blood pressures are measured one month later

13. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 13 of 34 Chapter 1 – Section 5 ●For this experiment  Factor – the type of drug  Response variable – blood pressure  Treatment – given Drug N or Drug E  Experimental units / subjects – the patients ●For this experiment  Factor – the type of drug  Response variable – blood pressure  Treatment – given Drug N or Drug E  Experimental units / subjects – the patients ●If patients given Drug N have significantly lower blood pressures than patients given Drug E, we would wish to conclude that Drug N is more effective

14. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 14 of 34 Chapter 1 – Section 5 ●Changes in behavior of subjects ●For an experiment comparing a new drug to no treatment at all  If the subject knows that he or she is given a drug, he or she may feel better (the placebo effect) ●Changes in behavior of subjects ●For an experiment comparing a new drug to no treatment at all  If the subject knows that he or she is given a drug, he or she may feel better (the placebo effect) ●For an experiment comparing a new drug to an existing drug  If the subject knows which drug he or she is given, that may change his or her behavior

15. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 15 of 34 Chapter 1 – Section 5 ●To avoid the effects of subject behavior  Subjects not given any medication are often given a placebo such as a sugar tablet  The subjects will not know which treatment they get ●To avoid the effects of subject behavior  Subjects not given any medication are often given a placebo such as a sugar tablet  The subjects will not know which treatment they get ●To avoid the effects of researcher behavior  The researchers are not told which drug they are administering ●To avoid the effects of subject behavior  Subjects not given any medication are often given a placebo such as a sugar tablet  The subjects will not know which treatment they get ●To avoid the effects of researcher behavior  The researchers are not told which drug they are administering ●When both the subjects and the researchers do not know which treatment, this is called double- blind

16. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 16 of 34 Chapter 1 – Section 5 ●Learning objectives  Define designed experiment  Understand the steps in designing an experiment  Understand the completely randomized design  Understand the matched-pairs design 1 2 3 4

17. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 17 of 34 Chapter 1 – Section 5 ●Conducting an experiment involves considerable planning ●Planning steps ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors  Determine the number of experimental units ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors  Determine the number of experimental units  Determine the level of each factor ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors  Determine the number of experimental units  Determine the level of each factor ●Implementation steps ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors  Determine the number of experimental units  Determine the level of each factor ●Implementation steps  Conduct the experiment ●Conducting an experiment involves considerable planning ●Planning steps  Identify the problem  Determine the factors  Determine the number of experimental units  Determine the level of each factor ●Implementation steps  Conduct the experiment  Test the claim

18. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 18 of 34 Chapter 1 – Section 5 ●Identify the problem ●The first step in planning an experiment (or in most any project at all) is to identify the problem ●The identification would include  The general purpose of the experiment  The response variable  The population ●This is also referred to as the claim

19. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 19 of 34 Chapter 1 – Section 5 ●Determine the factors ●The second step in planning an experiment is to determine the factors to be studied ●The factors could be identified  By subject matter experts  By the purpose of the experiment  Using results from previous studies ●Factors must be identified as either fixed, controlled, or uncontrolled

20. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 20 of 34 Chapter 1 – Section 5 ●Determine the number of experimental units ●In general, the more the experiment units, the more effective the experiment ●The number of experimental units  Could be limited by time  Could be limited by money ●There are techniques to calculate the number of experimental units (to be covered later)

21. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 21 of 34 Chapter 1 – Section 5 ●Determine the level of each factor ●Three ways to deal with the factors ●Determine the level of each factor ●Three ways to deal with the factors  Control – fix the levels at a constant level (for factors not of interest) ●Determine the level of each factor ●Three ways to deal with the factors  Control – fix the levels at a constant level (for factors not of interest)  Manipulate – set the levels at predetermined levels (for factors of interest) ●Determine the level of each factor ●Three ways to deal with the factors  Control – fix the levels at a constant level (for factors not of interest)  Manipulate – set the levels at predetermined levels (for factors of interest)  Randomize – randomize the experimental units (for uncontrolled factors not of interest) ●Determine the level of each factor ●Three ways to deal with the factors  Control – fix the levels at a constant level (for factors not of interest)  Manipulate – set the levels at predetermined levels (for factors of interest)  Randomize – randomize the experimental units (for uncontrolled factors not of interest) ●Randomization decreases the effects of uncontrolled factors, even ones not identified

22. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 22 of 34 Chapter 1 – Section 5 ●Conduct the experiment ●The subjects are assigned at random to the treatments  When a treatment is applied to more than one experimental unit, this is called replication  Replication is useful for accuracy, to further decrease the effects of uncontrolled factors ●Collect and process the data

23. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 23 of 34 Chapter 1 – Section 5 ●Test the claim ●This is inferential statistics ●Techniques of inferential statistics are studied in chapters 9 through 14

24. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 24 of 34 Chapter 1 – Section 5 ●Learning objectives  Define designed experiment  Understand the steps in designing an experiment  Understand the completely randomized design  Understand the matched-pairs design 1 2 3 4

25. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 25 of 34 Chapter 1 – Section 5 ●A completely randomized design is when each experimental unit is assigned to a treatment completely at random ●An example  A farmer wants to test the effects of a fertilizer  We choose a set of plants to receive the treatment  We randomly assign plants to receive different levels of fertilizer ●A completely randomized design is when each experimental unit is assigned to a treatment completely at random ●An example  A farmer wants to test the effects of a fertilizer  We choose a set of plants to receive the treatment  We randomly assign plants to receive different levels of fertilizer ●This has similarities to completely random sampling

26. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 26 of 34 Chapter 1 – Section 5 ●We control as many factors as we can  Amount of watering  Method of tilling  Soil acidity ●We control as many factors as we can  Amount of watering  Method of tilling  Soil acidity ●Randomization decreases the effects of uncontrolled factors  Rainfall  Sunlight  Temperature

27. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 27 of 34 Chapter 1 – Section 5

28. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 28 of 34 Chapter 1 – Section 5 ●Learning objectives  Define designed experiment  Understand the steps in designing an experiment  Understand the completely randomized design  Understand the matched-pairs design  Understand the randomized block design 1 2 3 5 4

29. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 29 of 34 Chapter 1 – Section 5 ●A matched-pair design is when the experimental units are paired up and each of the pair is assigned to a different treatment ●A matched pair design requires  Units that are paired (twins, the same person before and after the treatment, …)  Only two levels of treatment (one for each of the pair) ●A matched-pair design is when the experimental units are paired up and each of the pair is assigned to a different treatment ●A matched pair design requires  Units that are paired (twins, the same person before and after the treatment, …)  Only two levels of treatment (one for each of the pair) ●An example  A subject before receiving the medication  The same subject after receiving the medication

30. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 30 of 34 Chapter 1 – Section 5 ●Test whether students learn better while listening to music or not  Match students by IQ and gender (to control those factors)  Randomly choose one of each pair (to decrease the effects of other uncontrolled factors  Assign that one to a quiet room and the other to a room with music (the treatment)  Administer the test and analyze the test scores

31. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 31 of 34 Chapter 1 – Section 5

32. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 32 of 34 Chapter 1 – Section 5 ●An example  We are testing the effects of treatments A, B, and C on soybean plants  Assume that group 1 is treated with A and group 2 is treated with B  Assume that Chemgro plants have higher yields than Pioneer plants  Assume that group 1 has more Chemgro plants (happens because of randomization) than group 2

33. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 33 of 34 Chapter 1 – Section 5 ●If group 1 (treatment A) has higher yields than group 2 (treatment B)  Is this because treatment A is more effective than B?  Is this because there are more Chemgro plants in group 1? ●If group 1 (treatment A) has higher yields than group 2 (treatment B)  Is this because treatment A is more effective than B?  Is this because there are more Chemgro plants in group 1? ●It is not possible to distinguish  The effects of Treatment A versus B  The effects of Chemgro versus Pioneer ●If group 1 (treatment A) has higher yields than group 2 (treatment B)  Is this because treatment A is more effective than B?  Is this because there are more Chemgro plants in group 1? ●It is not possible to distinguish  The effects of Treatment A versus B  The effects of Chemgro versus Pioneer ●When two effects cannot be distinguished, this is called confounding

34. Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 1 Section 5 – Slide 34 of 34 Summary: Chapter 1 – Section 5 ●The planning for designed experiments is crucial to the success of the experiment ●A double-blind implementation of experiments reduces the amount of changes in behavior ●There are different good methods for assigning treatments to experimental units  Completely random  Matched-pairs  Randomized blocks