1. Designing Experiments1

2. I plan to test my new rabbit food.
Back to my rabbit food:
I plan to test my new rabbit food.
What are my experimental units?
What is my factor?
What is the response variable?
Rabbits
Type of food
How well they grow

3. I’ll use my pet rabbit, Lucky!
Hippity Hop
Discuss the need for a comparison group.
Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food! Done, right?

4. Control group – a group that is used to provide a “baseline” for comparing the factor against; can be a placebo or the “old” or current item
Placebo – a “dummy” treatment that can have no physical effect

5. WOW! Lucky is bigger & shinier so Hippity Hop is better!
Old Food
Hippity Hop
Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old rabbit food.
Discuss the need for replication.
WOW! Lucky is bigger & shinier so Hippity Hop is better!

6. Is performing my experiment on 2 rabbits enough?
Replication of the experiment on many subjects to quantify the natural variation in the experiment

7. The Hippity Hop rabbits have scored higher so it’s the better food!
Old Food
Hippity Hop
The first five rabbits that I catch will get Hippity Hop food and the remaining five will get the old food.
Discuss the need for random assignment to treatment groups.
The Hippity Hop rabbits have scored higher so it’s the better food!

8. Which rabbits got the new food?????? Is that fair?
Randomization – the use of chance to assign subjects to treatments. Balances the effects of lurking variables.

9. Old Food
Hippity Hop
Number the rabbits from 1 – 10.
Place the numbers in a hat.
The first five numbers pulled from the hat will be the rabbits that get Hippity Hop food.
The remaining rabbits get the old food. 5 8 7 3 9 6
Discuss the need for the evaluator to be blinded 2 4 5
I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food! 10 1 9 7 3 8

10. Hippity Hop Rabbit Food makes fur soft and shiny, & increases energy for ALL types of rabbits!
Discuss scope of inference.
Can I make this claim?

11. The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!!

12. Three Principles of Experimental Design
Control for lurking variables that might affect the response: Use a comparative design and ensure that the only systematic difference between the groups is the treatment administered.
2. Random assignment: Use impersonal chance to assign experimental units to treatments. This helps create roughly equivalent groups of experimental units by balancing the effects of lurking variables that aren’t controlled on the treatment groups.
3. Replication: Use enough experimental units in each group so that any differences in the effects of the treatments can be distinguished from chance differences between the groups.

13. Experimental design: Completely Randomized
Definition:
In a completely randomized design, the treatments are assigned to all the experimental units completely by chance.
Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment.
Group 1
Group 2
Treatment 1
Treatment 2
Compare Results
Experimental Units
Random Assignment

14. Experiment Designs
Completely randomized – all experimental units are allocated at random among all treatments
Hat method
Random digit table
Random number generator
Random assignment

15. Completely randomized design
Treatment A
Treatment B
Treatment C
Treatment D
Randomly assign experimental units to treatments
Completely randomized design

16. Dueling Diets
A health organization wants to know if a low-carb or low-fat diet is more effective for long-term weight loss. The organization decides to conduct an experiment to compare these two diet plans with a control group that is only provided with brochure about healthy eating. Ninety volunteers agree to participate in the study for one year.
Problem: Outline a completely randomized design for this experiment. How you would implement your design?
90 volunteers
Random
assignment
Group 1
(30 subjects)
Group 2
Group 3
Treatment 1
(low-carb)
Treatment 2
(low-fat)
Treatment 3
(control)
Compare
weight
loss

17. 90 volunteers
Random
assignment
Group 1
(30 subjects)
Group 2
Group 3
Treatment 1
(low-carb)
Treatment 2
(low-fat)
Treatment 3
(control)
Compare
weight
loss
To implement the design, use 90 equally sized slips of paper. Label 30 of the slips “1”, 30 of the slips “2” and 30 of the slips “3”. Then, mix them up in a hat and have each subject draw a number without looking. The number that each subject chooses will be the group he or she is assigned to. At the end of the year, the amount of weight loss will be recorded for each subject and the mean weight loss will be compared for the three treatments.

18. Does regularly taking aspirin help protect people against heart attacks?  
The Physicians’ Health Study looked at the effects of two drugs: aspirin and beta-carotene. Researchers wondered whether beta-carotene would help prevent some forms of cancer. The subjects in this experiment were 21,996 male physicians. There were two explanatory variables (factors), each having two levels: aspirin (yes or no) and beta-carotene (yes or no). Combinations of the levels of these factors form the four treatments. One-fourth of the subjects were assigned at random to each of these treatments.
On odd-numbered days, the subjects took either a tablet that contained aspirin or a dummy pill that looked and tasted like the aspirin but had no active ingredient (a placebo). On even-numbered days, they took either a capsule containing beta-carotene or a placebo. There were several response variables—the study looked for heart attacks, several kinds of cancer, and other medical outcomes. After several years, 239 of the placebo group but only 139 of the aspirin group had suffered heart attacks. This difference is large enough to give good evidence that taking aspirin does reduce heart attacks.23 It did not appear, however, that beta-carotene had any effect on preventing cancer.
A placebo is a “dummy pill” or inactive treatment that is indistinguishable from the real treatment.

19. Explain how each of the three principles of experimental design was used in the Physicians’ Health Study.
Researchers attempted to control for the effects of lurking variables by using a design that compared both of the active treatments to a placebo and by having all subjects follow the same schedule of pill taking. 
Random assignment was used to determine which subjects received each of the four treatment combinations. This helped ensure that the treatment groups were roughly equivalent to begin with. 
There were over 5000 subjects per treatment group. This replication helped ensure that any sizable difference in response among the groups was due to the treatments