1. Designing Experiments
Chapter 2
Designing Experiments 1

2. Definitions:
1) Observational study - observe outcomes without imposing any treatment
2) Experiment - actively impose some treatment in order to observe the response

3. 100% of daily vitamins & essential oils!
I’ve developed a new rabbit food, Hippity Hop.
Makes fur soft & shiny!
Hippity Hop
Rabbit Food
Increases energy!
100% of daily vitamins & essential oils!

4. Can I just make these claims?NO
What must I do to make these claims?
Do an experiment
Who (what) should I test this on?
Rabbits
What do I test?
The type of food

5. 3)Experimental unit – the single individual (person, animal, plant, etc.) to which the different treatments are assigned
4) Factor – is the explanatory variable – it’s what we test
5) Level – a specific value for the factor

6. 6) Response variable – what you measure
7) Treatment – a specific experimental condition applied to the units

7. I plan to test my new rabbit food. What are my experimental units?
What is my factor?
What is the response variable?
Rabbits
Type of food
How well they grow

8. I’ll use my pet rabbit, Lucky!
Hippity Hop
Discuss the need for a comparison group.
Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food!

9. 8) Control group – a group that is used to compare the factor against; can be a placebo or the “old” or current item
9) Placebo – a “dummy” treatment that can have no physical effect

10. WOW! Lucky is bigger & shinier so Hippity Hop is better!
Old Food
Hippity Hop
Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old rabbit food.
Discuss the need for replication.
WOW! Lucky is bigger & shinier so Hippity Hop is better!

11. The Hippity Hop rabbits have scored higher so it’s the better food!
Old Food
Hippity Hop
The first five rabbits that I catch will get Hippity Hop food and the remaining five will get the old food.
Discuss the need for random assignment to treatment groups.
The Hippity Hop rabbits have scored higher so it’s the better food!

12. Old Food
Hippity Hop
Number the rabbits from 1 – 10.
Place the numbers in a hat.
The first five numbers pulled from the hat will be the rabbits that get Hippity Hop food.
The remaining rabbits get the old food. 5 8 7 3 9 6
Discuss the need for the evaluator to be blinded 2 4 5
I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food! 10 1 9 7 3 8

13. 10) blinding - method used so that units do not know which treatment they are getting
11) double blind - neither the units nor the evaluator know which treatment a subject received

14. Hippity Hop Rabbit Food makes fur soft and shiny, & increases energy for ALL types of rabbits!
Discuss scope of inference.
Can I make this claim?

15. Definition:
Confounding variable – A variable whose effect on the response cannot be separated or distinguished from the effects of the factor on the response.
- a variable that is related to both group membership and the response variable of interest in a research study
Because observational studies may contain confounding variables, their results can NOT be used to show cause-effect relationships.

16. Principles of Experimental Design
Control the effects of extraneous variables on the response
Randomization – the use of chance to assign subjects to treatments
Replication of the experiment on many subjects to quantify the natural variation in the experiment

17. The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!!

18. Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with two different types of fertilizers. Sixteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.
Experimental units?
Factors?
Levels?
Response variable?
How many treatments?
Plots of land
Type of fertilizer
Fertilizer types A & B
Yield of crop 2

19. Suppose the researcher decides to fertilizer the 8 plots closest to the river with fertilizer A and the remaining plots with B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer?
Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. A B
NO! The amount of water or soil type is confounded with the type of fertilizer!

20. Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens.
Experiment units?
Factors?
Levels?
Response variable?
Number of treatments?
Cake batter
Two factors - type of pan & type of oven
Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas)
How evenly the cake bakes 6

21. Why is the same type of seed used on all 15 plots?
Example 3: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.
Why is the same type of seed used on all 15 plots?
What are other potential extraneous variables?
Does this experiment have a placebo? Explain
To control the factor of type of seed.
Type of soil; amount of water, sunlight, etc.
No, one would compare the three types of fertilizers
It is part of the controls in the experiment.
Type of soil, amount of water, etc.
NO – a placebo is not needed in this experiment

22. Experimental Designs
Completely randomized design –experimental units are assigned at random to treatments
Talk through the process of the experimental design.
Treatment A
Measure response for A
Experimental Units
Random Assignment
Compare treatments
Treatment B
Measure response for B

23. Suppose the researcher decides to randomly assign fertilizer A to 8 plots and the remaining plots with B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer?
Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. B A
Yes! The effects of water and soil type are evenly spread into both treatments. Thus there is no confounding.

24. Experimental Designs Continued . . .
Units should be blocked on a variable that effects the response!!!
2. Randomized block – units are blocked into groups (homogeneous) and then randomly assigned to treatments
Treatment A
Measure response for A
Block 1
Random Assignment
Compare treatments for block 1
Treatment B
Measure response for B
Compare the results from the 2 blocks
Experimental Units
Create blocks
Treatment A
Measure response for A
Block 2
Random Assignment
Compare treatments for block 2
Treatment B
Measure response for B

25. Then the researcher randomly assigns fertilizer A to 4 plots in the red block and to 4 plots in the blue block. The remaining plots receive fertilizer B. He has a greater yield with fertilizer A. Does this mean that A is the better fertilizer?
Suppose the researcher decides to block the 16 plots into two blocks based on their proximity to the river.
Suppose that the sixteen plots of land are not similar. Some are close to a river with trees along the bank, while others are farther away from the river. See the diagram below. A B
Yes! Blocking is a way to “control” the effects of extraneous variables.

26. Matched pairs - a special type of block design
match up experimental units according to similar characteristics & randomly assign one to one treatment & the other automatically gets the 2nd treatment
have each unit do both treatments in random order
the assignment of treatments is dependent

27. Pair experimental units according to specific characteristics.
Treatment A
Treatment B
Next, randomly assign one unit from a pair to Treatment A. The other unit gets Treatment B.
Pair experimental units according to specific characteristics.
This is one way to do a matched pairs design – another way is to have the individual unit do both treatments (as in a taste test).

28. Consider the following headlines which occurred on September 25, 2009.
These headlines imply that spanking is the CAUSE of the observed difference in IQ.
Is this conclusion reasonable?
Consider the following headlines which occurred on September 25, 2009.
“Spanking lowers a child’s IQ” (Los Angeles Times)
“Do you spank” Studies indicate it could lower your kids’ IQ.” (SciGuy, Houston Chronicle)
“Spanking can lower IQ” (NBC4i, Columbus, Ohio)
“Smacking hits kids’ IQ” (newscientist.com)
In this study, two groups of children were followed for 4 years; 806 children ages 2 to 4 and 704 children ages 5 to 9. IQ was measured at the beginning of the study and again four years later. Researchers found that the average IQ of children, ages 2 to 4, who were not spanked was 5 points higher than those who were spanked and 2.8 points higher for children, ages 5 to 9.

29. Spanking and IQ Continued . . .
In this study, two groups of children were followed for 4 years; 806 children ages 2 to 4 and 704 children ages 5 to 9. IQ was measured at the beginning of the study and again four years later. Researchers found that the average IQ of children, ages 2 to 4, who were not spanked was 5 points higher than those who were spanked and 2.8 points higher for children, ages 5 to 9.
Does spanking “CAUSE” a decrease in IQ? Why or why not?
Are there other variables connected to the response (decreased IQ) and the groups of children?
These are called confounding variables.
Have students consider variables like socio-economic status, education level of parent, home or school environments

30. Treatment & group are confounded
Treatment B
Treatment A
One group is assigned to treatment A & the other group to treatment B.
Treatment A
Treatment B
Confounding does NOT occur in a completely randomized design!
Treatment & group are confounded

31. Is this an experiment? Why or why not?
Example 4: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed.
Is this an experiment? Why or why not?
What is a potential confounding variable?
No, no treatment was imposed on people.
Age – younger people are more at risk to be victims of violent crimes

32. Is this an experiment? Why or why not?
Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured.
Is this an experiment? Why or why not?
Yes, a treatment is imposed.
Yes, a treatment was imposed
Completely randomized
one factor, word processing program & 4 levels, the four new programs
Speed at which standard tasks can be done
What type of design is this?
Factors? Levels?
Response variable?
Completely randomized
one factor: word-processing program with 4 levels
speed

33. Can this design be improved? Explain.
Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly designed to one of the four programs and their speeds are measured.
Is there a potential confounding variable?
You could do a block design where each person uses each program in random order.
a) Speed/expertise of each individual
b) Use a matched pairs design where each volunteer uses all four programs in random order
Can this design be improved? Explain.
NO, completely randomized designs have no confounding

34. What type of design is this? Why use this method?
Example 6: Suppose that the manufacturer wants to test a new fertilizer against the current one on the market. Ten 2-acre plots of land scattered throughout the county are used. Each plot is subdivided into two subplots, one of which is treated with the current fertilizer, and the other with the new fertilizer. Wheat is planted and the crop yields are measured.
What type of design is this? Why use this method?
When does randomization occur?
Matched - pairs design
Randomly assigned treatment to first acre of each two-acre plot

35. Blocking also helps reduce variability.
Randomization reduces bias by spreading any uncontrolled confounding variables evenly throughout the treatment groups.
Is there another way to reduce variability?
Blocking also helps reduce variability.
Bias is a systematic error in measuring the estimate
Variability is controlled by sample size. Larger samples produce statistics with less variability.

36. High bias & high variability
High bias & low variability
Low bias & high variability
Low bias & low variability