1. AP Statistics Experiments and Observational Studies
Chapter 13 Part 2

2. Objectives: Observational study Retrospective study Prospective study
Experiment
Experimental units
treatment
response
Factor
Level
Principles of experimental design
Statistically significant
Control group
Blinding
Placebo
Blocking
Matching
Confounding

3. Designing an Experiment Step-By-Step
Completely Randomized Experiment
(the ideal simple design)
Step 1: Choose treatments
Identify factors and levels
Control group
Step 2: Assign the experimental units to the treatments
Matching (similar units in each group)
Randomization

4. Designing an Experiment
Experimental units
Specify the experimental units.
Experimental Design
Observe the 4 principles of experimental design:
Control – any sources of variability you know of and can control.
Randomly – assign experimental units to treatments, to equalize the effects of unknown or uncontrollable sources of variation. Specify how the random numbers needed for randomization will be obtained.
Replicate – results by placing sufficient experimental units in each treatment group.
Blocking – if required, group similar individuals together.

5. Designing an Experiment
Specify any other experiment details
Give enough details so that another experimenter could exactly replicate your experiment.
How to measure the response.

6. Randomized Comparative Experiment Example:
Researchers believe that diuretics may be as effective in reducing a person’s blood pressure as the conventional drug (drug A), which is much more expensive and has more unwanted side effects. Design a randomized comparative experiment to test this hypothesis.

7. Randomized Comparative Experiment Example:
Explanatory Variable
Type of Medication
Diuretic
Treatments
Drug A
Response Variable
Change in Blood Pressure

8. Randomized Comparative Experiment Example:
# Subjects
(Measure BP)
SRS
Group #1
(# subjects)
Group #2
Treatment 1
Diuretic
Treatment 2
Drug A
Response
Measure BP
Compare
effects of
treatments
Comments: Blinding, other aspects of experiment not in flow chart, etc.

9. Randomized Comparative Experiment Your Turn:
Can chest pain be relieved by drilling holes in the heart? Since 1980, surgeons have been using a laser procedure to drill holes in the heart. Many patients report a lasting and dramatic decease in chest pain. Is the relief due to the procedure or is it a placebo effect?
Design a randomized comparative experiment, using a group of 298 volunteers with severe chest pain, to test this procedures effectiveness.

10. The Best Experiments… are: randomized. comparative. double-blind.
placebo-controlled.

11. Other Experimental Designs
Block Design
Matched Pairs Design

12. Blocking
When groups of experimental units are similar, it’s often a good idea to gather them together into blocks.
Blocking isolates the variability due to the differences between the blocks so that we can see the differences due to the treatments more clearly.
In effect, we are conducting two parallel experiments. We use blocks to reduce variability so that we can see the effect of the treatments. The blocks themselves are not treatments.
When randomization occurs only within the blocks, we call the design a randomized block design.

13. Blocking Blocks are another form of control.
Blocking is the same idea for experiments as stratifying is for sampling.
Both methods group together subjects that are similar and randomize within those groups as a way to remove unwanted variation.
We use blocks to reduce variability so we can see the effects of the factors; we’re not usually interested in studying the effects of the blocks themselves.

14. Blocking
Blocking is the same idea for experiments as stratifying is for sampling.
Both methods group together subjects that are similar and randomize within those groups as a way to remove unwanted variation.
We use blocks to reduce variability so we can see the effects of the factors; we’re not usually interested in studying the effects of the blocks themselves.

15. Block Design – Example:
Suppose the researchers in our Diuretics vs. Drug A example have reason to believe that men and women respond differently to blood pressure medication. Then gender would be the blocking variable.
Our goal is to be able to assess a cause-and-effect relationship between the treatment imposed and the response variable. Blocking reduces variability so that the differences we see can be attributed to the treatment that we imposed. Blocking is to experimental design as stratifying is to sampling design.

16. Block Design – Example:
Females
SRS
Group #1
(# subjects)
Group #2
Treatment 1
Diuretic
Treatment 2
Drug A
Response
Measure BP
Compare
effects of
treatments
Males
Block by gender
# Subjects
(measure BP)
Double-Blind: Both the subjects and the evaluators don’t know how the subjects have been allocated to treatment groups.

17. Block Design – Your Turn:
The progress of a type of cancer differs in women and men. Design a clinical experiment to compare 3 different therapies for this cancer using a subject pool made up of 80 men and 60 women (140 total subjects).

18. Experimental Design – Example:
An ad for OptiGro plant fertilizer claims that with this product you will grow “juicier, tastier” tomatoes. You’d like to test this clam, and wonder whether you might be able to get by with half the specified dose.
How can you set up an experiment, using 24 tomato plants from a garden store, to check out the claim?

19. Experimental Design – Example:
Completely randomized experiment in one factor (three levels)

20. Experimental Design – Example:
Suppose we wanted to use 18 tomato plants of the same variety for our experiment, but the garden store had only 12 plants left. So we drove down to the nursery and bought 6 more plants of that variety. We worry that the tomato plants from the two stores are different somehow, and, in fact, they don’t really look the same.
How can we design the experiment so that the differences between the stores don’t mess up our attempts to see differences among fertilizer levels?

21. Experimental Design – Example:
Randomized block design (block by store) in 1 factor (3 levels)

22. Adding More Factors
It is often important to include multiple factors in the same experiment in order to examine what happens when the factor levels are applied in different combinations.

23. Experimental Design – Example:
There are two kinds of gardeners. Some water frequently, making sure that the plants are never dry. Others let Mother Nature take her course and leave the watering to her. The makers of OptiGro want to ensure that their product will work under a wide variety of watering conditions. Maybe we should include the amount of watering as part of our experiment.

24. Experimental Design – Example:
Completely randomized two-factors, 3 levels experiment (6 treatments)

25. Matching
In a retrospective or prospective study, subjects are sometimes paired because they are similar in ways not under study.
Matching subjects in this way can reduce variability in much the same way as blocking.
Example:
A retrospective study of music education and grades might match each student who studies an instrument with someone of the same gender who is similar in family income but didn’t study an instrument.
When we compare grades of music students with non-music students, the matching would reduce the variation due to income and gender differences.

26. Matched Pairs Design A simple and common special type of block design.
Two types – One Subject or Two Subjects
Conditions
Compare only 2 treatments.
Each block consists of just 2 units, as closely matched as possible (two subjects).
Units are assigned at random to the treatments.
Each block may consist of one subject who gets both treatments one after the other.
Each subject serves as their own control.
The order of the treatments can influence the subject’s response, so the order is randomized for each subject.

27. Matched Pairs Design
One Subject: A common form of matched pairs design uses just one subject who receives both treatments. The order in which the subject receives the treatments is randomized.
Example: 1) Cola taste test – Matched Pairs
Each subject compares two colas (Pepsi/Coke) and picks the one they prefer.
The order in which they taste the colas is randomized.

28. Matched Pairs Design
Example: 2) A researcher believes that students are able to concentrate better while listening to classical music. To test this theory she plans to record the time it takes a student to complete a puzzle maze while listening to classical music and the time it takes him/her to complete another puzzle of the same difficulty level in a quiet room. Because there is so much variability in problem-solving abilities among students, a matched pairs design will be used to reduce this variability so that any difference recorded can be attributed to the conditions under which the student completed the puzzle.
Design - Each student will complete a puzzle under each of the conditions. A coin will be flipped to determine whether the task will be done in a quiet room first or while listening to classical music. The difference in the time it takes to complete each puzzle (Quiet-Music) is recorded for each student.

29. Matched Pairs Design
Two Subjects: The two subjects are paired based on common characteristics that might affect the response variable. One subject from each pair is randomly assigned to each of the treatment groups. The response variable is then the difference in the response to the two treatments for each pair.

30. Matched Pairs Design
Example: Marathon runners are matched by weight, physical build, and running times. They are asked to test the design of a new running shoe compared to the manufacturer’s old design for durability through a race. A coin is tossed to determine which runner in each pair will wear the new design. After the marathon the difference in wear pattern for each pair of runners is then measured and recorded.

31. Confounding
An experiment is said to be confounding if we cannot separate the effect of a factor or treatment (explanatory variable) from the effects of other influences (confounding variables) on the response variable.
Example: When the levels of one factor are associated with the levels of another factor, we say that these two factors are confounded.
When we have confounded factors, we cannot separate out the effects of one factor from the effects of the other factor.
In the lab, we try to avoid confounding by rigorously controlling the environment of the experiment so that nothing except the experimental treatment influences the response.

32. Lurking or Confounding
A lurking variable creates an association between two other variables that tempts us to think that one may cause the other.
This can happen in a regression analysis or an observational study.
A lurking variable is usually thought of as a prior cause of both y and x that makes it appear that x may be causing y.

33. Lurking or Confounding
Confounding can arise in experiments when some other variables associated with a factor has an effect on the response variable.
Since the experimenter assigns treatments (at random) to subjects rather than just observing them, a confounding variable can’t be thought of as causing that assignment.
A confounding variable, then, is associated in a noncausal way with a factor and affects the response.
Because of the confounding, we find that we can’t tell whether any effect we see was caused by our factor or by the confounding factor (or by both working together).

34. Lurking or Confounding
Association may be the result of any of several different types of relationships.
Causation
Common Response (lurking variable)
Confounding
association – dashed line,
causation – solid arrow