1. Chapter 13
Experiments and Observational Studies

2. Review Population Parameter Sample Statistic
This is a numerical summary of a population
Proportion, mean, standard deviation
We don’t know this number and usually never will.
Sample Statistic
This is a numerical summary computed from a subset of the population(a sample).
sample proportion, sample mean, sample standard deviation
We know this number because we can compute it. We use this number to try and get an idea of what the population parameter(which we don’t know) might be.

3. Observational Study Observing data in “the wild”.
Researchers don't assign treatments to subjects; they simply observe them.
Results show association, but do not prove causation.
Also called an “Investigative Study”.
Sample surveys are observational studies.

4. Observational Studies
Two types
Retrospective Study
Subjects are selected
Previous behaviors or conditions are determined
Prospective Study
Subjects are followed to observe future outcomes

5. Experiments
A way to prove a cause and effect relationship between two or more variables
Manipulate explanatory variable
Observe response
In an experiment you are applying a treatment to the subjects; in an observational study you just observe them

6. Experiment Terminology
Experimental Units – individuals on whom the experiment is performed
Factors – explanatory variable(s)
Need at least one factor for every experiment

7. Experiment Terminology
Levels – specific values that the experimenter chooses for a factor
Need at least two levels for each factor
Treatments – different levels of a single factor or combinations of levels of two or more factors

8. Example – OptiGro Fertilizer
An ad for OptiGro plant fertilizer claims that with this product we will grow “juicier, tastier” tomatoes. We'd like to test this claim and see if we can get by with a half-dose.
What are the experimental units in this example?
Tomato plants (the thing we apply the treatment to)
What is the response we are interested in?
“Juiciness and Tastiness” of the tomatoes

9. Example – OptiGro Fertilizer
What is the factor?
Fertilizer
What are the levels of this factor?
No Fertilizer (to see if the fertilizer even works)
Full-Dose
Half-Dose (want to see if the half dose is as good as the full)
What are the treatments?
No Fertilizer, Full-Dose, Half-Dose

10. Example – OptiGro Fertilizer
The makers of OptiGro want to ensure that their product will work under a wide variety of watering conditions. Let's include this as a second factor; specifically let’s look at daily watering and watering every other day.
How many factors are there now? 2
What are the factor(s)?
Fertilizer and water
What are factor levels
Fertilizer
no fertilizer, half dose, full dose
Water
every day, every other day

11. Example – OptiGro Fertilizer
What are the treatments in the new experiment?
Half-Dose of fertilizer, daily watering
Full-Dose, daily watering
No fertilizer, daily watering
Half-Dose, water every other day
Full-Dose, water every other day
No fertilizer, water every other day

12. Experiment Terminology
Control Group – experimental units assigned to a baseline treatment level
Provides basis for comparison
Baseline: either a “gold standard” or a placebo
Placebo – a null treatment known to have no effect
Placebo Effect – tendency of many human subjects to show response even when administered a placebo

13. Experiment Terminology
Blinding - the researcher disguises treatments
Single Blind - either the subject or the evaluator is unaware of the treatment
Double Blind - both the subject AND the evaluator are in the dark
Reduces personal bias

14. Experiment Terminology
Confounding - A condition where the effects of two variables on the response can’t be distinguished from each other
Lurking Variable - A variable that effects the relationship between the response variable and the explanatory variable and is not included among the variables in a study.

15. Principles of Experimental Design
Control
Make all conditions as similar as possible for all treatment groups
The only difference between groups should be the treatments
Controlling “outside” influences reduces variation in the responses, making it easier to detect differences among the groups due to the treatments

16. Principles of Experimental Design
Randomization
Experimental units should be randomly assigned to treatment groups
Order of the trials should also be randomized
Some uncontrolled sources of variation will be controlled through randomization
Experiments without randomization may have biased results

17. Principles of Experimental Design
Replication
In an experiment, each treatment is given to several different experimental units
Entire experiment should be repeated on a different group of experimental units

18. Blocking
This can be considered another principle of experimental design, but not required in an experimental design
Group similar individuals together and then randomize within each of these blocks
Reduces the effects of identifiable attributes of the subjects that cannot be controlled

19. Example
The leg muscles of men aged 60 to 75 were 50% to 80% stronger after they participated in a 16-week, high-intensity resistance-training program twice a week.
Experiment
Men aged 60 to (experimental units)
Exercise (1 level) (factors)
1 treatment 16-week resistance-training
Strength levels pre- and post-exercise program (response)
Not blinded
Applies only to men 60 to 75 who participate in similar exercise programs. (Nature and scope of conclusion)
Randomization?

20. Example
In 2001 a report in the Journal of the American Cancer Institute indicated that women who work nights have a 60% chance of developing breast cancer. Researchers based these findings on the work histories of 763 women with breast cancer and 741 women without the disease.
Observational Study
Retrospective
Women; unknown selection process with in formation taken from work histories
(subjects studied and how they were selected)
Risk of breast cancer
parameter of interest
Observational study, no way to know that working nights causes breast cancer.
Nature and scope of the conclusion.

21. Example
Some gardeners prefer to use non-chemical methods to control insect pests in their gardens. Two kinds of traps have been designed and Researchers want to know which one is more effective. They randomly choose 10 locations in a large garden and place one of each kind of trap at each location. After a week they count the number of bugs in each trap.
Experiment.
Locations in a garden. (experimental units)
1 factor: traps(2 levels). (factors)
2 treatments. (# of treatments)
Number of bugs in trap. (response variable measured)
Blocked by locations. (type of design)
Not blind.
One type of trap is more effective than the other (Nature and scope of conclusion)

22. Diagrams One way to portray an experiment is through a diagram.
UNITS  TREATMENT  RESPONSE
Group 1
Treatment 1
Random Assignment
Compare response
Group 2
Treatment 2

23. Example – Shingles
A research doctor believes a new ointment will be more effective than the current medication in treating shingles (a painful skin rash). Eight patients have volunteered for our study. We will design an experiment to help the doctor verify her claim.

24. Example – Shingles Experimental Unit = Person (we have 8)
Response = Severity of shingles
Factor = ointment
Levels = current ointment and new ointment
Why do we not want a placebo here?
Treatments = current ointment and new ointment
Treatments are the same as levels here because we only have one factor

25. Example - Shingles Control Replication Randomization
Ointment taken every day for both groups
All other care is similar
Replication
4 patients in each treatment group
Randomization
Randomly assign 4 patients to each treatment group
How do we do this?

26. Example - Shingles
Patients: Alex, Bethany, Carl, Denise, Evelyn, Fred, George, Hannah
Number the patients 1 through 8
Use random numbers to assign treatments
First 4 patients we select get new treatment, the rest get the current treatment

27. Example - Shingles
By one’s:
Throw out 0 and 9 (they don’t correspond to patients)
Throw out repeats
New ointment: 4, 1, 8, 3
Denise, Hannah, Alex, Carl
Current ointment: 2, 5, 6, 7 (everyone else)
Bethany, Evelyn, Fred, George

28. Compare severity of shingles
Example - Shingles
Group 1
4 patients
Treatment 1
New Ointment
Randomly Assign the
8 patients to groups
Compare severity of shingles
Group 2
4 patients
Treatment 2
Current Ointment

29. Example - Shingles Could we make this experiment double-blind?
Double-blind means that neither the patient nor the person evaluating the results knows who is receiving each treatment. To make this double-blind we would have to assume that the two ointments look alike: same color, unmarked tubes, similar texture, similar odors, etc.
If one ointment had a distinctive odor, it would probably not be possible to make the experiment double-blind.

30. Example – Blocking Cancer treatment (3 treatments)
We suspect that men and women respond differently so divide subjects into men and women
Randomly assign each block to three groups
The three groups in each block receive only one treatment
Compare survival rates

31. Examples – Blocking Group 1 Treatment 1 Compare survival rates
Random Assign.
Men
Group 2
Treatment 2
Group 3
Treatment 3
Subjects
Group 1
Treatment 1
Compare survival rates
Random Assign.
Women
Group 2
Treatment 2
Group 3
Treatment 3

32. Statistical Significance
If the differences between the treatment groups are big enough, we will attribute the differences to the treatments
How big is “big enough”?
Answer more precisely later
Looking for differences large enough to be due to something other than random variation