1. AP Statistics Exam Review Topic #4
Experimental Design
AP Statistics Exam
Review Topic #4

2. SAMPLING Why even take a sample?
After all, the best way to represent an entire population ought to be just to use the entire population, right?
Logistical Problems with a Census:
> Too difficult to reach everything in the popn.
> Too expensive to reach everyone
> The process of testing items in the popn might harm or destroy them

3. What is the Difference Between an Experiment and an Observational study?
Experiment – researcher imposes a treatment on experimental units, then measures their response
Observational Study – researcher observes individuals and measures variables of interest but does not impose treatments

4. Experiments > Observational Studies
* An experiment would be the best; we could give the treatments to some of the subjects and not give it to others. * sometimes, though, an observational study is the best we can do.

5. Does Smoking Cause Cancer?

6. What are the characteristics of a well designed experiment?
Randomization (random assignment of treatments to subjects)
Control (a placebo group to give us something to compare the treatment group to)
Replication (as many subjects as possible)
If needed, Blocking (dividing up the subjects ahead of time by some characteristic)

7. Randomization
Experimental units should be randomly assigned to treatment groups, or treatments should be randomly allocated to experimental units.

8. Randomization, page 2
The purpose of random assignment is to even out extraneous variables and make treatment groups that are similar in all respects except for the treatment.
Note that often the experimental units are not a random sample of the population of interest. This does not flaw the design, but it may limit the scope of inference.

9. Randomization, continued
For example: The school buys a new leg press for the weight room. If I wanted to investigate whether it improved basketball players’ dunking ability…..
The easiest thing to do would be to let the Varsity players use the new leg press and give the old one to the J.V. team.

10. Suppose, at the end of the season, the Varsity players had 185 dunks during games and the J.V. JV JV players had 12. It must be due to the new leg press, right?
Maybe, but maybe not. The Varsity players are most likely better players already, not to mention older and bigger. The treatment (the new leg press) wasn’t randomly assigned to the players.
Thus, even though 185 dunks are definitely more than 12 dunks, I can’t say which variable made it happen that way – the leg press or something else.

11. Control Keep extraneous variables as constant as possible.
Randomization helps with this:
> We should randomly assign all of the basketball players – regardless of their team, their ability, grade level, height, etc. – to one of the two leg press machines.
> That way, the impacts of the variables “basketball ability,” “grade level,” or “height” are reduced.

12. Confounding
A confounding variable is one that affects the response variable and also is related to group membership (we do not expect that the variable balances out across groups).
A variable that affects the response variable but is expected to balance out across groups is not a confounding variable. We can refer to this as an extraneous variable.
Confounding is more likely to occur in an observational study than in a well designed experiment. Random assignment lessens the opportunity for confounding to occur.

13. What is a Control Group?
A control group is a group of experimental units that receive no treatment or receive only a placebo.
A control group is not necessary for a well designed experiment, but a comparison group of some sort is necessary.
Example: In medicine, if a new drug is being tested, the old drug could be the control - you don’t want to hurt someone by giving them no medicine.
In other contexts, we could have three groups: new treatment, old treatment, and no treatment.

14. Do Apples Really Help Keep the Doctor Away?
Suppose we give an apple a day to a group of middle-aged men.
At the end of the study, 12% of the men had been to the doctor at some point.
Is 12% high or low? We need some other group to compare this percentage to, so we can see if it is truly good or not.

15. Explanatory and Response Variables
The response variable is like “y” – it is the outcome. This is what we are trying to measure – for example, women’s health when they exercise regularly, children’s heights when they take vitamins, or student’s grades when they study regularly.
The explanatory variable is like “x” – it is the thing that we think causes a change in the response variable.

16. Be sure you can correctly use these terms:
Explanatory variable
Response variable
Confounding variable
Extraneous variable
Note: It is best to avoid using the term lurking variable – it will not be tested and often gets students in trouble on the AP exam.

17. Replication Be careful here . . .
Replication is about having more than one experimental unit in each treatment group.
In this context, replication is NOT about doing the entire experiment a second time.

18. Factors, etc Factors – the explanatory variables in the experiment
Levels – different values of the factors
Treatments – the combinations of the levels of each factor
Response variable – outcome of interest

19. Example . . .
Suppose we want to determine the effect of medicine and exercise on reducing blood pressure.
There are 2 factors: medicine and exercise
1. Medicine has two levels: 10mg and 20 mg
2. Exercise has three levels: none, 2xweek, daily
There are 6 treatments: 10mg, none
10mg, 2xweek
10mg, daily
20mg, none
20mg, 2xweek
20mg, daily
Response variable: change in blood pressure

20. Experimental Units
The smallest independent “objects” to which treatments are assigned and on which a response is measured.
Human experimental units are called subjects.

21. Blinding
Double Blind experiment – neither the subjects nor those interacting with them and measuring the response know who is receiving which treatment.
For examples, if a study is investigating whether aspirin is associated with lower levels of heart disease, then the patients wouldn’t know what they are taking (identical pills all around) and the doctors wouldn’t know which patients are assigned to which groups.
Note that someone must know who receives which treatment. This is what SAS does!

22. * A Single-Blind Study is when only one group of people – either the subjects or the researchers – knows which treatments have been assigned to which people. * Sometimes, this is the best we can do. For example, the people in the “no exercise” group would definitely be aware of which group they are in – they don’t exercise!

23. Types of Experimental Designs
Completely Randomized Design (CRD)
Randomized Block
Matched Pairs

24. Completely Randomized Design (also called a “Randomized Comparative Experiment” in our class sometimes)
Treatment 1
Experimental  random Compare
units assignment response
Treatment 2

25. Blocking
Blocks are groups of experimental units (subjects) that are homogeneous with respect to some inherent characteristic that is expected to affect the response to treatments.
Blocks help control known sources of variability among the experimental units so that the experimenter is better able to detect differences in the response variable that are due to the treatments.
Sometimes results will show one treatment is better for one block and a different treatment is better for the other block.
Blocking takes place after the set of subjects has been obtained.

26. Randomized Block Design
Treat 1
Males RA Compare response
Treat 2
All subjects
Females RA Compare response

27. Matched Pairs A special type of block design
Sometimes each block consists of two experimental units that are closely matched, e.g. twins
Sometimes a block consists of one subject who gets two treatments, e.g. right hand /left hand, pre-test/post-test
The matched pairs analysis is based on differences; for example, strength with right hand minus strength with left hand

28. Finally, generalization of results…..
The only way to determine cause and effect is to do a well-designed experiment
Scope of inference – to what population can the results be extrapolated.
Example – drug study using only middle-aged men as subjects may have limited scope of inference; that is, the conclusion drawn would only be applicable to middle-aged men.
This might be good – perhaps we believe that the drug works differently for middle-aged men than for other people.