1. Designing Experiments
Section 4.2 Part 1 & 2
Reference Text:
The Practice of Statistics, Fourth Edition.
Starnes, Yates, Moore

2. Starter
In late 1995, a Gallup survey reported that Americans approved sending troops to Bosnia by 46 to 40 percent. The poll did not mention that 20,000 U.S. troops were committed to go. A CBS News poll mentioned the 20,000 number and got the opposite outcome – a 58 to 33 percent disapproval rate. Briefly explain why the mention of the number of troops would cause such a big difference in the poll results. Write the name of the kind of bias that is at work here.
A church group interested in promoting volunteerism in a community chooses an SRS of 200 community addresses and sends members to visit these addresses during weekday working hours and inquire about the residents’ attitude toward volunteer work. Sixty percent of all respondents say that they would be willing to donate at least an hour a week to some volunteer organization. Bias is present in this sample design. Identify the type of bias involved and state whether you think the sample proportion obtained is higher or lower than the true population proportion.
1) Wording of the question, influences response
2) Weekday working hours  that’s when all the working people work! All the unemployed/non busy members of course would respond with yes. Undercoverage Bias. Much higher than the true population.

3. Objectives
Describe the difference between an observational study and an experiment.
Observational:
Lurking Variables -Confounding
Experiment:
Treatment Experimental units Subjects
Bad Experiments Vs. Good Experiments
Random Assignment -Control Group
Complete Randomized Design
Principals of Experimental Design
1, 2, 3 important things to have!
What can go wrong?!

4. Observational Vs. Experiment
An observational study surveys individuals to measure variables of interest.
No attempt is made to influence the response.
The goal of an observational study can be to describe some group or situation, to compare groups, or to examine relationships between variables.
An experiment deliberately imposes a treatment on individuals and observes the response.
The goal of an experiment is to determine whether the treatment causes a change in the response.
The treatment imposed is the explanatory variable (x)
The response observed is the response variable (y)

5. ONLY EXPERIMENTS CAN EXPLAIN CAUSE AND EFFECT!
An observational study, even one based on a random sample, is an incorrect and poor way to gauge the effect that changes in one variable have on another variable.
To see the response variable change, we must actually impose the change.
When our goal is to understand cause and effect, experiments are the only source of fully convincing data.
ONLY EXPERIMENTS CAN EXPLAIN CAUSE AND EFFECT!

6. Check for Understanding
Does reducing screen brightness increase battery life in lap top computers? To find out, researchers obtained 30 new laptops of the same brand. They chose 15 of the computers at random and adjusted their screens to the brightness setting. The other 15 laptop screens were left at a default setting- moderate brightness. Researchers then measured how long each machine’s battery lasted. Was this an observational study or an experiment? Justify your answer.

7. Lurking Variables: Confounding
We already know about lurking variables…
If Lurking variables start to become intertwined with the explanatory variable to the point where we don’t know which variable is responsible, then it becomes a confounding variable…
Think of it this way: Each lurking variable is a potential confounding variable. If a well-designed experiment is implemented, we can avoid lurking variables from becoming confounding variables.
Definition:
A lurking variable is a variable that is not among the explanatory or response variables in a study but that may influence the response variable.
Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.

8. Check for Understanding
Does eating dinner with their families improve students’ academic performance? According to an ABC News article, “teenagers who eat with their families at least 5 times a week are more likely to get better grades in school.” This finding was based on a sample survey conducted by researchers at Columbia University.
Was this an observational study or an experiment? Justify your answer.
What are the explanatory and response variable?
Explain clearly why such a study cannot establish a cause-and-effect relationship. Suggest a lurking variable that may be confounded with whether families eat dinner together.
Observational study, no treatment was imposed.
Explanatory: eating dinner w/ fam Response: Grades
Family members put more interest in grades and push their kids to do well in school. Family oriented and more personal time spent.

9. Remember: Experiments can give good evidence of causation
The language of Experiments Remember: Experiments can give good evidence of causation
Treatment- A specific condition applied to the individuals in an experiment.
This is what we actually do to “them”
“them” can be people, animals, or objects
Experimental Units- are the smallest collection in individuals to which treatments are applied. When units are human beings, they are often called subjects.
Factor - Another name for an explanatory variable (x)
Remember: Experiments can give good evidence of causation

10. Basic Experiment Model
Many laboratory experiments use a design like the following:
Experiment
Experimental Units
Treatment
Measure Response
In the lab environment, simple designs often work well.
Field experiments and experiments with animals or people deal with more variable conditions.
Outside the lab, badly designed experiments often yield worthless results because of confounding.

11. Example: A Louse-Y-Situation
A study published in the New England Journal of Medicine (March 11,2010) compared two medicines to treat head lice:
An oral medication called Ivermectin.
A topical lotion containing Malathion.
Researchers studied 812 people in 376 households in seven areas around the world. Of the 185 households randomly assigned to ivermectin,171 were free from head lice after two weeks compared with only 151 of the 191 households randomly assigned to malathion.
Identify:
The experimental units
Explanatory and response variables
Treatments in this experiment
Experimental units: Households
Explanatory: Type of medication Response: if the household has lice or not
Treatments: ivermectin, and malathion

12. How to experiment Badly 
We can take a look at how to experiment badly by looking at some examples:
“Does caffeine affect pulse rate?”
Many students regularly consume caffeine to help them stay alert. Thus, it seems plausible that taking caffeine might increase an individual’s pulse rate. Is this true? One way to investigate this is to have volunteers measure their pulse rates, drink some cola with caffeine, measure their pulses again after 10 minutes, and calculate the increase in pulse rate.
…….

13. “Does caffeine affect pulse rate?”
Unfortunately, even if every student’s pulse rate went up, we couldn’t attribute the increase to caffeine. Perhaps the excitement of being in an experiment made their pulse rates increase. Perhaps it was the sugar in the cola and not the caffeine. Perhaps their teacher told them a funny joke during the 10-minute waiting period and made everyone laugh!
In other words, there are many variables that are potentially confounding with taking caffeine.

14. How to experiment well 
If treatments are given to groups that differ greatly when the experiment begins, bias will result.
Allowing personal choice will bias our results in the same way that volunteers bias the results in online opinion polls.
Solution: Random Assignment…

15. To avoid AP exam error (points loss) default to the “Hat Method”
Random Assignment
In an experiment, random assignment means that experimental units are assigned to treatments at random, that is, using some sort of chance process.
Completely Randomized Design: The treatments are assigned to all the experimental units completely by chance.
To avoid AP exam error (points loss) default to the “Hat Method”

16. 50 students agreed to participate in an experiment.
Hat Method
50 students agreed to participate in an experiment.
We want a completely random design. Describe how you randomly assign 25 students to each of the two treatments:
(A and B).
Answer: To have a completely randomized design I would first assign a unique identifier to each subject by numbering them Then, using a random number generator, I would take an SRS of 25 numbers, excluding repeats. These 25 numbers, coinciding with 25 subjects would be assigned to group 1 which get treatment A. The remaining 25 subjects would be assigned to group 2 which get treatment B. I would then apply the treatments. Then I will record and compare results.

17. The Randomized Comparative Experiment
Experiments
Definition:
In a completely randomized design, the treatments are assigned to all the experimental units completely by chance.
Group 1
Group 2
Treatment 1
Treatment 2
Compare Results
Experimental Units
Random Assignment

18. Outline for Complete Random Design
Group 1
25 students
Impose
Treatment A
50 students
Random
Assignment
Compare
Results
Group 2
25 students
Impose
Treatment B

19. Control Group
The primary purpose of a control group is to provide a baseline for comparing the effects of the other treatments.
Necessary?
Some experiments don’t include control groups. That’s appropriate if researchers simply want to compare the effects of several treatments, and not to determine whether any of the work better than an inactive treatment.

20. Summary: Principles of Experimental Design: Hand Out
The basic principles of designing experiments are as follows:
Control for lurking variables that might affect the response. Use comparative design and ensure that the only systematic difference between the groups is the treatment administered.
Random Assignment: Use impersonal chance to assign experimental units to treatments. This helps create roughly equivalent groups of experimental units by balancing the effect of lurking variables that aren’t controlled on the treatment groups.
Replication: Use enough experimental units in each group so that any differences in the effect of the treatment can be distinguished from chance differences between groups.

21. Placebo Effect: A response to a dummy treatment.
What Can Go Wrong?!
Placebo Effect: A response to a dummy treatment.
Example: A good illustration of the placebo effect is when a parent kisses a child’s “boo-boo” when the child gets injured. Even though the kiss has no “active treatment,” it makes the child feel better!

22. Do Placebos Work?
Want to help balding men keep their hair? Give them a placebo- one study found that 42% of balding men maintained or increased the amount of hair on their heads when they took a placebo.
The strength of the placebo is hard to pin down because it depends on the exact environment, but “placebos work” is a good place to start when you think about planning medical experiments.

23. Checking for Understanding
You want to test a new drug to see if it provides pain relief to migraine sufferers.
Draw a schematic diagram of an experimental design that could be used on a group of 30 subjects.
Note AP Students: Drawing a diagram is for a visual help, you WILL be required to explain in depth and detail of your experimental design on the test and AP Test.

24. The Randomized Comparative Experiment
Experiments
You want to test a new drug to see if it provides pain relief to migraine sufferers. Draw a schematic diagram of an experimental design that could be used on a group of 30 subjects.
SRS 15 Group 1
SRS 15 Group 2
Treatment 1: Drug
Treatment 2: Placebo
Compare Results
30 people
Random Assignment

25. Experiments: What Can Go Wrong?
The logic of a randomized comparative experiment depends on our ability to treat all the subjects the same in every way except for the actual treatments being compared.
Good experiments, therefore, require careful attention to details to ensure that all subjects really are treated identically.
Experiments
A response to a dummy treatment is called a placebo effect. The strength of the placebo effect is a strong argument for randomized comparative experiments.
Whenever possible, experiments with human subjects should be double-blind.
Definition:
In a double-blind experiment, neither the subjects nor those who interact with them and measure the response variable know which treatment a subject received.

26. Double-Blind
In a double blind experiment, neither the subjects nor those who interact with them and measure the response variable know which treatment a subject is received.
Balding example: its foolish to tell the subject they’re getting a placebo, but its also foolish for the doctor to know that because they might expect to see less than if the subject was receiving the actual treatment.
So both administrator and subject don’t know which is which. Placebo or treatment. Double-blind

27. Statistically Significant
If we observe statistically significant differences among the groups in a randomized comparative experiment, we have good evidence that the treatment actually caused these differences!
We will learn much more about what it means to be statistically significant in Ch 7-12
“Unlikely to Happen by Chance”
Definition:
An observed effect so large that it would rarely occur by chance is called statistically significant.
A statistically significant association in data from a well-designed experiment does imply causation.

28. Statistically Significant: Causation
We know that in general, a strong association does not imply causation. A statistically significant association in data from a well-designed experiment does imply causation.
“Unlikely to Happen by Chance”
Remember: Experiments can give good evidence of causation

29. Objectives
Describe the difference between an observational study and an experiment.
Observational:
Lurking Variables -Confounding
Experiment:
Treatment Experimental units -Subjects
Bad Experiments vs good experiments
Random assignment -Control Group
Complete Randomized Design
Principals of Experimental design
1, 2, 3 important things to have!
What can go wrong?!

30. Test Results! 5B Grade: Amount: Marginal % ……A......……....11.…….35%
…….B…………...9……...29% 80% Passed
…….C…………..5……....16%
…….D…………..4… %
…….F… ……….6% 20% Failed
Mean: 81.21% Max: 100% Min: 39.5% No Outliers

31. Test Results! 2A Grade: Amount: Marginal % ……A......……....8.……….38%
…….B…………...4……...19% 76% Passed
…….C…………..4……....19%
…….D…………..3… %
…….F… ……….10% 24% Failed
Mean: 82.52% Max: 100% Min: 54% No Outliers

32. Tracking AP Stats 2015-2016 (WHS) Ch. 1 Test Ch. 2 Test Ch. 3 Test
A 4/3 A 6/7 A 8/11
B 3/6 B 6/6 B 4/9
C7/13 C4/6 C 4/ 5
D5/5 D2/3 D 3/ 4
F7/2 F7/6 F 2/ 2

33. 14/15 VS 15/16 AP Stats 14/15 15/16 Chapter 1 Test A -5 A -7 B-5 B-9
D-2
D-10
F-1
F-9
14/15
15/16
Chapter 2 Test
A -5
A -13
B-6
B-12
C-4
C-10
D-1
D-5
F-2
F-13
14/15
15/16
Chapter 3 Test
A -3
A -19
B-5
B-13
C-6
C-9
D-2
D-7
F-2
F-4
14/15
15/16
Chapter 4 Test
A - B- C- D- F-
14/15
15/16
Chapter 5 Test
A - B- C- D- F-

34. Homework Homework Worksheet 4.2 Part 1&2
Continue working on Chapter 4 Reading Guide