1. EXPERIMENTS AND OBSERVATIONAL STUDIES Chance Hofmann and Nick Quigley
Chapter 13
EXPERIMENTS AND OBSERVATIONAL STUDIES
Chance Hofmann and Nick Quigley

2. Observational Studies
Observational Studies- Observational studies do not change variables like experiments, they just observe existing situations with different situations and they are valuable for discovering trends and possible relationships when an experiment is too difficult.
Types of Observational Studies
Prospective Study- Subjects are identified in advance and the data is collected as time passes.
Retrospective Study- The subjects are identified in the present and researchers collect data based on historical records.

3. Experiments
Experiments- A study that allows proof of a cause-and-effect relationship.
Manipulate factor levels to create treatments.
Randomly assign subjects to these treatment levels.
Compare the responses of the subject groups across treatment levels.
In an experiment, the experimenter must identify at least one explanatory variable, called a factor, to manipulate and at least one response variable to measure.
Experimental unit- The individuals or objects which are experimented on.
Treatment- the combination of specific levels from all the factors that an experimental unit receives.

4. 4 Principles of Experimental Design
Control:
We control sources of variation other than the factors we are testing by making conditions as similar as possible for all treatment groups.
Randomize:
Randomization allows us to equalize the effects of unknown or uncontrollable sources of variation. It does not eliminate the effects of these sources, but it spreads them out across the treatment levels so that we can see past them.
Without randomization, you do not have a valid experiment and will not be able to use the powerful methods of Statistics to draw conclusions from your study.
Replicate:
Repeat the experiment, applying the treatments to a number of subjects.
The outcome of an experiment on a single subject is an anecdote, not data.
When the experimental group is not a representative sample of the population of interest, we might want to replicate an entire experiment for different groups, in different situations, etc.
Block:
Sometimes, attributes of the experimental units that we are not studying and that we can’t control may nevertheless affect the outcomes of an experiment.
If we group similar individuals together and then randomize within each of these blocks, we can remove much of the variability due to the difference among the blocks.
Note: Blocking is an important compromise between randomization and control, but, unlike the first three principles, is not required in an experimental design.

5. Sampling and Control Treatments
Control: the control treatment is used in order to get a baseline as a comparison.
Blinding: limits the possible bias of the subjects, judges, and technicians who know the intended results of the experiment and may allow that knowledge to influence the results.
When every individual in either one of these classes is blinded, an experiment is said to be single-blind.
When everyone in both classes is blinded, the experiment is called double- blind.
A “fake” treatment that looks just like the treatment being tested is called a placebo.
Placebos are the best way to blind subjects from knowing whether they are receiving the treatment or not.

6. Blinding, Blocking, and Confounding
Sampling: the manner in which subjects are chosen for survey, experimentation, or observation is the sampling and randomization of the sampling is necessary in order to reduce unwanted variation.
Blocking:
Blocking isolates the variability due to the differences between the blocks so that we can see the differences due to the treatments more clearly.
When randomization occurs only within the blocks, we call the design a randomized block design.
Confounding:
When the levels of one factor are associated with the levels of another factor, we say that these two factors are confounded.
When we have confounded factors, we cannot separate out the effects of one factor from the effects of the other factor.

7. How to set up an experiment
Start with a Plan. State what you know.
Then identify the Response variable.
Specify the factor levels and Treatments.
Specify the Experimental Units.
To set up your Experimental Design, start by observing the design and specify the Control.
Then Randomly Assign experimental units to treatments, to equalize the effects of unknown or uncontrollable sources of variation. Don’t forget to specify how the random numbers needed for randomization will be obtained.
Next, Replicate results by placing more than one experimental unit in each treatment group.
Then Make A Picture to help you think about it clearly. (Optional, unless specifically asked for it.

8. How to set up an experiment, Step 2
Specify any other experiment details. You must give enough details so another person could exactly replicate your results. It is best to include as many as possible, just in case.
Once you have collected the data, you’ll need to Display them and Compare the results of the different treatment groups.
Analyze the data and write a Conclusion to your experiment.
Congratulations! You are now ready to set up your own experiment, with your teacher’s permission, of course.

9. Practice Problems: #31
A 2001 Danish study published in the Archives of Internal Medicine casts significant doubt on suggestions that adults who drink wine have higher levels of “good” cholesterol and fewer heart attacks. These researchers followed a group of individuals born at a Copenhagen hospital between 1959 and 1961 for 40 years. Their study found that in this group the adults who drank wine were richer and better educated than those who did not.
a) What kind of study was this?
b) It is generally true that people with high levels of education and high socioeconomic status are healthier than others. How does this call into question the supposed health benefits of wine?
c) Can studies such as these prove causation (that wine helps prevent heart attacks, that drinking wine makes one richer, that being rich helps to prevent heart attacks, etc.)? Explain.

10. Practice Problems: #31 a) What kind of study was this?
It is an Observational Study- Prospective.
b) It is generally true that people with high levels of education and high socioeconomic status are healthier than others. How does this call into question the supposed health benefits of wine?
The relationship between good health and drinking wine could actually be because those who can afford wine are in a higher socioeconomic situation. There could be lurking variables, such as income and economics.
c) Can studies such as these prove causation (that wine helps prevent heart attacks, that drinking wine makes one richer, that being rich helps to prevent heart attacks, etc.)? Explain.
While there is some correlation between these variables, there is no causality indicated for the relation.

11. Practice Problems: #33
A water dowser claims to be able to sense the presence of water using a forked stick. Suppose we wish to set up and experiment to test his ability. We get 20 identical containers, fill some with water, and ask the dowser to tell which ones are full and which empty.
a) How will we randomize this procedure?
b) The dowser correctly identifies the contents of 12 out of 20 containers. Do you think this level of success is statistically significant? Explain.
c) How many correct identifications (out of 20) would the dowser have to make to convince you that the forked stick trick works? Explain.

12. Practice Problems: #33 a) How will we randomize this procedure?
We will arrange the 20 containers in 20 different locations. With random numbers we will decide which 10 containers will hold the water.
b) The dowser correctly identifies the contents of 12 out of 20 containers. Do you think this level of success is statistically significant? Explain.
No, that is close to his chances of just guessing, which would have been 10 out of 20. The dowser only got 60% correct, which is well within the margin of guessing.
c) How many correct identifications (out of 20) would the dowser have to make to convince you that the forked stick trick works? Explain.
95%-100%. Also, the dowser would have to achieve the high accuracy over multiple trials to truly get an accurate assessment of their abilities.