1. Statistical Thinking Experiments, Good and Bad
Chapter 5
Experiments, Good and Bad
Chapter 5
Chapter 5

2. What are some terms we need to know?
Statistical Thinking
What are some terms we need to know?
Response variable
what is measured as the outcome or result of a study
Explanatory variable
what we think explains or causes changes in the response variable
often determines how subjects are split into groups
Subjects
the individuals that are participating in a study
Treatments
specific experimental conditions (related to the explanatory variable) applied to the subjects
Chapter 5
Chapter 5

3. Randomized Experiment versus Observational Study
Statistical Thinking
Randomized Experiment versus Observational Study
Both typically have the goal of detecting a relationship between the explanatory and response variables.
Experiment
create differences in the explanatory variable and examine any resulting changes in the response variable
Observational Study
observe differences in the explanatory variable and notice any related differences in the response variable
Chapter 5
Chapter 5

4. Randomization: Case Study
Statistical Thinking
Randomization: Case Study
Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp )
Variables:
Explanatory: Treatment assignment
Response: Cessation of smoking (yes/no)
Treatments
Nicotine patch
Control patch
Random assignment of treatments
Chapter 5
Chapter 5

5. Statistical Thinking
Case Study
Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp )
Variables:
Explanatory: Treatment assignment
Response: Cessation of smoking (yes/no)
Treatments
Nicotine patch
Control patch
Random assignment of treatments
Chapter 5
Chapter 5

6. Why Not Always Use a Randomized Experiment?
Sometimes it is unethical or impossible to assign people to receive a specific treatment.
Certain explanatory variables, such as handedness or gender, are inherent traits and cannot be randomly assigned.
Chapter 5

7. Statistical Thinking
Placebo: Case Study
Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp )
Variables:
Explanatory: Treatment assignment
Response: Cessation of smoking (yes/no)
Treatments
Nicotine patch
Placebo: Control patch
Random assignment of treatments
Chapter 5
Chapter 5

8. Meditation and Aging (Noetic Sciences Review, Summer 1993, p. 28)
Statistical Thinking
Case Study
Meditation and Aging (Noetic Sciences Review, Summer 1993, p. 28)
Variables:
Explanatory: Observed meditation practice (yes/no)
Response: Level of age-related enzyme
Treatment not randomly assigned
Chapter 5
Chapter 5

9. Experiments: Basic Principles
Statistical Thinking
Experiments: Basic Principles
Randomization
to balance out extraneous variables across treatments
Placebo
to control for the power of suggestion
Control group
to understand changes not related to the treatments
Chapter 5
Chapter 5

10. Control Group: Case Study
Statistical Thinking
Control Group: Case Study
Mozart, Relaxation and Performance on Spatial Tasks (Nature, Oct. 14, 1993, p. 611)
Variables:
Explanatory: Relaxation condition assignment
Response: Stanford-Binet IQ measure
Active treatment: Listening to Mozart
Control groups:
Listening to relaxation tape to lower blood pressure
Silence
Chapter 5
Chapter 5

11. Confounding (Lurking) Variables
Statistical Thinking
Confounding (Lurking) Variables
The problem:
in addition to the explanatory variable of interest, there may be other variables that make the groups being studied different from each other
the impact of these variables cannot be separated from the impact of the explanatory variable on the response
Chapter 5
Chapter 5

12. Confounding (Lurking) Variables
Statistical Thinking
Confounding (Lurking) Variables
The solution:
Experiment: randomize experimental units to receive different treatments (possible confounding variables should “even out” across groups)
Observational Study: measure potential confounding variables and determine if they have an impact on the response (may then adjust for these variables in the statistical analysis)
Chapter 5
Chapter 5

13. Confounding Variables: Case Study
Statistical Thinking
Confounding Variables: Case Study
Heart or Hypothalamus? (Scientific American, May 1973, pp )
Infants were not randomized to either hear the heartbeat sound or not
Same nursery was used on subsequent days with different groups of babies
Environment variables
construction noise
temperature
Chapter 5
Chapter 5

14. Statistical Significance
Statistical Thinking
Statistical Significance
If an experiment or observational study finds a difference in two (or more) groups, is this difference really important?
If the observed difference is larger than what would be expected just by chance, then it is labeled statistically significant.
Rather than relying solely on the label of statistical significance, also look at the actual results to determine if they are practically important.
Chapter 5
Chapter 5

15. Statistical Thinking
Key Concepts
Critical evaluation of an experiment or observational study
Common terms
explanatory vs. response variables
treatments, randomization
Randomized experiments
basic principles and terminology
problem with confounding variables
Chapter 5
Chapter 5

16. Statistical Thinking
Do Now Question 1
In studies to determine the relationship between two conditions (activities, traits, etc.), one of them is often defined as the explanatory (independent) variable and the other as the outcome or response (dependent) variable. In an experiment to determine whether the drug memantine improves cognition of patients with moderate to severe Alzheimer’s disease, whether or not the patient received memantine is one variable, and cognitive score is the other. Which is the explanatory variable and which is the response variable?
Chapter 5
Chapter 5

17. Statistical Thinking
Thought Question 2
In an observational study, researchers observe what individuals do (or have done) naturally, while in an experiment, they randomly assign the individuals to groups to receive one of several “treatments.” Give an example of a situation where an experiment would not be feasible and thus an observational study would be needed.
Chapter 5
Chapter 5

18. Statistical Thinking
Thought Question 3
In testing the effect of memantine on the cognition of Alzheimer’s disease patients (from TQ #1), how would you go about randomizing 100 patients to the two treatment groups (memantine group & placebo group)? Why is it necessary to randomly assign the subjects, rather than having the experimenter decide which patients should get which treatment?
Chapter 5
Chapter 5