Designing Experiments Observational Study v. Experiment

An Observational Study observes individuals and measures variables of interest but does not attempt to influence the responses.The usual goal is to draw conclusions about the corresponding population or about differences between two or more populations. Retrospective study – observational study in which subjects are selected and then their previous conditions or behaviors are determined. Usually focus on estimating differences between groups or associations between variables. Prospective study – observational study in which subjects are followed to observe future outcomes. Usually focus on estimating differences among groups as the groups are followed during the course of the study.

Experiment An Experiment deliberately imposes some treatment on individuals in order to observe their responses. The usual goal of an experiment is to observe the effect of the manipulated factors on some response variable. R. A. Fisher: "To consult a statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of."

The individuals on which the experiment is done are called experimental units If the units are people, they are called subjects The experimental condition we apply to the units is called the treatment The explanatory variable - the variable whose effect you want to study. The response variable –the variable you expect s affected by the other variable.

When designing an experiment we want to minimize the effect of lurking variables so that our results are not biased. Because we may not be able to identify and eliminate all lurking variables, it is essential that we use a control group. The control group gets an alternate treatment (ex: current drug on the market) or a fake treatment (placebo) to counter the placebo effect and/or any other lurking variables present. Having a control group allows us to compare the results of the treatments.

Confounding Variables Two variables (explanatory variables or lurking variables) are said to be confounded when their effects on a response variable cannot be distinguished from each other. Example: Nursing Home and Herbal Tea Study

The difference between confounding and lurking is in what is influencing what. When a variable, Z, influences both X and Y, making it look as if X and Y are related, when arguably, they only share a relationship with Z, the variable Z is lurking. When a variable Z influences Y, but we are really interested in how X influences Y, we may not be able to separate the effects of X from the effects of Z. In that case, X and Z are confounded. In particular, there is no need for Z to have any more than a coincidental relationship to X for there to be confounding.

Lurking There is a strong positive association between the number of firefighters at a fire and the amount of damage. Perhaps you shouldn't call the fire department! The lurking variable is the size of the blaze, which "causes" both damage and the need for additional fire fighters. Confounding In our herbal tea study, visiting patients had an effect on their attitude but we wanted to know what effect herbal tea had on attitude. We couldn’t separate the effects of herbal tea from the effects of visits on the response variable, patient attitude.