Chapter 13 Experiments and Observational Studies

Review Population Parameter Sample Statistic This is a numerical summary of a population Proportion, mean, standard deviation We don’t know this number and usually never will. Sample Statistic This is a numerical summary computed from a subset of the population(a sample). sample proportion, sample mean, sample standard deviation We know this number because we can compute it. We use this number to try and get an idea of what the population parameter(which we don’t know) might be.

Observational Study Observing data in “the wild”. Researchers don't assign treatments to subjects; they simply observe them. Results show association, but do not prove causation. Also called an “Investigative Study”. Sample surveys are observational studies.

Observational Studies Two types Retrospective Study Subjects are selected Previous behaviors or conditions are determined Prospective Study Subjects are followed to observe future outcomes

Experiments A way to prove a cause and effect relationship between two or more variables Manipulate explanatory variable Observe response In an experiment you are applying a treatment to the subjects; in an observational study you just observe them

Experiment Terminology Experimental Units – individuals on whom the experiment is performed Factors – explanatory variable(s) Need at least one factor for every experiment

Experiment Terminology Levels – specific values that the experimenter chooses for a factor Need at least two levels for each factor Treatments – different levels of a single factor or combinations of levels of two or more factors

Example – OptiGro Fertilizer An ad for OptiGro plant fertilizer claims that with this product we will grow “juicier, tastier” tomatoes. We'd like to test this claim and see if we can get by with a half-dose. What are the experimental units in this example? Tomato plants (the thing we apply the treatment to) What is the response we are interested in? “Juiciness and Tastiness” of the tomatoes

Example – OptiGro Fertilizer What is the factor? Fertilizer What are the levels of this factor? No Fertilizer (to see if the fertilizer even works) Full-Dose Half-Dose (want to see if the half dose is as good as the full) What are the treatments? No Fertilizer, Full-Dose, Half-Dose

Example – OptiGro Fertilizer The makers of OptiGro want to ensure that their product will work under a wide variety of watering conditions. Let's include this as a second factor; specifically let’s look at daily watering and watering every other day. How many factors are there now? 2 What are the factor(s)? Fertilizer and water What are factor levels Fertilizer no fertilizer, half dose, full dose Water every day, every other day

Example – OptiGro Fertilizer What are the treatments in the new experiment? Half-Dose of fertilizer, daily watering Full-Dose, daily watering No fertilizer, daily watering Half-Dose, water every other day Full-Dose, water every other day No fertilizer, water every other day

Experiment Terminology Control Group – experimental units assigned to a baseline treatment level Provides basis for comparison Baseline: either a “gold standard” or a placebo Placebo – a null treatment known to have no effect Placebo Effect – tendency of many human subjects to show response even when administered a placebo

Experiment Terminology Blinding - the researcher disguises treatments Single Blind - either the subject or the evaluator is unaware of the treatment Double Blind - both the subject AND the evaluator are in the dark Reduces personal bias

Experiment Terminology Confounding - A condition where the effects of two variables on the response can’t be distinguished from each other Lurking Variable - A variable that effects the relationship between the response variable and the explanatory variable and is not included among the variables in a study.

Principles of Experimental Design Control Make all conditions as similar as possible for all treatment groups The only difference between groups should be the treatments Controlling “outside” influences reduces variation in the responses, making it easier to detect differences among the groups due to the treatments

Principles of Experimental Design Randomization Experimental units should be randomly assigned to treatment groups Order of the trials should also be randomized Some uncontrolled sources of variation will be controlled through randomization Experiments without randomization may have biased results

Principles of Experimental Design Replication In an experiment, each treatment is given to several different experimental units Entire experiment should be repeated on a different group of experimental units

Blocking This can be considered another principle of experimental design, but not required in an experimental design Group similar individuals together and then randomize within each of these blocks Reduces the effects of identifiable attributes of the subjects that cannot be controlled

Example The leg muscles of men aged 60 to 75 were 50% to 80% stronger after they participated in a 16-week, high-intensity resistance-training program twice a week. Experiment Men aged 60 to 75 (experimental units) Exercise (1 level) (factors) 1 treatment 16-week resistance-training Strength levels pre- and post-exercise program (response) Not blinded Applies only to men 60 to 75 who participate in similar exercise programs. (Nature and scope of conclusion) Randomization?

Example In 2001 a report in the Journal of the American Cancer Institute indicated that women who work nights have a 60% chance of developing breast cancer. Researchers based these findings on the work histories of 763 women with breast cancer and 741 women without the disease. Observational Study Retrospective Women; unknown selection process with in formation taken from work histories (subjects studied and how they were selected) Risk of breast cancer parameter of interest Observational study, no way to know that working nights causes breast cancer. Nature and scope of the conclusion.

Example Some gardeners prefer to use non-chemical methods to control insect pests in their gardens. Two kinds of traps have been designed and Researchers want to know which one is more effective. They randomly choose 10 locations in a large garden and place one of each kind of trap at each location. After a week they count the number of bugs in each trap. Experiment. Locations in a garden. (experimental units) 1 factor: traps(2 levels). (factors) 2 treatments. (# of treatments) Number of bugs in trap. (response variable measured) Blocked by locations. (type of design) Not blind. One type of trap is more effective than the other (Nature and scope of conclusion)

Diagrams One way to portray an experiment is through a diagram. UNITS  TREATMENT  RESPONSE Group 1 Treatment 1 Random Assignment Compare response Group 2 Treatment 2

Example – Shingles A research doctor believes a new ointment will be more effective than the current medication in treating shingles (a painful skin rash). Eight patients have volunteered for our study. We will design an experiment to help the doctor verify her claim.

Example – Shingles Experimental Unit = Person (we have 8) Response = Severity of shingles Factor = ointment Levels = current ointment and new ointment Why do we not want a placebo here? Treatments = current ointment and new ointment Treatments are the same as levels here because we only have one factor

Example - Shingles Control Replication Randomization Ointment taken every day for both groups All other care is similar Replication 4 patients in each treatment group Randomization Randomly assign 4 patients to each treatment group How do we do this?

Example - Shingles Patients: Alex, Bethany, Carl, Denise, Evelyn, Fred, George, Hannah Number the patients 1 through 8 Use random numbers to assign treatments 41098 18329 78458 31685 55259 First 4 patients we select get new treatment, the rest get the current treatment

Example - Shingles By one’s: 4 1 0 9 8 1 8 3 2 9 7 8 4 5 8 3 1 6 8 5 5 5 2 5 9 Throw out 0 and 9 (they don’t correspond to patients) Throw out repeats New ointment: 4, 1, 8, 3 Denise, Hannah, Alex, Carl Current ointment: 2, 5, 6, 7 (everyone else) Bethany, Evelyn, Fred, George

Compare severity of shingles Example - Shingles Group 1 4 patients Treatment 1 New Ointment Randomly Assign the 8 patients to groups Compare severity of shingles Group 2 4 patients Treatment 2 Current Ointment

Example - Shingles Could we make this experiment double-blind? Double-blind means that neither the patient nor the person evaluating the results knows who is receiving each treatment. To make this double-blind we would have to assume that the two ointments look alike: same color, unmarked tubes, similar texture, similar odors, etc. If one ointment had a distinctive odor, it would probably not be possible to make the experiment double-blind.

Example – Blocking Cancer treatment (3 treatments) We suspect that men and women respond differently so divide subjects into men and women Randomly assign each block to three groups The three groups in each block receive only one treatment Compare survival rates

Examples – Blocking Group 1 Treatment 1 Compare survival rates Random Assign. Men Group 2 Treatment 2 Group 3 Treatment 3 Subjects Group 1 Treatment 1 Compare survival rates Random Assign. Women Group 2 Treatment 2 Group 3 Treatment 3

Statistical Significance If the differences between the treatment groups are big enough, we will attribute the differences to the treatments How big is “big enough”? Answer more precisely later Looking for differences large enough to be due to something other than random variation