Chapter 5: Producing Data 5.2 – Designing Experiments

Overview There are good techniques for producing data. There are also bad techniques that produce worthless data. Random sampling and randomized comparative experiments are extremely important and effective statistical practices. The use of chance is vital in statistical design.

Basic Definitions Observational study... no treatment imposed on individuals. Simply measure variables of interest. Experiment... treatment imposed on individuals in order to observe responses. Experimental units: Individuals on which experiment is done.

Basic Definitions Treatment: A specific experimental condition applied to experimental units. Factors: The explanatory variables Placebo: A dummy treatment that can have no physical effect. Control group... receive dummy treatment. (Helps experimenter control effects of lurking variables.)

Basic Definitions Completely randomized experimental design: All experimental units are allocated at random among the treatments. Statistically significant observation... an observed result too unusual to be an outcome determined by pure chance. Goal: To establish a link between the treatment and the response

Well Designed Experiment There are 3 principles of experimental design 1.Control 1.Replication 1.Randomization

Control Needed to counter effects of lurking variables, etc. Simplest form of control is comparison. Experiments should compare two or more treatments in order to avoid confounding the effect of a treatment with other influence.

Replication Do experiment on many subjects to reduce chance variation in the results Increase sensitivity to differences between treatments Effects of chance will average out Beware: cannot truly eliminate chance variation

Randomization Most important Subjects assigned to treatments by pure chance. Creates groups that are similar, except for chance variation. (avoids bias) Table of random digits can be used to choose treatment groups.

Cell Phone and Driving Does talking on a hands-free cell phone distract drivers? Using a high-fidelity simulator, researchers can measure brake-response time, when the car in front suddenly brakes. Researchers have 40 volunteers for the experiment.

Cell Phone and Driving Identify a method of randomizing this experiment. Construct an outline of a randomized comparative experiment.

Randomized Comparative Experiments Randomization produces two groups of subjects that we expect to be similar in all respects before the treatments are applied. Comparative design helps ensure that influences other than the cell phone operate equally on both groups. Therefore, differences in average brake reaction time must be due either to talking on the cell phone or to the play of chance in the random assignment of subjects to the two groups.

Identify the experimental subjects, the factor(s), the treatments, and the response variable(s). Can the drug hydroxyurea reduce the severe pain caused by sickle-cell disease? A study gave the drug to 150 sickle-cell sufferers and a placebo to another 150. The researchers then counted the episodes of pain reported by each subject

Sickle-cell Experiment Subjects: 300 sickle-cell patients Factor: Type of medication Treatments: Hydoxyurea and placebo Response variable: Number of pain episodes

Treating Prostate Disease You are designing an experiment to study the effectiveness of two ways to treat prostate disease: traditional surgery and a new method that does not require surgery. You have 300 prostate patients who are willing to serve as subjects. Use a diagram to outline the design of a randomized comparative experiment. Be sure to indicate the size of the treament groups and the response variable.

Treating Prostate Disease Random Allocation Group patients Group patients Treatment 1 Surgery Treatment 2 Alternative Compare recovery

HW: pg. 357 # , 5.40, 5.41

Blocking Used to control lurking variables –Ex: Men and women react differently to stress. Group subject by known factor from before –Ex: You may want to put stronger people together What they have in common is expected to systematically affect the response to the treatments. Randomly assign each block separately to the treatments

Blocking continued Don’t always block by gender –If exercise level is known to reduce cholesterol level, then exercise is a blocking variable (experiment testing new cholesterol-lowering drug) Form blocks based on the most important unavoidable sources of variability among the experimental units Block to control variables you know about that might influence the response.

Blocking continued Randomize to control variables you don’t know about “Control what you can, block what you can’t, and randomize the rest.”

Matched Pairs (Type of Blocking) Only two treatments are being compared Impose both treatments on the same subjects –Ex: Pre- and post-tests –Note: The measurements will not be independent Appeared on the exam a couple of years ago –Ex: Comparing old vs. new waterproofing treatment on leather boots…take one boot of the pair and apply new on one and old on the other and compare results

New-Drug Study A drug manufacturer is studying how a new drug behaves in patients. There are 120 patients for this study. Investigators compare two doses: 5 mg and 10 mg. The drug can be administered by injection, by a skin patch, or by intravenous drip. Concentration in the blood after 30 minutes may depend both on the dose and on the method of administration.

New-Drug Study Subjects: 120 volunteer patients Factors: Type of dose Method of administration Treatments: 5 mg and 10 mg injection, skin patch, IV drip (there are 6 in total) Response variable: concentration in blood after 30 minutes

New-Drug Study The drug may behave differently in men and women. Create an experimental design that takes this circumstance into account.

Subjects Men Women Random Assignment Random Assignment Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Trtmt 1 Trtmt 2 Trtmt 3 Trtmt 4 Trtmt 5 Trtmt 6 Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Trtmt 1 Trtmt 2 Trtmt 3 Trtmt 4 Trtmt 5 Trtmt 6 Compare Levels Compare Levels Block Design

Cautions about Experimentation Good experiments pay extra attention to detail Double-Blind: neither the subjects nor those who measure the response know which treatment subject received –Controls the placebo effect

Caution: Environment can influence in unexpected ways Lack of realism: setting may not realistically duplicate conditions that are wanted –Ex: A study of the effects of marijuana recruited young men who used marijuana. Some were randomly assigned to smoke marijuana cigarettes, while others were given placebo cigarettes. –This failed! The control group recognized that their cigarettes were phony.

Cautions: Enviromment Statistical analysis of an experiment cannot tell us how far the results will generalize to other settings. HW: pg. 371 #