Experimental Design AP Stats 5.2

Experiment vs. Observational Study 1. In an experiment, you impose treatment. 2. In an observational study, you do not. You simply gather data through a census or survey of available information. (Often no less challenging than an experiment!)

Principles of Experimental Design Control (of extraneous lurking variables—All principles really are about such Control) Control (of extraneous lurking variables—All principles really are about such Control) Randomization (through random assignment to treatment groups) Randomization (through random assignment to treatment groups) Replication (lots of subjects, lots of trials) Replication (lots of subjects, lots of trials) Blocking (Create blocks of similar individuals to make that similarity moot.) Blocking (Create blocks of similar individuals to make that similarity moot.)

A few definitions Explanatory / Response variables Explanatory / Response variables Same as Independent / Dependent Same as Independent / Dependent Same as Predictor / Response Same as Predictor / Response Same as x / y Same as x / y Same as Input / Output Same as Input / Output

A few definitions Experiments investigate the effect of the predictor variable on the response variable Experiments investigate the effect of the predictor variable on the response variable Question: Do you think the data here come from an experiment? Why/why not? Question: Do you think the data here come from an experiment? Why/why not?

Three Primary Designs 1. Random Comparative 2. Block 3. Matched Pairs

Random Comparative (AKA “Completely Randomized”) All Participants Compare Response Variable Group 2 Group 1 Random Assignment

Blocking (Get this down) The purpose of blocking is to guard against the influence of a variable known or suspected to affect the response variable, but which is not the variable being studied (Get this down) The purpose of blocking is to guard against the influence of a variable known or suspected to affect the response variable, but which is not the variable being studied E.g. In a study of the effect of coffee on teaching performance, you might block on gender, because the two genders might respond differently to coffee. E.g. In a study of the effect of coffee on teaching performance, you might block on gender, because the two genders might respond differently to coffee. This does not mean we are studying “Do men and women differ on the effects of coffee on their teaching.” This does not mean we are studying “Do men and women differ on the effects of coffee on their teaching.”

Blocking Males Only Compare Response Variable Group 2 Group 1 Random Assignment Females Only Compare Response Variable Group 2 Group 1 Random Assignment

Blocking Big Caution! Big Caution! Do not re-do your hypothesis after you see the results of the two (or more) blocks! Example: You’re doing a study of medicine for dogs experiencing hip pain. You hypothesize that your medicine works. After blocking for breed, you notice that among small dogs, the medicine works fine, but among medium and large dogs, the meds don’t work. What do you conclude? Nothing! Your hypothesis as originally stated is not supported. Begin afresh with new hypothesis, new data.

Matched Pairs If a blocking variable with many levels (such as height, weight, age) presents itself, consider matched pairs blocking. If a blocking variable with many levels (such as height, weight, age) presents itself, consider matched pairs blocking. Clue: If your blocking variable is a continuous quantitative variable, matched pairs might well be a good idea! Clue: If your blocking variable is a continuous quantitative variable, matched pairs might well be a good idea! Create pairs of individuals matched on that level. Create pairs of individuals matched on that level. Randomly assign treatment to each member of the pair Randomly assign treatment to each member of the pair Compare average of differences. Compare average of differences.

Matched Pairs yr old Compare Response Variable Ibuprofin Aspirin Random Assignment yr old Compare Response Variable Ibuprofin Aspirin Random Assignment Compare Response Variable Ibuprofin Aspirin Random Assignment Etc!

Describing the three experimental designs

Random Comparative, AKA “Completely randomized”. Example (Pain Medicine Study 1.Randomly assign participants to one of two treatment groups—Aspirin or Advil—by taking names of participants, writing them on business cards, and randomly choosing them blindly from a bag. Alternate assignment to groups (first Aspirin, then Advil) until all names are assigned. 2.Induce a headache by (some ethical method) Following the onset of a headache, measure the time needed for pain to subside to “Barely Noticeable.” 3.After each participant has experienced headache and treatment, calculate average times between treatment groups

Random Comparative, AKA “Completely randomized”. Example (Training Technique Study among Football Players) 1. Block on Position Category (Defensive Line, Offensive Line, Defensive Back, Offensive Back) 2. Within each block, randomly assign players to each of three treatments (Taped ankles, orthopedic sleeve or no ankle constraint) using ___________ (method.) 3. Following each game, gather player input using the Ankle Stability Improvement Scale. 4. At the end of the season compare average ASIS scores among the three groups within each block. 5. Verify that results don’t differ significantly among the blocks.

Matched pairs 1. Create “blocks” of individuals sharing a particular level of a blocking variable. 2. Randomly assign treatment to each member of the pair. 3. Take the difference in response variable between the two members of the pairing. 4. Report the Average of the differences. 1. (For fully randomized, it’s “Difference of the averages”)

Block Design (Pain Study) 1. Block on Gender 2. Within each block, randomly assign subjects to each treatment group. 3. Administer treatment 4. Measure the difference in averages between treatment groups within that block only

Lurking (AKA Extraneous) Variables Lurking variables are variables that have the potential to mask a true understanding of the relationship between the explanatory and response variables Lurking variables are variables that have the potential to mask a true understanding of the relationship between the explanatory and response variables Lurking variables operate through two mechanisms: Lurking variables operate through two mechanisms: A confound is a variable that affects the response variable only, but upsets our understanding of the true relationship between the explanatory & response variables. A confound is a variable that affects the response variable only, but upsets our understanding of the true relationship between the explanatory & response variables. A common response variable is a variable that affects both explanatory and response variables, to the detriment of our understanding of the true relationship. A common response variable is a variable that affects both explanatory and response variables, to the detriment of our understanding of the true relationship.