1. Unit 4: Gathering Data LESSON 4-4 – EXPERIMENTAL STUDIES ESSENTIAL QUESTION: WHAT ARE GOOD WAYS AND POOR WAYS TO EXPERIMENT?

2. Learning Objective 1: BASICS OF EXPERIMENTAL STUDIES  Experimental units: the subjects of an experiment  Treatment: A specific experimental condition imposed on the subjects of the study corresponding to the explanatory variable.  Explanatory variable: Defines the groups by which treatment they receive.  Response variable: The outcome measured on the subjects to reveal the effect of the treatment(s).

3. What makes an experiment an experiment?  An experiment deliberately imposes treatments on the experimental units in order to observe and compare their responses. Randomized Experiments occur when the subjects are randomly assigned to the treatments which helps to eliminate the effects of lurking variables and potential bias.

4. EXAMPLE: (not in notes) A study finds 100 volunteers with elevated cholesterol levels not taking medication. Half are assigned to eat a bran muffins each day for breakfast and half eat a plain bagel for breakfast. After 4 months, the cholesterol levels of the volunteers are checked. Identify the experimental units, treatments, explanatory variable, and response variable. How can the research team make sure that the experiment is randomized? Learning Objective 1: BASICS OF EXPERIMENTAL STUDIES 100 volunteers w/ high cholesterol bran muff. vs. bagel type of breakfast cholesterol level (up, down, same) number volunteers 001 to 100, use random number to assign groups (first 50 in one group or back and forth)

5. EXAMPLE: (not in notes) A study finds 100 volunteers with elevated cholesterol levels not taking medication. Half are assigned to eat a bran muffins each day for breakfast and half eat a plain bagel for breakfast. After 4 months, the cholesterol levels of the volunteers are checked.  Are there any ethical concerns with this study?  What lurking variables are eliminated?  Still exist? Learning Objective 1: BASICS OF EXPERIMENTAL STUDIES If people are about to die and need medication then yes, if just average health with high cholesterol, not already seeking help, then no. Randomly assigning to spread cholesterol levels out, we know they are All qualified as elevated already. Medication will not affect results. Exercise levels of participants, what else they are eating or drinking, the type of bran muffin or bagel

6. Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT 1) Control/Comparison Group: Allows the researcher to analyze the effectiveness of the primary treatment  A placebo is a dummy treatment, i.e. sugar pill. Many subjects respond favorable to any treatment, even a placebo.  A control group need not receive a placebo. Clinical trials often compare a new treatment for a medical condition, not with a placebo, but with a treatment that is already on the market.

9. Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT 1) Control/Comparison Group: Allows the researcher to analyze the effectiveness of the primary treatment  Being in a control group might also mean that you simply receive no treatment at all.  The experiment does not need to be medically related to have a control group.

10. Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT 1) Control/Comparison Group: Allows the researcher to analyze the effectiveness of the primary treatment EXAMPLE: You want to study how different colored paper affects test scores. What is the control group treatment? The control group would have standard, white paper as their treatment.

11. Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT  2) Randomization: To have confidence in our results we should randomly assign subjects to the treatments. In doing so, we  Eliminate bias that may result from the researcher assigning the subjects.  Balance the groups on variables known to affect the response.  Balance the groups on lurking variables that may be unknown to the researcher. age ‘I like #31’s personality let’s give them the stronger treatment’ genderpre-existing conditions perceptions life experiences unknown health conditions personality traits

12. Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT EXAMPLE: (not in notes) A research team invites volunteers to their research facility where they will be placed in to two groups. They will be participating in mental tests with and without a specific energy drink being given to them. What happens if the researchers put all the younger people in the energy drink group and the older people as control? What if they assign groups based on when the person walks through the door; first 50 in group A, last 50 in group B? The fact that they are young may help them do better and make it look like the energy drink helped them. The first 50 might be more prompt people, have more control over their schedule, attention to detail, depending on test, may skew results.

13. 3) Replication is the process of assigning several experimental units to each treatment  This could be a matter of gathering a larger sample, or repeating the experiment on a new sample group.  The difference due to ordinary variation is smaller with larger samples  We have more confidence that the sample results reflect a true difference due to treatments when the sample size is large  Since it is always possible that the observed effects were due to chance alone, replicating the experiment also builds confidence in our conclusions Learning Objective 2: COMPONENTS OF A GOOD EXPERIMENT

14. EXAMPLE: (not in notes) The research group (from previous mental test study) finds that people average a score of 300 with the energy drink and 280 without the drink. They replicate the experiment and find that the next group of 100 volunteers average 305 with the energy drink and 282 without. What did replication do for their results? The fact that the results were very similar between first and second trial, and that they have now seen these results for 200 people, helps to validate the study and confidently claim that the energy drink is having some type of positive effect on mental performance.

15. Learning Objective 3: BLINDING THE EXPERIMENT  Ideally, subjects are unaware, or blind, to the treatment they are receiving  If an experiment is conducted in such a way that neither the subjects nor the investigators working with them know which treatment each subject is receiving, then the experiment is double-blinded  A double-blinded experiment  controls response bias from the  respondent and experimenter

16. Learning Objective 3: BLINDING THE EXPERIMENT EXAMPLE (not in notes): A researcher team is studying the effect of a new sleeping pill and a placebo in a lab facility. The volunteers in the study don’t know which they are getting prior to entering the sleep study room. What is this called and why does it matter? The people watching the monitors as they sleep don’t know which treatment they received. What is this called and why does it matter? BLIND – If they know what they took, they might deliberately or unintentionally keep themselves awake or sleep better, to prove or disprove the study. DOUBLE-BLIND – They might look for something that isn’t there or change the way they speak to the subjects before, during, or after the study. Better for them to have no clue.

17. Learning Objective 4: STATISTICAL SIGNIFICANCE  Results are determined to be ‘statistically significant’ if the observed difference is larger than what would be expected just by chance.  Rather than relying solely on the label of statistical significance, also look at the actual results to determine if they are practically significant.

18. Learning Objective 4: STATISTICAL SIGNIFICANCE EXAMPLE: A study attempts to find out if offering $1000 to students for every A they receive at the end of each quarter of school, will help them do better. A group of 10 students is selected at random, all maintaining a C average in the previous school-year. Their total number of A grades out of 6 classes are shown below. Student 0A1B2C3D4E5F6G7H8I9J A Total 2645110354 Students B, C, G, H, I were the money group Students A, D, E, F, J were the control group (no money) 6+4+0+3+5 = 18 = mean of 3.6 2+5+1+1+4 = 13 = mean of 2.6 Is the difference large enough? Statistically significant?

19. Learning Objective 4: STATISTICAL SIGNIFICANCE Student 0A1B2C3D4E5F6G7H8I9J A Total 2645110354 RANDOM GROUP A RANDOM GROUP B Students: 7, 5, 4, 0, 33+1+1+2+5 = 12 = mean 2.8 A difference of 1 (real result) can happen easily by pure chance so there is not sufficient statistical significance to prove an association between monetary reward and improvement in school. Students: 1, 2, 6, 8, 9 6+4+0+5+4 = 19 = mean 3.8

20. Learning Objective 5: GENERALIZING RESULTS  Recall that the goal of experimentation is to analyze the association between the treatment and the response for the population, not just the sample  However, care should be taken to generalize the results of a study only to the population that is represented by the study.