1. Experimental Design

2. Some Definitions Observational Study –Observes outcomes as they occur without imposing any treatment Experiment –Actively imposes some treatment in order to observe the response –Sometimes still referred to as a “study” in media reports

3. Observational Studies In an observational study, researchers don’t assign choices; they simply observe them. –Researchers in one study looked at the relationship between the presence of music education and students’ grades in all subjects. –Since the researchers did not assign students to get music education and simply observed students “in the wild,” this is observational study.

4. Observational Studies (cont.) If the researchers first identified subjects who studied music and then collected data on their past grades, this would be considered a retrospective study. If the researchers identified subjects in advance and collected data as events unfolded, this would be considered a prospective study.

5. Observational Studies (cont.) Observational studies are valuable for discovering trends and studying the correlation. However, it is not possible for observational studies to demonstrate a causal relationship.

6. I’ve developed a new rabbit food, Hippity Hop. Rabbit Food Makes fur soft & shiny! Increases energy! 100% of daily vitamins & essential oils!

7. Can I just make these claims? What must I do to make these claims? Who (or what) should I test this on? What do I test? No, if you aim to be truthful/ethical. Do an experiment Rabbits The type of food

8. More Definitions Experimental Unit –The single individual (person, animal, plant, etc.) to which the different treatments are assigned Factor –The explanatory variable –May have multiple levels (specific values)

9. Let’s say I plan to test my new rabbit food… What are the experimental units? What is the factor? What is the response variable? Rabbits Type of food How well they grow

10. Hippity Hop I’ll use my pet rabbit, Lucky! Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food! Right?

11. More Definitions Control Group –A group used to compare the factor against; can be placebo or the “old item” Placebo –A “dummy” treatment that can have no physical effect –Used in blinding the experiment

12. Old Food Hippity Hop Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old rabbit food. WOW! Lucky is bigger & shinier so Hippity Hop is better! Right?

13. More Definitions Replication –Running the experiment through several trials and/or experimental units to demonstrate consistency in results

14. Blinding When we know what treatment was assigned, it’s difficult not to let that knowledge influence our assessment of the response, even when we try to be careful. In order to avoid the bias that might result from knowing what treatment was assigned, we use blinding.

15. Blinding (cont.) There are two main classes of individuals who can affect the outcome of the experiment: –those who could influence the results (subjects, treatment administrators, technicians) –those who evaluate the results (judges, treating physicians, etc.)

16. More Definitions Blind –Experimental units do not know which treatment they are getting Double Blind –Neither the units nor the evaluator know which treatment is being assigned

17. Old Food Hippity Hop The first five rabbits that I catch will get Hippity Hop food and the remaining five will get the old food. The Hippity Hop rabbits have scored higher so it’s the better food! Right?

18. Principles of Experimental Design Control of effects of extraneous variables on the response – by comparing treatment groups to a control group (placebo or “old”) Replication of the experiment on many subjects to quantify the natural variation in the experiment Randomization – the use of chance to assign subjects to treatments

19. Statistical Significance An observed effect so large that it would rarely occur by chance –We’ll call this probability of the effect happening a “p-value” soon…

20. Old Food Hippity Hop Number the rabbits from 1 – 10. Place the numbers in a hat. 12 3 4 5 6 78 9 10 The first five numbers pulled from the hat will be the rabbits that get Hippity Hop food. I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than those eating old food! The remaining rabbits get the old food. 3 9 85 7

21. The ONLY way to show cause & effect is with a well-designed, well- controlled experiment!

22. Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Experimental units? Factors? Levels? Response variable? How many treatments? Plots of land Type of fertilizer Fertilizer types A, B, & C Yield of crop 3

23. Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. Why is the same type of seed used on all 15 plots? What are other potential extraneous variables? Does this experiment have a placebo? Explain It is part of the controls in the experiment. Type of soil, amount of water, etc. NO – a placebo is not needed in this experiment

24. Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens. Experiment units? Factors? Levels? Response variable? Number of treatments? Two factors - type of pan & type of oven Type of pan has 3 levels (aluminum, glass, & plastic & type of oven has 2 levels (electric & gas) How evenly the cake bakes 6 Cake batter

25. Experimental Designs Completely randomized – all experimental units are allocated at random among all treatments

26. Treatment C Treatment B Randomly assign experimental units to treatments Treatment A Treatment D

27. Randomized block – experimental units are first blocked into groups and then randomly assigned to treatments

28. Treatment B Randomly assign experimental units to treatments Treatment A Put into homogeneous groups Treatment ATreatment B

29. Matched pairs design – a special type of block design –Match up experimental units according to similar characteristics and randomly assign ½ to one treatment and 1/2 automatically to the other OR –Have each experimental unit do both treatments in a random order

30. Pair experimental units according to specific characteristics. Next, randomly assign one unit from a pair to Treatment A. The other unit gets Treatment B. Treatment A Treatment B This is one way to do a matched pairs design – another way is to have the individual unit do both treatments (as you might do in a taste test).

31. More Definitions Confounding Variable –The effect of the variable on the response cannot be separated from the effects of the explanatory variable (factor)

32. Treatment B Treatment & group are confounded Treatment A Treatment B One group is assigned to treatment A & the other group to treatment B. Confounding Confounding does NOT occur in a completely randomized design!

33. Example 3: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed. Is this an experiment? Why or why not? What is a potential confounding variable? Age – younger people are more at risk to be victims of violent crimes. No, no treatment was imposed on people.

34. Example 4: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured. Is this an experiment? Why or why not? What type of design is this? Factors? Levels? Response variable? Yes, a treatment (program) is imposed on the volunteers. Completely randomized one factor: word-processing program with 4 levels speed

35. Example 4: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured. Can confounding occur in this design? Can this design be improved? Explain. NO, completely randomized designs have no confounding You could do a block design where each person uses each program in random order.

36. Example 5: Suppose that the manufacturer wants to test a new fertilizer against the current one on the market. Ten 2-acre plots of land scattered throughout the county are used. Each plot is subdivided into two subplots, one of which is treated with the current fertilizer, and the other with the new fertilizer. Wheat is planted and the crop yields are measured. What type of design is this? Why use this method? When does randomization occur? Matched - pairs design Randomly assigned treatment to first acre of each two-acre plot

37. Randomization reduces bias by spreading any uncontrolled confounding variables evenly throughout the treatment groups. Variability is controlled by sample size. Larger samples produce statistics with less variability. Blocking also helps reduce variability. Is there another way to reduce variability?

38. Low bias & low variability High bias & high variability Low bias & high variability High bias & low variability Bias vs. Variability