1. Chapter 9
Experiments!!

2. A study is an experiment when we actually do something to people, animals, or objects in order to observe a response.
The purpose of an experiment is to reveal the response of one variable to changes in other variables.
Experiments are the only way to determine the actual cause of a relationship.

3. Basic Vocabulary of Experiments
Individuals = the people, animals, or objects studied in an experiment
Often called subjects, particularly when they are people 
Factors = the explanatory variables in an experiment
Treatment = any specific experimental condition applied to the subjects
If an experiment has several factors, a treatment is a combination of specific values of each factor.

4. Examples:
p214 #9.1: Does day care help low-income children stay in school and hold good jobs later in life? The Carolina Abecedarian Project has followed a group of 111 children since 1972, who at that time were healthy, low-income black infants in Chapel Hill, NC. All the infants received nutritional supplements and help from social workers. Half, chosen at random, were also placed in an intensive preschool program. Over more than 20 years, the CAP recorded data on such things as academic test scores, college attendance, and employment.

5. Half without intensive preschool program
Who are the subjects?
111 low income infants
What are the factor(s)?
Intensive preschool program, Nutritional Supplements, help from social workers
What are the treatment(s)?
Half with intensive preschool program
Half without intensive preschool program
 What are the response variable(s)?
Academic test scores, college attendance, employment

6. Example
p214 #9.2: What are the effects of repeated exposure to an advertising message? The answer may depend both on the length of the ad and on how often it is repeated. An experiment investigated this question using undergraduate students. All of the students viewed a 40-minute television program that included ads for a digital camera. Some subjects saw a 30-second commercial; others, a 90-second version. The same commercial was shown either 1, 3, or 5 times during the program. After viewing, all of the subjects answered questions about their recall of the ad, their attitude toward the camera, and their intention to purchase it.

7. Who are the subjects?
Undergraduate students 
What are the factor(s)?
Length and repetition of an advertisement  
What are the treatment(s)?
30 second/1 rep, 30 second/3 reps, 30 second/5 reps
90 second/1 rep, 90 second/3 reps, 90 second/5 reps
What are the response variable(s)?
Recall of ad, attitude toward the camera, intent to purchase

8. Experimenting Badly Beware of Lurking Variables!
Controlled laboratory experiments usually have little or no problems with simple designs. But, field experiments and experiments with human subjects are exposed to more variable conditions and deal with more variable subjects. A simple design often yields worthless results because of confounding with lurking variables.

9. Example
9.3 on page 216:
A college regularly offers a review course to prepare candidates for the Graduate business schools. This year, it offers only an online version of the course. The average GMAT score of students in the online course is 10% higher than the longtime average for those who took the classroom review course. Is the online course more effective?

10. Home Work
HW: page 215 #1-3, page 229 #25,26

11. Solutions

12. Solutions

13. Experiments
Lesson 2

14. Randomized Comparative Experiment
=an experiment that uses both comparison of two or more treatments and chance assignment of subjects to treatments
Eliminates bias that would result from personal choice or volunteering into groups
One group should always be labeled the control group = group of individuals with no treatment used to see if imposed treatments have any effect

15. Randomized Comparative Experiment cont
The control group is sometimes given a placebo (ex. Sugar pill) so they do not know they are not being given the real treatment.
A completely randomized experimental design has all of the subjects allocated at random among all of the treatments.

16. We will use a flowchart outline to summarize a randomized comparative experiment because it includes:
Randomization
Sizes of groups and treatments they receive
Response variables

17. Example 9.4 p.217:
A college decides to compare the progress of 25 on-campus students taught in the classroom with that of 25 students taught the same material online. The college selects students who will be taught online by taking an SRS of size 25 from the 50 available subjects. The remaining 25 students form the control group and will receive classroom instruction. Outline the experiment with a flowchart.

19. Example 9.5 p.218: Many utility companies have introduced programs to encourage energy conservation among their customers. An electric company considers placing electronic meters in households to show what the cost would be if the electricity use at that moment continued for a month. A cheaper approach is to give customers a chart and information about monitoring their electricity use. The experiment compares these two approaches (meter, chart) and also a control which consists of customers receiving information about energy conservation but no help in monitoring electricity use. The company wants to record total electricity used in a year. They find 60 single-family residences in the same city willing to participate, and assigns 20 at random to each of the three treatments.

21. Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the differences we see in the response. The logic behind this is:
Random assignment of subjects forms groups that should be similar in all respects before treatments are applied.
Comparative design ensures that influences other than the experimental treatments operate equally on all groups.
Differences in average response must be due either to the treatments or to the play of chance in the random assignment of subjects to the treatments.

22. Principles of Experimental Design
Control the effects of lurking variables on the response, most simply by comparing two or more treatments.
Randomize – use impersonal chance to assign subjects to treatments.
Use enough subjects in each group to reduce chance variation in the results.

23. In general, we hope to see difference in the responses that are very large so that it is unlikely that it was the result of chance variation. An observed effect so large that it would rarely occur by chance is called statistically significant.
If statistically significant differences occur, we have good evidence that treatments actually caused the differences.
A statistically significant association in data from a well-designed experiment does imply causation.

24. Home Work
HW: p #28,29, 30 (don’t worry about doing the randomization part of each, just do the flowcharts and answer any other questions there are.)

25. 9.28 Solution

26. 9.29 Solution

27. 9.30 Solution

28. Experiments
Lesson 3

29. Cautions about Experimentation
Randomized comparative experiments depend on the experimenter treating all subjects identically in every way except for the actual treatments being compared. Some problems that can occur are:

30. The Placebo Effect = response to a “dummy” treatment
Many people respond favorably to any treatment (even a placebo), most likely because they trust the doctor
One possible solution is to make the study double-blind = neither the subjects nor the people who interact with them know which treatment they are receiving (only statistician who assigns groups knows which treatments each have)
Helps avoid unconscious bias by experimenter

31. Lack of Realism
= subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study
Most serious potential weakness of experiments
Limits our ability to apply the conclusions of an experiment to the settings of greatest interest (usually we want to generalize our conclusions to some setting much wider than that of the actual experiment)

32. Special Experimental Designs
Completely randomized designs are often inferior to more elaborate statistical designs which can match subjects in various ways to produce more precise results. Two designs are:
Matched pairs design
compares just two treatments
chooses pairs of subjects that are as closely matched as possible

33. Matched Pairs cont…
uses chance to decide which subject in each pair gets each treatment (so random assignment is done in each pair, not for all subjects at once)
reduces the effect of variation among the subjects
sometimes each “pair” is actually one subject who gets both treatments one after the other (ex: taste tests)

34. Ex: page 231 #34a:
Here’s the opening of a press release from June 2004: “Starbucks Corp. on Monday said it would roll out a line of blended coffee drinks intended to tap into the growing popularity of reduced-calorie and reduced-fat menu choices for Americans.” You wonder if Starbucks customers like the new “Mocha Frappuccino Light” as well as the regular Mocha Frappuccino coffee. Describe a matched pairs design to answer this questions. Be sure to include proper blinding of your subjects.

35. Answer: Randomly chosen customers will each taste two cups of coffee in identical unlabeled cups. The cups will also be presented
Random order. Subjects will also cleanse their pallet between tastes.

36. Block Design
block = a group of individuals that are known before the experiment to be similar in some way that is expected to affect the response to the treatments
the random assignment of individuals to treatments is carried out separately within each block

37. Block Design cont…
Combines the idea of creating equivalent treatment groups by matching with the principle of forming treatment groups at random.
Allows us to draw separate conclusions about each block.
Allows more precise overall conclusions because differences between blocks can be removed
Blocks are usually formed based on the most important unavoidable sources of variability among subjects

38. Example 9.9 p225:
Women and men respond differently to advertising. An experiment to compare the effectiveness of three advertisements for the same product will want to look separately at the reactions of men and women, as well as assess the overall response to the ads. A completely randomized design considers all subjects (both men and women) as a single group and assigns subjects to three treatment groups without regard to their gender. This ignores the differences between men and women. Sketch a flowchart for this study if it is done this way.

40. A better design considers women and men separately
A better design considers women and men separately. Randomly assign the women to three groups, one to view each advertisement. Then separately assign the men at random to three groups. Sketch a flowchart for this study.