If you have your parent letter, please turn in at my desk (scissors on my desk). Get out your homework and materials for notes!

CHAPTER 4: Designing Studies 4.2 Experiments, cont.

Experiments: What Can Go Wrong? The response to a dummy treatment is called the placebo effect. In a double-blind experiment, neither the subjects nor those who interact with them and measure the response variable know which treatment a subject received. The logic of a randomized comparative experiment depends on our ability to treat all the subjects the same in every way except for the actual treatments being compared. This requires attention to details to ensure that all subjects really are treated identically.

Inference for Experiments An observed effect so large that it would rarely occur by chance is called statistically significant. A statistically significant association in data from a well-designed experiment does imply causation. Point out the well-designed experiment.

Blocking A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a randomized block design, the random assignment of experimental units to treatments is carried out separately within each block. Completely randomized designs are the simplest statistical designs for experiments. But just as with sampling, there are times when the simplest method doesn’t yield the most precise results. When a population consists of groups of individuals that are “similar within but different between,” a stratified random sample gives a better estimate than a simple random sample. This same logic applies in experiments.

Example p. 251 – 252 A Smarter Design? Suppose that a mobile phone company is considering two different keyboard designs (A and B) for its new smart phone. The company decides to perform an experiment to compare the two keyboards using a group of 10 volunteers. The response variable is typing speed, measured in words per minute. How should the company deal with the fact that four of the volunteers already use a smart phone, whereas the remaining six volunteers do not? They could use a completely randomized design and hope that the random assignment distributes the smart-phone users and non-smart-phone users about evenly between the group using keyboard A and the group using keyboard B. Even so, there might be a lot of variability in typing speed in both groups because some members of each group are much more familiar with smart phones than others. This additional variability might make it difficult to detect a difference in the effectiveness of the two keyboards. What should the researchers do?

Example p. 251 – 252 A Smarter Design? Because the company knows that experience with smart phones will affect typing speed, they could start by separating the volunteers into two groups—one with experienced smart-phone users and one with inexperienced smart-phone users. Each of these groups of similar subjects is known as a block. Within each block, the company could then randomly assign half of the subjects to use keyboard A and the other half to use keyboard B. To control other variables, each subject should be given the same passage to type while in a quiet room with no distractions. This randomized block design helps account for the variation in typing speed that is due to experience with smart phones.

Example p. 251 – 252 A Smarter Design?

Example p. 254 - 255 Men, Women, and Advertising Women and men respond differently to advertising. Researchers would like to design an experiment to compare the effectiveness of three advertisements for the same product. Explain why a randomized block design might be preferable to a completely randomized design for this experiment. Outline a randomized block design using 300 volunteers (180 men and 120 women) as subjects. Describe how you would carry out the random assignment required by your design. Group 1 n = 40 Group 2 Group 3 Ad 1 Ad 2 Ad 3 Women n = 120 Random Assignment Compare Reaction 300 volunteer subjects Completely randomized design would consider the men and women from a single pool ignoring the difference between men and women, while a randomized block design accounts for them responding differently to advertising. Compare Results Group 1 n = 60 Group 2 Group 3 Ad 1 Ad 2 Ad 3 Men n = 180 Random Assignment Compare Reaction

Matched Pairs Design A matched pairs design is a randomized blocked experiment in which each block consists of a matching pair of similar experimental units. Chance is used to determine which unit in each pair gets each treatment. Sometimes, a “pair” in a matched-pairs design consists of a single unit that receives both treatments. Since the order of the treatments can influence the response, chance is used to determine which treatment is applied first for each unit. A common type of randomized block design for comparing two treatments is a matched pairs design. The idea is to create blocks by matching pairs of similar experimental units.

Exercise 81, p. 263 Look, Ma, no hands! Does talking on a hands-free cell phone distract drivers? Researchers recruit 40 student subjects for an experiment to investigate this question. They have a driving simulator equipped with a hands-free phone for use in the study. Each subject will complete two sessions in the simulator: one while talking on the hands-free phone and the other while just driving. The order of the two sessions for each subject will be determined at random. The route, driving conditions, and traffic flow will be the same in both sessions.

Exercise 81, p. 263 Look, Ma, no hands! Does talking on a hands-free cell phone distract drivers? Researchers recruit 40 student subjects for an experiment to investigate this question. They have a driving simulator equipped with a hands-free phone for use in the study. Each subject will complete two sessions in the simulator: one while talking on the hands-free phone and the other while just driving. The order of the two sessions for each subject will be determined at random. The route, driving conditions, and traffic flow will be the same in both sessions. (a) What type of design did the researchers use in their study? Matched-Pairs Design

Exercise 81, p. 263 Look, Ma, no hands! Does talking on a hands-free cell phone distract drivers? Researchers recruit 40 student subjects for an experiment to investigate this question. They have a driving simulator equipped with a hands-free phone for use in the study. Each subject will complete two sessions in the simulator: one while talking on the hands-free phone and the other while just driving. The order of the two sessions for each subject will be determined at random. The route, driving conditions, and traffic flow will be the same in both sessions. (b) Explain why the researchers chose this design instead of a completely randomized design. Differences between students could add variability to the response, making it harder to detect if there is a difference caused by the treatments. Matched-pairs design allows each student to be compared to their self.

Exercise 81, p. 263 Look, Ma, no hands! Does talking on a hands-free cell phone distract drivers? Researchers recruit 40 student subjects for an experiment to investigate this question. They have a driving simulator equipped with a hands-free phone for use in the study. Each subject will complete two sessions in the simulator: one while talking on the hands-free phone and the other while just driving. The order of the two sessions for each subject will be determined at random. The route, driving conditions, and traffic flow will be the same in both sessions. (c) Why is it important to randomly assign the order of the treatments? If all of the students use the hands-free phone first, we won’t know if they do better the second time because they learned from mistakes or because they don’t have the hands-free phone.

Experiments DISTINGUISH between an observational study and an experiment. EXPLAIN the concept of confounding. IDENTIFY the experimental units, explanatory and response variables, and treatments in an experiment. EXPLAIN the purpose of comparison, random assignment, control, and replication in an experiment. DESCRIBE a completely randomized design for an experiment. DESCRIBE the placebo effect and the purpose of blinding in an experiment. INTERPRET the meaning of statistically significant in the context of an experiment. EXPLAIN the purpose of blocking in an experiment. DESCRIBE a randomized block design or a matched pairs design for an experiment.