Chapter 13: Experiments and Observational Studies By: Sahar Abdullah and Jimin Kim

Vocabulary Observational study: a study based on data in which no manipulation of factors has been employed Retrospective study: an observational study in which subjects are selected and then their previous conditions or behaviors are determined. Prospective study: an observational study in which subjects are followed to observe future outcomes Experiment: manipulates factors levels to create treatments, randomly assigns subjects to treatment levels, and then compares the responses of the subject groups across treatment levels Random assignment: assigning experimental units to treatment groups at random.

Vocabulary Factor: a variable whose levels are manipulated by the experimenter Response: a variable whose values are compared across different treatments Experimental units: individuals on whom experiment is performed. Called subjects or participates when the individual is a human Level: the specific values that the experimenter chooses for a factor Treatment: the process, intervention, or other controlled circumstance applied to randomly assigned experimental units; the different levels of a single factors or a combination of levels of two or more factors

Vocabulary Principles of experimental design Control Randomize Replicate Block Statistically significant: observed difference that is too large to believe that they are a result of random fluctuation Control group: this group receives the baseline treatment level; responses provide a basis for comparison

Vocabulary Blinding: Keep individuals from knowing how the subjects are assigned to treatment groups; helps eliminate bias Single blind: when those who can influence the results (ex. subjects) are blind Double blind: when those who can influence the results (ex. subjects) and those who can evaluate the results (ex. treating physicians) are both blind Placebo: a treatment known to have no effect; administered so that all groups experience the same conditions Placebo effect: the tendency of many human subjects to show a response even when administered a placebo

Vocabulary Blocking: isolating the variability attributed to the differenced Matching: reduces unwanted variation Designs: in a randomized block design, the randomization occurs only within blocks Confounding: when the levels of one factor are associated with the levels of another factor so their effects cannot be separated

Main concept - Principles of Experimental Design (“Control what you can and randomize the rest.”) Control sources of variation that are not being studied, but will have an impact on the results. This makes conditions as similar as possible for all treatment groups. Note: the results of the study cannot be generalized (i.e., extrapolated) to other levels of the controlled variables. Replicate over as many experimental units as possible. Just as for simulations, replication reduces random variability in an experiment. Randomize experimental units to treatments. This equalizes the effects of unknown or uncontrollable sources of variation. Block to reduce the effects of things that cannot be controlled. Blocking is like stratifying a sample; we separate dissimilar experimental units into separate blocks, then randomize by treatment within each block. In effect, the experiment is run separately on each block; the results are combined across blocks for purposes of study.

Elements of an Experiment Plan: State what you want to know. Response: Specify the response variable. Treatments: Specify the factor levels and treatments. Experimental Units: Specify the experimental units. Experimental Design: Control, replicate and randomly assign (also, block if it helps). Draw a Diagram Perform the Experiment Describe the Results: And, determine whether the results are statistically significant

Homework Problems 35. Some schools teach reading using phonics (the sounds made by letters) and others using whole languages (word recognition). Suppose a school district wants to know which method works better. Suggest a design for an appropriate experiment. Treatment: This experiment has 1 factor (reading program), at 2 levels (phonics and whole language), resulting in 2 treatments. Response variable: the reading score on an appropriate reading test after a year in the program. Design: Randomly assign half of the teachers to use phonics and the other half to use whole language. Then randomly assign students to teachers. There may be variation in reading score in different schools within the district, as well as by grade. Blocking by both school and grade will reduce this variation.

Homework Problems 37. A study published in the New England Journal of Medicine (Aug. 2001) suggests that it’s dangerous to enter a hospital on a weekend. During a 10-year period, researchers tracked over 4 million emergency admissions to hospitals in Ontario, Canada. Their findings revealed that patients admitted on weekends had a much higher risk of death than those who went to the emergency room on weekdays. a) The researchers said the difference in death rates was “statistically significant.” Explain in this context what this means. The difference between the death rate on the weekend and the death rate during the week is greater than would be expected due to natural sampling variation. b) What kind of study was this. Explain. It is a prospective observational study. The researchers identifies hospitals in Ontario, Canada, and tracked admissions to the emergency rooms. This cannot be an experiment since people cannot be assigned to become injured on a specific day of the week.

Homework Problems 37. A study published in the New England Journal of Medicine (Aug. 2001) suggests that it’s dangerous to enter a hospital on a weekend. During a 10-year period, researchers tracked over 4 million emergency admissions to hospitals in Ontario, Canada. Their findings revealed that patients admitted on weekends had a much higher risk of death than those who went to the emergency room on weekdays. c) If you think you’re quite ill on Saturday, should you wait until Monday to seek medical help? Explain. No, it would be foolish to wait since it is likely that confounding variables accounted for the higher death rate on the weekends. For example, people may have participated in riskier activities on the weekend. d) Suggest some possible explanations for this troubling finding. Drinking may have lead to a higher death rate. Perhaps, more people drank alcohol, which may have lead to more traffic accidents, and higher rated of violence.