1. Design of Studies: Important Concepts
Linda J. Young
Department of Statistics
University of Florida

2. The Goal
Help students become informed users of the information that they encounter in their daily lives

3. Experimental Design Concepts
Experiments, Observational Studies, and Sample Surveys: How can we distinguish among these? What types of inference can be drawn from each?
Graphical displays of data
Identifying Experimental Units: What are they? Why would we want to identify them?
Blocking: Does it matter what we block by? Is it always useful?

4. Experimental Design Concepts (Continued)
Interaction: What is it? How can one design a study to identify it? Why would you want to?
Regression and extrapolation
Other possible topics:
Relative risk, odds ratios
Case control studies
Prospective, retrospective studies

6. Cell Phones Cancer Connection Scientists link eye cancer
to mobile phones
Study: No Link Between
Cell Phones and Cancer

7. Experiment with Broad Scope of Inference:
Allocation of Units to Treatment Groups
Statistical Inference Permitted
At Random
Not At Random
Selection Of
Units At
Random
Experiment with Broad Scope of Inference:
Random sample of units is selected from one population; units randomly assigned to different treatments.
Sample Survey:
Random samples of units are selected but there is no randomization of treatment among these units (the population).
Inferences to the populations can be drawn
Not At
Experiment with Narrow Scope of Inference:
A group of units is found; units are then randomly assigned to treatments.
Observational Study: Collections of available units from distinct groups are not randomly assigned treatments.
Inferences are limited to only the units included in the study
Statistical Inference
Permitted
Causal inference can be drawn
Associations, but NO casual inference

8. Study: No Link Between Cell Phones and Cancer
469 Men and Women, Aged 18 to 80, with Primary Brain Cancer
422 Control Individuals Without Brain Cancer
Subjects matched by age, gender and other characteristics
Cell Phone Use: Cancer group: 2.5 hours/month
Control group: 2.2 hours/month
Conclusion: No connection between cell phone use and cancer

9. Scientists Link Eye Cancer to Mobile Phones
118 People with Uveal Melanoma
475 People in Control Group
Much higher rate of mobile phone use found in cancer group

10. Cell Phones Cancer Connection
200 mice, all fitted with head phones
100 randomly chosen to receive pulsed digital phone radiation
100 do not receive the radiation Tumor rate 2.4 times higher in phone radiation group

11. Experiment with Broad Scope of Inference:
Allocation of Units to Treatment Groups
Statistical Inference Permitted
At Random
Not At Random
Selection Of
Units At
Random
Experiment with Broad Scope of Inference:
Random sample of units is selected from one population; units randomly assigned to different treatments.
Sample Survey:
Random samples of units are selected but there is no randomization of treatment among these units (the population).
Inferences to the populations can be drawn
Not At
Experiment with Narrow Scope of Inference:
A group of units is found; units are then randomly assigned to treatments.
Observational Study: Collections of available units from distinct groups are not randomly assigned treatments.
Inferences are limited to only the units included in the study
Statistical Inference
Permitted
Causal inference can be drawn
Associations, but NO casual inference

12. Experimental Unit: The smallest entity to which a treatment is applied at random
What is the experimental (or observational) unit for each of the previous studies?

13. Study: No Link Between Cell Phones and Cancer
469 Men and Women, Aged 18 to 80, with Primary Brain Cancer
422 Control Individuals Without Brain Cancer
Subjects matched by age, gender and other characteristics
Observational Unit: An Individual

14. Scientists Link Eye Cancer to Mobile Phones
118 People with Uveal Melanoma
475 People in Control Group
Much higher rate of mobile phone use found in cancer group
Observational Unit: An Individual

15. Cell Phones Cancer Connection
200 mice, all fitted with head phones
100 randomly chosen to receive pulsed digital phone radiation
100 do not receive the radiation Experimental Unit: A mouse

16. Suppose that 100 mice were in each of two cages
Suppose that 100 mice were in each of two cages. If turning on the cell phones is randomly assigned to a cage, what is the experimental unit?

17. Treatment would then be randomly assigned to a cage
Treatment would then be randomly assigned to a cage. Thus a cage of 100 mice would be the experimental unit.
PROBLEM!!!
There would be only one experimental unit receiving each treatment. We would have no measure of variability among experimental units treated alike. Thus, if a difference is observed, we would not know whether it was due to difference in cages, differences in treatments, or both.

18. Possible Solutions:
Assign treatments at random to individual mice.
Have more than two cages and assign treatments at random to cages.

19. Does Restricting Caloric Intake Increase Life Expectancy?
Female mice were randomly assigned to one of the following six treatment groups:
NP: Nonpurified, standard (normal) diet with no restrictions on quantity
N/N85: Normal diet with no restrictions before weaning and 85 kcal/week after weaning
N/R50: Normal diet before weaning and a reduced 50 kcal/week after weaning
N/R40: Normal diet before weaning, reduced 40 kcal/week after weaning

20. Experiment with Broad Scope of Inference:
Allocation of Units to Treatment Groups
Statistical Inference Permitted
At Random
Not At Random
Selection Of
Units At
Random
Experiment with Broad Scope of Inference:
Random sample of units is selected from one population; units randomly assigned to different treatments.
Sample Survey:
Random samples of units are selected but there is no randomization of treatment among these units (the population).
Inferences to the populations can be drawn
Not At
Experiment with Narrow Scope of Inference:
A group of units is found; units are then randomly assigned to treatments.
Observational Study: Collections of available units from distinct groups are not randomly assigned treatments.
Inferences are limited to only the units included in the study
Statistical Inference
Permitted
Causal inference can be drawn
Associations, but NO casual inference

21. Study Results n Range (Months) Average (Months) NP 49 6.4 – 35.5 27.4
N/N 85 57
17.9 – 42.3
32.7
N/N 50 71
42.3
N/N 40 60
19.6 – 54.6
45.1

22. Box Plots of Mouse Lifetime Data

23. Conclusion: For the four diets in this study, the average lifetime of female mice increased significantly as diet became more restricted.
Two key words: AVERAGE
SIGNIFICANTLY

24. Some studies have more than one kind of experimental unit.
A measure of error must be computed for each type of experimental unit.

25. Blocking Similar experimental units are grouped to form blocks.
What is meant by “similar”? For blocking to be effective, the anticipated variability among experimental units within the same block should be less than the anticipated variability among experimental units in different blocks.
Examples: litter mates
similar weights
similar ages

27. Factors and Levels of Factors
Factor: Dog food
Levels: Brand 1, Brand 2, …
Factor: Chocolate chip
Levels: Chocolate-flavored, Semi-sweet
Factor: Nicotine Patch
Levels: None, Nicotine Patch
Factor: Antidepressant
Levels: None, Antidepressant

28. Factor: Nicotine Patch
Levels: None, Nicotine Patch
Factor: Anti-depressant
Levels: None, Anti-depressant
Factorial Arrangement of Treatments: All combinations of levels of the two factors.
Treatments: Nothing, Nicotine patch alone, Anti-depressant alone, Nicotine patch and Anti-depressant

29. No Interaction
Population Treatment Mean b1 b0

30. Interaction
Population Treatment Mean b1 b0

31. Interaction
Population Treatment Mean b0 b1

32. In practice, we do not know the population treatment means
In practice, we do not know the population treatment means. We have estimates of them. If there is NO interaction, the interaction plots will not have lines that are exactly parallel.
We use statistics to help us assess whether the lack of parallelism is real or simply due to variation.

33. Interaction?
Estimated Treatment Mean b1 b0

34. Smoking Cessation Study
Researchers at the University of Wisconsin in Madison randomized 893 adults who smoked at least 15 cigarettes per day to one of three therapies (antidepressant, nicotine patch, antidepressant and nicotine patch) or a placebo.
This was a double-blind, placebo-controlled study with a completely randomized design. The antidepressant was given for 9 weeks, and the patch was used for 8 weeks. For each treatment, the proportion of smokers who were still abstaining after one year was observed.

35. Smoking Cessation Study
Patch
Antidepressant n
Estimated p No
160
0.156
Yes
244
0.300
0.164
245
0.360
p = proportion who abstained from smoking for at least a year after study’s start

36. Smoking Cessation Study
Estimated proportion
Antidepressant
No Antidepressant

37. Smoking Cessation Study
Estimated proportion
Patch
No Patch

38. Conclusions for Smoking Cessation Study
There is no statistically significant interaction between the nicotine patch and the antidepressant.
The nicotine patch does not have a statistically significant effect on the proportion who have abstained after one year.
The antidepressant has a statistically significant effect on the proportion who have abstained after one year.

39. Gross Domestic Product of the United States (Billions of Dollars)
Year
GNP
1984
3933.2
1989
5484.4
1994
7072.2
1999
9268.4
1985
4220.3
1990
5803.1
1995
7397.7
2000
9817.0
1986
4462.8
1991
5995.9
1996
7816.9
2001
1987
4739.5
1992
6337.7
1997
8304.3
2002
1988
5103.8
1993
6657.4
1998
8747.0
2003

40. Gross Domestic Product During Past Twenty Years

41. Linear Model for Gross Domestic Product Over Past Twenty Years

42. Gross Domestic Product From 1930 to 2003

43. Beware of Extrapolation Beyond the Range of the Data!!!
(Oh how hard it is to be a weather man, an econometrician,…)

44. The Titantic Alive Dead Total Female 308 154 462 Male 142 709 851 450
863
1313

45. The Titanic
Number of Passengers

46. Another View of the Titanic Data
Proportion of Passengers

47. The Odds
Of Living
Female: 308/154 = 2 (Females are twice as likely to live than to die.)
Male: 142/709 =
Odds Ratio: 2/2.003 = 9.986
Of Dying
Female: 154/308 = 0.5
Male: 709/142 = 4.993
Odds Ratio: 0.5/4.993 =
Conclusion: The odds of living for females is ten fold greater than that for males

48. Probabilities and Risk
Probability of Living
Women: 308/462 = 0.67
Men: 142/851 = 0.17
Relative Risk: /0.17 = 4.00
OR /0.67 = 0.25
Probability of Dying
Women: 154/462 = 0.33
Men: 709/851 = 0.83
Relative Risk: /0.83 = 0.4
OR 0.83/0.33 = 2.5
Conclusion: The probability of death was 2.5 times greater for males than for females. (The probability of living was 4 times greater for females than for males.)

49. Odds Versus Risk
The odds of death for males was 10 times greater than the odds of death for females.
The probability of death of 2.5 times greater for males than for females.
When an event is rare, odds ratio and relative risk have similar values; otherwise, they do not.
Relative risk is easier to interpret and more consistent with how people think.
BUT relative risk cannot always be computed.

50. Case-Control Studies
Cases: People with a particular disease who agree to participate in the study
Controls: People without the particular disease, but who are similar to the cases, who agree to participate in the study
Numbers of cases and controls exposed to a particular factor are identified

51. Case-Control Study of High Fat/Cholesterol Diet
Heart Disease
No Heart Disease
Total
High Cholesterol Diet 11 4 15
Low Cholesterol Diet 2 6 8 13 10 23

52. Heart Disease and Diet
Percentage of Participants

53. Odds Ratio Odds of Heart Disease: High Cholesterol Diet: 11/4 = 2.75
Low Cholesterol Diet: 2/6 = 0.33
Odds Ratio: 2.75/0.33 = 8.25
Conclusion: The odds of heart disease is 8.25 times greater for those eating a high cholesterol diet than for those not eating a high cholesterol diet.

54. Something to Think About
Suppose you now find out that all of the people on the high cholesterol diet were older than anyone on the low cholesterol diet.
Was it the differences in diet, the difference in age, or both that led to the high odds ratio?
The odds ratio can be adjusted for differences in age, gender, etc., using more sophisticated methods than the ones presented here.

55. Why Not Use Relative Risk?
To compute relative risk, we must first estimate the probability of heart disease for each diet group.
We are not able to do that in case-control studies. The probabilities are affected by the numbers of cases and controls recruited into the study. To estimate the probabilities, we would need to take a random sample from the population and then classify them by heart disease and diet.

56. Other Types of Studies?
Case-Control Study—a type of retrospective study
Prospective Studies?
Others?

57. Recall the Goal!
Help students become informed users of the information that they encounter in their daily lives

58. E-mail: LYoung@biostat.ufl.edu
Thank You!