1. DrugEpi 4-4 Bias Module 4 Overview Context Content Area: Interpretation of Epidemiological Evidence Essential Question (Generic): Is the association causal? Essential Question (Drug Abuse Specific): Is an association with drug abuse causal? Enduring Epidemiological Understanding: Causation is only one explanation for finding an association between an exposure and a disease. Because observational studies are complicated by factors not controlled by the observer, other explanations must also be considered. Synopsis: In Module 4, students explore the rationale and methods of interpreting epidemiological studies. Students develop skills to assess possible explanations for an association found in a study, with consideration of explanations of causality, chance, confounding, reversed time order, and bias. Module 4 concludes with consideration of methods for weighing the overall evidence for an association. Lessons: Lesson 4-1: Introduction to Interpreting Associations Lesson 4-2: Causality Lesson 4-3: Chance Lesson 4-4: Bias Lesson 4-5: Confounding Lesson 4-6: Reverse Time Order Lesson 4-7: Weighing the Evidence

2. DrugEpi 4-4 Bias Module 4 - Interpretation of Epidemiological Evidence Lesson 4-4 Bias Content How a study can be in error Bias in selection of study subjects and how it can produce an erroneous result, using an example of student drug testing Bias in measurement during a study and how it can produce an erroneous result, using an example of measurement of cocaine use Big Ideas Association does not necessarily mean causation When an association is found, several possible explanations must be considered, including the possibility that the association is observed because of bias in the study. Systematic error in selection of study participants or in measurement of exposure or outcomes will result in study results that are not accurate. This project is supported by a Science Education Drug Abuse Partnership Award, Grant Number 1R24DA016357-01, from the National Institute on Drug Abuse, National Institutes of Health.

3. DrugEpi 4-4 Bias Where Are We? Essential QuestionsEnduring Understandings 1.How is this disease distributed? Health-related conditions and behaviors are not distributed uniformly in a population. They have unique distributions that can be described by how they are distributed in terms of person, place, and time. 2.What hypotheses might explain the distribution of disease? Clues for formulating hypotheses can be found by observing the way a health-related condition or behavior is distributed in a population. 3.Is there an association between the hypothesized cause and the disease? Causal hypotheses can be tested by observing exposures and diseases of people as they go about their daily lives. Information from these observational studies can be used to make and compare rates and identify associations. 4.Is the association causal? Causation is only one explanation for an association between an exposure and a disease. Because observational studies are complicated by factors not controlled by the observer, other explanations also must be considered. 5.What should be done when preventable causes of disease are found? Policy decisions are based on more than the scientific evidence. Because of competing values - social, economic, ethical, environmental, cultural, and political factors may also be considered.

4. DrugEpi 4-4 Bias Bias Is the association due to bias? 1.Cause 2.Chance 3. Bias

5. DrugEpi 4-4 Bias Bias Explanations for Finding an Association

6. DrugEpi 4-4 Bias Definition Deviation from truthful estimates of risk or inference from a study because of systematic error in the design or conduct of the study A faulty assumption in a study that produces a result or interpretation that is not true Bias

7. DrugEpi 4-4 Bias Error in a study because of systematic differences in characteristics between those who are selected to be in a study and those who are not selected. Selection Bias Definition

8. DrugEpi 4-4 Bias Definition Error in a study because of systematic, inaccuracies in measurement of exposures or outcomes Measurement Bias

9. DrugEpi 4-4 Bias Explanations for Finding an Association Is the association due to bias?

10. DrugEpi 4-4 Bias Real Scenario - in an effort to curb drug abuse among students, some schools have implemented random mandatory drug-testing programs These activities have generated controversy about the ethics and the effectiveness of such programs Evaluating A Drug-Related Program

11. DrugEpi 4-4 Bias 1.Should high school drug-testing programs be evaluated? Evaluating A Drug-Related Program “… the efficacy of this means for addressing the problem (is) self-evident.” Justice Antonin Scalia, 1995 “Schools should not implement a drug testing program until they’re proven to work. …. They are too expensive. It’s like having experimental surgery that’s never been shown to work.” Linn Goldberg, Oregon Health and Science University

12. DrugEpi 4-4 Bias “(W)e find that testing students who participate in extracurricular activities is a reasonably effective means of addressing the School District’s legitimate concerns in preventing, deterring, and detecting drug use.” Justice Clarence Thomas (June 27, 2002) Background on Testing Controversies

13. DrugEpi 4-4 Bias 1.Should high school drug-testing programs be evaluated? 2.What are the advantages of evaluating high school drug testing programs? Evaluating Strategies

14. DrugEpi 4-4 Bias 1.Should high school drug-testing programs be evaluated? 2.What are the advantages of evaluating high school drug testing programs? 3.How should high school drug-testing programs be evaluated? Evaluating Strategies

15. DrugEpi 4-4 Bias b d a c or % % Evaluating Strategies

16. DrugEpi 4-4 Bias b d a c or % % Total Drug Testing No Drug Testing Drug Use No Drug Use Risks of Drug Use Relative Risk 1 Evaluating Strategies

17. DrugEpi 4-4 Bias b d a c or % % Total Drug Use No Drug Use Risks of Drug Use Relative Risk < 1 Drug Testing No Drug Testing Evaluating Strategies

18. DrugEpi 4-4 Bias b d a c or % % Total Drug Use No Drug Use Risks of Drug Use Relative Risk > 1 Drug Testing No Drug Testing Evaluating Strategies

19. DrugEpi 4-4 Bias Controlled Trial Cohort Study Case-Control Study Cross-Sectional Study Healthy People E Random Assignment E DZ Healthy People E E DZ E E E E E E Review - The Four Study Designs

20. DrugEpi 4-4 Bias Cross-Sectional Study Observational Study Flow Diagram E E DZ Review - Cross-Sectional Study Flow Diagram Time

21. DrugEpi 4-4 Bias Time Cross-Sectional Study Drug Testing Program Drug Use

22. DrugEpi 4-4 Bias Time ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Healthy People - E Random Assignment E DZ Controlled Trial Time ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Healthy People - E E DZ Cohort Study Time ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Case-Control Study - DZ E E E E Time ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Cross-Sectional Study - E E DZ Should high school students be drug tested?

23. DrugEpi 4-4 Bias Cross-Sectional Study R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

24. DrugEpi 4-4 Bias Cross-Sectional Study ExposureOutcome R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

25. DrugEpi 4-4 Bias Cross-Sectional Study - Exposure R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. Exposure Youth, Education and Society

26. DrugEpi 4-4 Bias Youth, Education, and Society Study Cross-Sectional Study - Exposure Mailed Survey Answered by a school administrator, usually a principal 82% response rate School-Level Information

27. DrugEpi 4-4 Bias Exposure Measures Questions about school policy In the school year, were there... ? Any random searches of students for illicit drugs ? Any searches of students based on cause or suspicion for illicit drugs ? Any tests of students for illicit drugs Youth, Education, and Society Study

28. DrugEpi 4-4 Bias Cross-Sectional Study - Outcome Monitoring the Future survey Outcome R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

29. DrugEpi 4-4 Bias Monitoring the Future Survey Self-Administered Questionnaires 30,000 8 th grade students in 260 schools 23,000 10 th grade students in 227 high schools 23,000 12 th grade students in 235 high schools Cross-Sectional Study - Outcome

30. DrugEpi 4-4 Bias Outcome Measures On how many occasions (if any) have you used marijuana? - In your lifetime - During the last 12 months - During the last 30 days? On how many occasions (if any) have you used drugs other than marijuana (separate questions for LSD, crack, amphetamines, heroin and several others - In your lifetime - During the last 12 months - During the last 30 days? Monitoring the Future Survey

31. DrugEpi 4-4 Bias 1.What percentage of schools employ a drug testing policy? 2.Which students are tested for drugs in these schools? 3.On what bases are students tested for drugs in school 4.What relationship exists between school drug testing and student drug use? Cross-Sectional Study R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

32. DrugEpi 4-4 Bias Descriptive Epidemiology R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. 1.What percentage of schools employ a drug testing policy? 2.Which students are tested for drugs in these schools? 3.On what bases are students tested for drugs in school 4.What relationship exists between school drug testing and student drug use?

33. DrugEpi 4-4 Bias Prevalence of Testing, Groups Tested, Reasons for Testing

34. DrugEpi 4-4 Bias How do you interpret the difference between the % of schools and the % of students? Descriptive Epidemiology - Prevalence of Testing

35. DrugEpi 4-4 Bias What do you think happened to the % of schools that did “drug testing of any kind” between 1998 and 2001? Descriptive Epidemiology - Prevalence of Testing

36. DrugEpi 4-4 Bias What do you think happened to the % of schools that did “drug testing of any kind” between 1998 and 2001? Descriptive Epidemiology - Prevalence of Testing

37. DrugEpi 4-4 Bias Descriptive Epidemiology - Prevalence of Testing What group of students do you think was most frequently drug tested?

38. DrugEpi 4-4 Bias Descriptive Epidemiology - Groups Tested

39. DrugEpi 4-4 Bias What do you think was the most frequent bases for drug testing? Descriptive Epidemiology - Reasons for Testing

40. DrugEpi 4-4 Bias Descriptive Epidemiology - Reasons for Testing

41. DrugEpi 4-4 Bias 1.What percentage of schools employ a drug testing policy? 2.Which students are tested for drugs in these schools? 3.On what bases are students tested for drugs in schools? Descriptive Epidemiology R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

42. DrugEpi 4-4 Bias 1.What percentage of schools employ a drug testing policy? 2.Which students are tested for drugs in these schools? 3.On what bases are students tested for drugs in schools? 4.What relationship exists between school drug testing and student drug use? Analytical Epidemiology - Associations R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

43. DrugEpi 4-4 Bias What do we mean when we say that there is an association between two things? Associated TiedRelated Linked Things that are associated are linked in some way that makes them turn up together. Review - Things That Turn Up Together

44. DrugEpi 4-4 Bias b d a c or % % Total Drug Use No Drug Use Risks of Drug Use Relative Prevalence ? Drug Testing No Drug Testing Relationship between Drug Testing and Drug Use In this study, percent of drug use is calculated for the drug testing schools versus the schools that did not drug test

45. DrugEpi 4-4 Bias b d a c or Total Drug Use No Drug Use Risks of Drug Use Relative Prevalence Drug Testing No Drug Testing Relationship between Drug Testing and Drug Use The study report provides the percentages If we divide one percentage by the other percentage, what does this give us? The relative prevalence % %

46. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use

47. DrugEpi 4-4 Bias 1 = 0 occasions 2 = 1-2 occasions 3 = 3-5 occasions 4 = 6-9 occasions 5 = 10-19 occasions 6 = 20-39 occasions 7 = 40 or more occasions 1 – 7 Scale Prevalence The number of events in a given population at a designated time Standard Deviation (SD) A measure of dispersion or variation. A summary of how widely dispersed the values are around the mean. Relationship between Drug Testing and Drug Use

48. DrugEpi 4-4 Bias ?? Relationship between Drug Testing and Drug Use

49. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use b d a c 16% 15% Total Drug Testing No Drug Testing Drug Use No Drug Use ~1 Risk of Over-weight Relative Risk b d a c 16% 15% Total Drug Testing No Drug Testing Drug Use No Drug Use ~1 Prevalence of Over-weight Relative Prevalence

50. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use Optional - Math teachers may want to point out the great variability in these data, as shown by the large standard deviations relative to the means

51. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use

52. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use ??

53. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use

54. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use

55. DrugEpi 4-4 Bias Relationship between Drug Testing and Drug Use CONCLUSION?

56. DrugEpi 4-4 Bias CONCLUSION Study Conclusions R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

57. DrugEpi 4-4 Bias CONCLUSION This study explored the association between student drug use and drug-testing policies in schools. While lack of evidence for the effectiveness of drug testing is not definitive, results suggest that drug testing in schools may not provide a panacea for reducing student drug use that some (including some on the Supreme Court) had hoped. Research has shown that the strongest predictor of student drug use is students' attitudes toward drug use and perceptions of peer use. To prevent harmful student behaviors such as drug use, school policies that address these key values, attitudes, and perceptions may prove more important in drug prevention than drug testing. Study Conclusions R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

58. DrugEpi 4-4 Bias Drug testing was not associated with students’ reported illicit drug use, …. “… the efficacy of this means for addressing the problem (is) self-evident.” Justice Antonin Scalia Study Conclusions R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

59. DrugEpi 4-4 Bias While this study offers some valuable new findings on an important policy matter, the study has limitations. Study Limitations R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164.

60. DrugEpi 4-4 Bias “While this study offers some valuable new findings on an important policy matter, the study has limitations...... Perhaps schools that instituted drug testing initially had higher use, and drug testing reduced those levels to levels similar to those at other schools. The net result would indicate no association, as observed in this study, despite some effect from drug testing... “ Study Limitations R Yamaguchi, LD Johnston, O’Malley. Relationship between student illicit drug use and school drug-testing policies. Journal of School Health 2003;73(4):159-164. Selection Bias

61. DrugEpi 4-4 Bias Study Limitations Robert L. DuPont, M.D. President of the Institute for Behavior and Health First Director of the National Institute on Drug Abuse

62. DrugEpi 4-4 Bias Study Limitations “These schools were dichotomized based on each school principal’s answer to this question: ‘In the school year, did your school test any students for illicit drug use?’”

63. DrugEpi 4-4 Bias “Not only is it not surprising that no difference in drug use rates was found between these two heterogeneous groups, but it is almost inconceivable that there would be any difference based on this question since there is no assessment of how many drug Study Limitations tests were done at each school or whether there was any sort of student drug testing program underway at each school or not. A school that did a single drug test in a year would be included in the “yes” along with a school that had a comprehensive drug prevention program that included carefully-structured student drug testing.” Measurement Bias

64. DrugEpi 4-4 Bias “The best study design that could establish whether student drug testing produced a deterrent effect would be ….” Better Design?

65. DrugEpi 4-4 Bias Time Healthy People Randomized Controlled Trial Random Assignment

66. DrugEpi 4-4 Bias Drug Testing Program Time Healthy People Drug Testing Program Random Assignment Non-Drug Using Students Randomized Controlled Trial

67. DrugEpi 4-4 Bias Drug Testing Program Drug Use Time Healthy People Drug Testing Program Random Assignment Drug Use Non-Drug Using Students Randomized Controlled Trial

68. DrugEpi 4-4 Bias “The best study design that could establish whether student drug testing produced a deterrent effect would be a randomized experimental design or an examination of longitudinal results from a Randomized Controlled Trial “natural experiment” (such as studying the same school over time, before and after student drug testing). This study did neither. Instead, it was a simple “cross–sectional” design that merely compared schools to each other in two meaningless categories – those whose principals said “yes” and those whose said “no” to that single, simplistic question.”

69. DrugEpi 4-4 Bias Postscript - Another Study

70. DrugEpi 4-4 Bias Postscript - Another Study Excerpts from Abstract Purpose: To assess the effects of random drug and alcohol testing (DAT) among high school athletes Methods: This was a 2-year prospective randomized controlled study... among five intervention high schools with a DAT policy and six schools with a deferred policy, serially assessed by voluntary, confidential questionnaires. DAT student athletes were at risk for random testing during the full academic year.

71. DrugEpi 4-4 Bias Postscript - Another Study Excerpts from Abstract Results: Student athletes from intervention and control schools did not differ in past 1-month use of illicit drug or a combination of drug and alcohol use at any of the four follow-up periods... At the end of the initial school year and after 2 full school years, student athletes at DAT schools reported less drug use during the past year (p<.01) compared to athletes at the deferred policy schools... At the final assessment, DAT athletes believed less in testing benefits (p<.05) and less that testing was a reason not to use drugs (p<.01).

72. DrugEpi 4-4 Bias Postscript - Another Study Excerpts from Abstract Conclusions: No DAT deterrent effects were evident for past month use during any of four follow-up periods Prior-year drug use was reduced in two of four follow- up self-reports, and a combination of drug and alcohol use was reduced at two assessments as well Overall, drug testing was accompanied by an increase in some risk factors for future substance use More research is needed before DAT is considered an effective deterrent for school-based athletes.

73. DrugEpi 4-4 Bias All scientific work is incomplete – whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us the freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time. Sir Austin Bradford Hill Drug Testing Programs

74. DrugEpi 4-4 Bias Bias 1.Cause 2.Chance 3. 4. 5. Bias Is the association due to bias?

75. DrugEpi 4-4 Bias Error in a study because of systematic differences in characteristics between those who are selected to be in a study and those who are not selected. Selection Bias Review

76. DrugEpi 4-4 Bias Review Error in a study because of systematic, inaccuracies in measurement of exposures or outcomes Measurement Bias

77. DrugEpi 4-4 Bias Outcome If an association was found due to bias, …. Exposure ? found due to bias, …. Review - Public Health Significance of Bias X … and you avoided or eliminated the hypothesized cause, what would happen to the outcome?

78. DrugEpi 4-4 Bias Big Ideas in this Lesson (4-4) Association does not necessarily mean causation When an association is found, several possible explanations must be considered, including the possibility that the association is observed because of bias in the study. Systematic error in selection of study participants or in measurement of exposure or outcomes will result in study results that are not accurate. This project is supported by a Science Education Drug Abuse Partnership Award, Grant Number 1R24DA016357-01, from the National Institute on Drug Abuse, National Institutes of Health. Re-Cap

79. DrugEpi 4-4 Bias Next Lesson Cause Association Cause X Exposure Outcome Confounder