1. Causation ? Tim Wiemken, PhD MPH CIC Assistant Professor Division of Infectious Diseases University of Louisville, Kentucky

2. 1. Testing for an Association 3. Confidence Intervals 2. Other Measures of Association Overview

3. 3. Confidence Intervals 2. Other Measures of Association Overview 1. Testing for an Association

4. Null hypothesis: There is no association Alternative hypothesis: There is an association 1. Develop hypothesis Testing for Association

5. 1. Develop hypothesis Testing for Association

6. What P-value will you consider statistically significant? Usually 0.05 - arguments for bigger/smaller 2. Choose your level of significance α value Testing for Association

7. Call your statistician. A bad test gives bad results. A good test may give bad results (bad data?). A good statistician may tell you if the results are bad, but cannot always tell you if the data were bad. 3. Choose Your Test Testing for Association

8. Will tell you if there is an association between two variables Chi-squared Test Testing for Association

9. Will tell you if there is an association between two variables Chi-squared Test Testing for Association Measures observed versus expected counts in study groups

10. Will tell you if there is an association between two variables Chi-squared Test Testing for Association Measures observed versus expected counts in study groups Must have adequate sample size

11. 2x2 table – categorical data Chi-squared Test Outcome +Outcome - Predictor + Predictor - Testing for Association

12. Example Research question: Does HIV impact mortality in hospitalized patients with community-acquired pneumonia?

13. Hospitalized CAP Patients HIV+ HIV- Dead Alive Does HIV Have an Effect on Patient In-Hospital Mortality? Example

14. Hospitalized CAP Patients HIV+ HIV- Dead Alive Predictor Variable: ? Example

15. Hospitalized CAP Patients HIV+ HIV- Dead Alive Outcome Variable: ? Example

16. Significance Level Null Hypothesis What Test? Does HIV Have an Effect on Patient In-Hospital Mortality? Example

17. Does HIV Have an Effect on Patient In- Hospital Mortality? Outcome +Outcome - Predictor + Predictor - Example

18. Does HIV Have an Effect on Patient In- Hospital Mortality? + HIV, - died: - HIV, - died: + HIV, + died : - HIV, + died : Example

19. Does HIV Have an Effect on Patient In- Hospital Mortality? Outcome +Outcome - Predictor + Predictor - Example

20. Does HIV Have an Effect on Patient In- Hospital Mortality? How many patients died in- hospital? Example

21. Does HIV Have an Effect on Patient In- Hospital Mortality? How many patients died in- hospital? n=27 Example

22. Does HIV Have an Effect on Patient In- Hospital Mortality? How many patients had HIV? Example

23. Does HIV Have an Effect on Patient In- Hospital Mortality? How many patients had HIV? n=30 Example

24. Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+ HIV- Example n=27 n=30 n=100

25. =countifs(b2:b101, 1, z2:z101, 1) Does HIV Have an Effect on Patient In- Hospital Mortality? How many patients with HIV died? Example count the number of cases of deaths (column b, in_hosp_mort=1) that had HIV (column z, hiv=1)

26. Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+11 HIV- Example n=27 n=30 n=100

27. Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+11 HIV- 27 - 11 = 16 Example n=27 n=30 n=100

28. Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+1130 - 11 = 19 HIV- 27 - 11 = 16 Example n=27 n=30 n=100

29. Check this! Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+1130 - 11 = 19 HIV- 27 - 11 = 16 Example n=27 n=30 n=100 =countifs(b2:b101, 0, z2:z101, 1)

30. Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+1130 - 11 = 19 HIV- 27 - 11 = 16100 – (11+16+19) = 54 Example n=27 n=30 n=100

31. Plug the data into your excel stats program Does HIV Have an Effect on Patient In- Hospital Mortality? Dead +Dead - HIV+1130 - 11 = 19 HIV- 27 - 11 = 16100 – (11+16+19) = 54 Example

32. Do they? Example

33. No! P=0.154 P>0.05 Do they? Example

34. Where to publish? Example

35. Example Maybe those without HIV are older than those with HIV, so the mortality ends up the same

36. Example How do we check this?

37. Null Hypothesis: Example Alternative Hypothesis:

38. Null Hypothesis: The age of patients with and without HIV are NOT different. Example Alternative Hypothesis: The age of patients with and without HIV ARE different.

39. Is age different in patients with and without HIV? patients? Example

40. Back to your dataset! Total cases of HIV mean age HIV SD age HIV Total cases of non-HIV mean age non HIV SD age non HIV Example

41. Total Cases Total cases of HIV =countif(Z2:Z101,1) Total cases of non-HIV =countif(Z2:Z101,0) Example

42. Average Age =averageif(Z2:Z101,1,AN2:AN101) Example =averageif(Z2:Z101,0,AN2:AN101) HIV+ HIV-

43. Standard Deviations… not as easy. =stdev(if(Z2:Z101=1,AN2:AN101)) Example Need to use an Array and a nested IF HIV+ DON’T HIT ENTER!!!!!!!!!

44. Standard Deviations… not as easy. =stdev(if(Z2:Z101=1,AN2:AN101)) Example Need to use an Array and a nested IF HIV+ ON WINDOWS: Control+Shift+Enter ON MAC: Command+Enter

45. Back to your stats program! Total cases of HIV = 30 mean age HIV: 50.3 SD age HIV: 13.62 Total cases of non-HIV = 70 mean age non HIV: 56.5 SD age non HIV: 15.96 Example

46. Is it? Example

47. NO! P>0.05 Do they? Example BUT IT IS SOOOOO CLOSE!

48. 3. Confidence Intervals 1. Testing for an Association 2. Other Measures of Association Overview

49. Used for cohort studies or clinical trials Gold standard measure for observational studies 1. Risk Ratio Answers: How much more (less) likely is this group to get an outcome versus this other group? Measures of Association

50. Do those admitted to the ICU die more than those not admitted to the ICU? Example Use the 2x2 Totals Tab Total with outcome: Total without outcome:

51. Do those admitted to the ICU die more than those not admitted to the ICU? Example Use the 2x2 Totals Tab Total with outcome: =countif(B2:B101,1) n=27 Total without outcome: 100 – 27 n=73

52. Do those admitted to the ICU die more than those not admitted to the ICU? Example Total with outcome in the ICU: Total without outcome in the ICU:

53. Do those admitted to the ICU die more than those not admitted to the ICU? Example Total with outcome in the ICU: =countifs(B2:B101,1,I2:I101,1) n=9 Total without outcome in the ICU: =countifs(B2:B101,0,I2:I101,1) n=7

54. Do those admitted to the ICU die more than those not in the ICU? Example Dead +Dead - ICU+97 ICU-?? P=0.004

55. Do those admitted to the ICU die more than those not in the ICU? Example Dead +Dead - ICU+97 ICU-27 - 9 = 1873 – 7 = 66 P=0.004

56. How much more likely are those admitted to the ICU to die? Example Risk of death in ICU group: 9/ 9+7= 56.3% Dead +Dead - ICU+97 ICU-1866

57. How much more likely are those admitted to the ICU to die? Example Risk of death in ICUgroup: 9/ 9+7= 56.3% Risk of death in non ICU group: 18/ 18+66= 21.4% Dead +Dead - ICU+97 ICU-1866

58. How much more likely are those admitted to the ICU to die? Example Risk of death in ICUgroup: 9/ 9+7= 56.3% Risk of death in non ICU group: 18/ 18+66= 21.4% Dead +Dead - ICU+97 ICU-1866 Risk Ratio: 0.563/0.214 = 2.63

59. Interpret the Risk Ratio Example Who wants to interpret a risk ratio of 2.63?

60. Interpret the Risk Ratio Example Patients admitted to the ICU are 2.63 times more likely to die than those patients not admitted to the ICU.

61. Example

62. CAP Patients Empiric Atypical Pathogen Coverage No Empiric Atypical Pathogen Coverage Dead Alive Does Empiric Atypical Pathogen Coverage Have an Effect on Patient Mortality? Example

63. Assuming a cohort study… Do those patients who have empiric atypical pathogen coverage die less often than those without atypical coverage? + Atypical : 2220 - Atypical : 658 + Atypical + died : 217 - Atypical + died : 110 Example

64. Assuming a cohort study… Do those patients who have atypical pathogen coverage die more often than those without atypical coverage? Outcome +Outcome - Predictor + Predictor - Example

65. Assuming a cohort study… Do those patients who have empiric atypical pathogen coverage die less often than those without atypical coverage? + Atypical : 2220 - Atypical : 658 + Atypical + died : 217 - Atypical + died : 110 Example

66. Assuming a cohort study… Do those patients who have atypical pathogen coverage die more often than those without atypical coverage? Outcome +Outcome - Predictor + 2172003 Predictor - 110548 Example

67. Anyone?? Interpret the Risk Ratio Example

68. Interpret the Risk Ratio Example Those with atypical coverage are 42% less likely to die as compared to those without atypical coverage

69. Remember your baseline risk. What does that mean? Assuming 8% of CAP patients die, what is the risk of death with empiric atypical pathogen coverage? Example

70. What does that mean? Example 8% x 0.58 = 4.64 Just multiply original risk by the risk ratio!

71. Even Better: Example Number Needed to Treat 1/Absolute Risk Reduction (ARR) ARR = Unexposed Risk – Exposed Risk

72. Even Better: Example Number Needed to Treat ARR = Unexposed Risk – Exposed Risk ARR = Risk w/out atypical coverage – Risk w/atypical coverage

73. Even Better: Example Number Needed to Treat

74. Even Better: Example Number Needed to Treat 16.7 = unexposed risk

75. Even Better: Example Number Needed to Treat 9.8 = exposed risk

76. Even Better: Example Number Needed to Treat 1 / (16.7 – 9.8) = 15 (round up!) Need to treat 15 patients to save 1

77. Used for case-control studies Is an approximation of the risk ratio 2. Odds Ratio Answers: How much more (less) likely are those with the outcome to have been in this group versus this other group? Measures of Association

78. Only a good approximation when the outcome is rare Can be an extremely bad approximation 2. Odds Ratio Can correct with a formula Zhang, J., & Yu, K. F. (1998). What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA, 280(19), 1690-1691. Measures of Association

79. Acinetobacter outbreak You gather information from 100 patients with Acinetobacter and 200 patients without. Example Need to identify the risk factors Measures of Association Select sample based on the outcome (Acinetobacter)

80. Key: Example Measures of Association Because the sample was selected based on the outcome (a subset of everyone who might get the outcome in your population), you can never know the actual incidence of the outcome in everyone who was exposed.

81. Cohort Study Sample Example Measures of Association Everyone Exposed Everyone Not Exposed Outcome

82. Case-Control Study Sample Example Measures of Association Subset with Outcome Subset without Outcome Exposure Status

83. Case-Control Study Sample Example Measures of Association Subset with Outcome Subset without Outcome Exposure Status Cannot know everyone exposed who gets the outcome

84. Example Analyze a number of risk factors to see if they are associated with Acinetobacter infection Measures of Association

85. + Acinetobacter : 100 - Acinetobacter : 200 + Acinetobacter + wound : 55 - Acinetobacter + wound : 10 Outbreak Investigation: Was having a traumatic wound associated with Acinetobacter baumannii infection? Example

86. Assuming a case-control study… Outbreak Investigation: Was having a traumatic wound associated with Acinetobacter baumannii infection? Outcome +Outcome - Predictor + Predictor - Example

87. + Acinetobacter : 100 - Acinetobacter : 200 + Acinetobacter + wound : 55 - Acinetobacter + wound : 10 Outbreak Investigation: Was having a traumatic wound associated with Acinetobacter baumannii infection? Example

88. Assuming a case-control study… Outbreak Investigation: Was having a traumatic wound associated with Acinetobacter baumannii infection? Acinetobacter +Acinetobacter - Wound + 55 10 Wound - 45190 Example

89. Anyone?? Interpret the Odds Ratio Example

90. Those with Acinetobacter have a 23 times higher odds of having a nonsurgical wound compared to those without Acinetobacter. Interpret the Odds Ratio Example

91. What? Interpret the Odds Ratio Outcome +Outcome - Predictor + Predictor - Order of interpretation: Example

92. Risk: Know the incidence of the outcome. So what’s the difference? How you choose your population Odds: Don’t know the incidence of the outcome. Risk Versus Odds

93. So what’s the difference? How you choose your population You can’t identify the likelihood of someone with a predictor getting an outcome because you don’t know who all had the outcome Risk Versus Odds

94. Correct the Odds Common Outcomes = Odds is a poor approximation of Risk Risk Versus Odds

95. Even Chuck Norris Hates Odds. So what’s the difference? How you choose your population Risk Versus Odds

96. Used for Time-to-event data As good as the risk ratio 3. Hazard Ratio Answers: How much more (less) likely are those in this group to get the outcome versus this other group at any given time? Measures of Association

97. 1. Testing for an Association 2. Other Measures of Association 3. Confidence Intervals Overview

98. Patients in the Universe Patients in the Sample Sampling Generalizing Confidence Intervals

99. Uses an arbitrary cutoff (0.05) Doesn’t give info on precision P-value is not good. Doesn’t help you generalize Confidence Intervals Fix: Use Confidence Interval

100. You are 95% confident that the risk (odds) of the patients in the universe is between that interval. Definition – 95% CI Confidence Intervals

101. You are 95% confident that the risk (odds) of the patients in the universe is between that interval. Definition – 95% CI “Universe” is not everyone in the world – it is everyone you can generalize back to. Confidence Intervals

102. You are 95% confident that the risk (odds) of the patients in the universe is between that interval. Definition – 95% CI “Universe” is not everyone in the world – it is everyone you can generalize back to. Confidence Intervals If the CI includes 1, that measure of association is not statistically significant (like a P-value >0.05)

103. You are 95% confident that the risk (odds) of the patients in the universe is between that interval. Definition – 95% CI “Universe” is not everyone in the world – it is everyone you can generalize back to. Confidence Intervals ‘Tighter’ CI = more power, more precision, larger sample If the CI includes 1, that measure of association is not statistically significant (like a P-value >0.05)

104. Caveat Confidence Intervals Since CI gets tighter with more people in the sample, every measure of association (except exactly 1) will eventually be significant with a large enough sample size.

105. Is this risk ratio statistically significant? Dead +Dead - Bacteremia + 25 100 Bacteremia - 310 1537 Confidence Intervals

106. No – 95% Confidence Interval includes 1 Is the RR from the bacteremia example statistically significant? Risk Ratio: 1.19 95% CI: (0.83, 1.72) Confidence Intervals

107. Using the same proportions of Predictors and Outcomes What happens as we increase the sample size? Dead +Dead - Bacteremia +200800 Bacteremia -250012400 Example

108. Yes – 95% CI does not include 1. Now is the RR from the bacteremia example statistically significant? Risk Ratio: 1.19 (Same as before) 95% Confidence Interval: (1.05, 1.36) Sample Size

109. The confidence interval becomes tighter What happens as we increase the sample size? Sample Size

110. The confidence interval becomes tighter What happens as we increase the sample size? Assuming the proportion of patients in each group stays the same, the risk ratio eventually becomes statistically significant. Sample Size

111. The confidence interval becomes tighter What happens as we increase the sample size? Assuming the proportion of patients in each group stays the same, the risk ratio eventually becomes statistically significant. Sample Size This is because the power you have to detect that effect size has increased.

112. The larger your sample, the closer you are to actually sampling the entire universe. What happens as we increase the sample size? Sample Size Therefore, your confidence interval is tighter and closer to “the truth in your universe.”

113. This makes sense. What happens as we increase the sample size? Sample Size The more people in your study, the closer you are to having the universe as your sample. Therefore your statistic should be pretty close to the ‘truth in the universe’.

114. Patients in the Universe Patients in the Sample Sampling (easy) Generalizing (hard) Confidence Intervals

115. Patients in the Universe Patients in the Sample Sampling (hard) Generalizing (easy) Confidence Intervals