1. BC Jung A Brief Introduction to Epidemiology - XIII (Critiquing the Research: Statistical Considerations) Betty C. Jung, RN, MPH, CHES

2. BC Jung Learning/Performance Objectives u Quick review –Basics of inferential statistics –Common measures of association u To be able to statistically critique studies –Statistical Caveats –Statistical Issues –Statistical Rules of Thumb

3. BC Jung Introduction u Refresh your memory –Basics of inferential statistics –Common measures of associations used in epidemiologic studies

4. BC Jung Measures of Association & Hypothesis Testing Test Statistic = Observed Association - Expected Association Standard Error of the Association u Type I Error: Concluding there is an association when one does not exist u Type II Error: Concluding there is no association when one does exist

5. BC Jung Measures of Association u Two Main Types of Measures –Difference Measures (Two Independent Means, Two Independent Proportions, The Attributable Risk) –Ratio Measures (Relative Risk, Relative Prevalence, Odds Ratio)

6. BC Jung Measures of Association: Difference Measures u Two Independent Means u Two Independent Proportions u The Attributable Risk

7. BC Jung Attributable Risk (AR) u The difference between 2 proportions u Quantifies the number of occurrences of a health outcome that is due to, or can be attributed to, the exposure or risk factor u Used to assess the impact of eliminating a risk factor

8. BC Jung Measures of Association: Ratio Measures u Relative Risk (RR) u Relative Prevalence (RP) u Odds Ratio (OR)

9. BC Jung Strength of Association Relative Risk;(Prevalence); Odds Ratio Strength of Association 0.83-1.001.0-1.2None 0.67-0.831.2-1.5Weak 0.33-0.671.5-3.0Moderate 0.10-0.333.0-10.00Strong 10.0Approaching Infinity Source: Handler,A, Rosenberg,D., Monahan, C., Kennelly, J. (1998) Analytic Methods in Maternal and Child Health. p. 69.

10. BC Jung Caveats about Classifying Data u All persons in an epidemiologic study must be classifiable u All study reports should clearly state criteria used for classifying variables u Studies that use different criteria for defining the presence of any health state are not comparable with respect to reported rates of that health state

11. BC Jung Caveats about Quantitative & Categorical Variables u Information on variability between persons is lost when quantitative data are categorized u Collapsing a quantitative variable into a categorical variable with two or more categories may obscure the fact that the underlying variable has a much larger range in one category than in another category

12. BC Jung Caveats about Quantitative & Categorical Variables (Continued) u Be careful about comparing ranges because a larger sample will generally have a larger range u Collapsing quantitative variables into categories limits the choices of appropriate statistical tests of significance u Try using commonly used categories (as five- or ten-year age bands) to facilitate comparisons across studies

13. BC Jung Berkson’s Fallacy u Associations based on hospital or clinic data are influenced by differential admission rates among groups of people u Similar source of selection bias occur when associations are based on autopsy data

14. BC Jung Caveats about P-Values u The size of the p-value has no relationship to the potential practical significance of the findings u The P-value reveals nothing about the magnitude of effect (i.e., how much one group differs from another), or the precision of measurement (i.e., the amount of random error) u The nature of the sample, not the p-value, will determine whether inferences to the population of interest can be made (and the sample must be representative of the population)

15. BC Jung Confidence Interval Estimation u Uses the sample mean to construct an interval (range) of numbers to estimate the effect u Provides some indication of how probable it is (e.g., 68%, 90%, 95%), or how “confident” one can be, that the true mean lies within the range of numbers in the interval estimate

16. BC Jung Greenhalgh’s Questions to Ask About the Analysis (A) u Have the authors set the scene correctly? u Have they determined whether their groups are comparable, and, if necessary, adjusted for baseline differences? u What sort of data have they got, and have they used appropriate statistical tests?

17. BC Jung Greenhalgh’s Questions to Ask About the Analysis (B) u If the authors have used obscure statistical tests, why have they done so and have they referenced them? u Are the data analyzed according to the original protocol? u Were paired tests performed on paired data?

18. BC Jung Greenhalgh’s Questions to Ask About the Analysis (C) u Was a two-tailed test performed whenever the effect of an intervention could conceivably be a negative one? u Were “outliers” analyzed with both common sense and appropriate statistical adjustments? u Have assumptions been made about the nature and direction of causality?

19. BC Jung Greenhalgh’s Questions to Ask About the Analysis (D) u Have “P values” been calculated and interpreted appropriately? u Have confidence intervals been calculated, and do the authors’ conclusions reflect them? u Have the authors expressed the effects of an intervention in terms of the likely benefit or harm which an individual patient can expect?

20. BC Jung Statistical Issues: Epidemiological Studies u Logistic regression for binary outcomes u Cox regression for survival analysis u Poisson distribution for disease incidence or prevalence u Odds ratio approximates relative risk when disease is rare

21. BC Jung Statistical Issues: Environmental Studies u Good statistical models are hard to come by u Publication bias can exaggerate excess risk u Odds ratios less than two (or greater than 0.5) can be interesting

22. BC Jung Statistical Issues: Environmental Studies u What is the statistical basis for the environmental standard? u Variability vs. uncertainty u What’s the quality of the metadata u Biomarkers as surrogates for clinical outcomes

23. BC Jung Statistical Issues: Risk Assessment u Hazard identification u Dose-response evaluation u Exposure assessment u Risk characterization u Risk management

24. BC Jung Statistical Rules of Thumb u Use a logarithmic formulation to calculate sample size for cohort studies u Use no more than 4 or 5 controls per case for case-control studies u Obtain at least 10 subjects for every variable investigated for logistic regression

25. BC Jung Statistical Rules of Thumb u Increase sample size in proportion to dropout rate. If dropout rate is expected to be 20%, then increase n/0.80 u If dropout is greater than 20%, review reasons for dropouts u Accept substitutes with caution

26. BC Jung Statistical Rules of Thumb u Choosing cutoff points u Do not dichotomize unless absolutely necessary u Select an additive or multiplicative model according to: theoretical justification, practical application, and computer implication

27. BC Jung References u For Internet Resources on the topics covered in this lecture, check out my Web site: http://www.bettycjung.net/ u Other lectures in this series: http://www.bettycjung.net/Bite.htm