1. Tests for Continuous Outcomes II

2. Overview of common statistical tests Outcome Variable Are the observations correlated? Assumptions independentcorrelated Continuous (e.g. blood pressure, age, pain score) Ttest ANOVA Linear correlation Linear regression Paired ttest Repeated-measures ANOVA Mixed models/GEE modeling Outcome is normally distributed (important for small samples). Outcome and predictor have a linear relationship. Binary or categorical (e.g. breast cancer yes/no) Chi-square test Relative risks Logistic regression McNemar’s test Conditional logistic regression GEE modeling Chi-square test assumes sufficient numbers in each cell (>=5) Time-to-event (e.g. time-to-death, time-to-fracture) Kaplan-Meier statistics Cox regression n/a Cox regression assumes proportional hazards between groups

3. Overview of common statistical tests Outcome Variable Are the observations correlated? Assumptions independentcorrelated Continuous (e.g. blood pressure, age, pain score) Ttest ANOVA Linear correlation Linear regression Paired ttest Repeated-measures ANOVA Mixed models/GEE modeling Outcome is normally distributed (important for small samples). Outcome and predictor have a linear relationship. Binary or categorical (e.g. breast cancer yes/no) Chi-square test Relative risks Logistic regression McNemar’s test Conditional logistic regression GEE modeling Sufficient numbers in each cell (>=5) Time-to-event (e.g. time-to-death, time-to-fracture) Kaplan-Meier statistics Cox regression n/a Cox regression assumes proportional hazards between groups

4. Continuous outcome (means) Outcome Variable Are the observations correlated?Alternatives if the normality assumption is violated (and small n): independentcorrelated Continuous (e.g. blood pressure, age, pain score) Ttest: compares means between two independent groups ANOVA: compares means between more than two independent groups Pearson’s correlation coefficient (linear correlation): shows linear correlation between two continuous variables Linear regression: multivariate regression technique when the outcome is continuous; gives slopes or adjusted means Paired ttest: compares means between two related groups (e.g., the same subjects before and after) Repeated-measures ANOVA: compares changes over time in the means of two or more groups (repeated measurements) Mixed models/GEE modeling: multivariate regression techniques to compare changes over time between two or more groups Non-parametric statistics Wilcoxon sign-rank test: non-parametric alternative to paired ttest Wilcoxon sum-rank test (=Mann-Whitney U test): non- parametric alternative to the ttest Kruskal-Wallis test: non- parametric alternative to ANOVA Spearman rank correlation coefficient: non-parametric alternative to Pearson’s correlation coefficient

5. Divalproex vs. placebo for treating bipolar depression Davis et al. “Divalproex in the treatment of bipolar depression: A placebo controlled study.” J Affective Disorders 85 (2005) 259-266.

6. Repeated-measures ANOVA Statistical question: Do subjects in the treatment group have greater reductions in depression scores over time than those in the control group? What is the outcome variable? Depression score What type of variable is it? Continuous Is it normally distributed? Yes Are the observations correlated? Yes, there are multiple measurements on each person How many time points are being compared? >2  repeated-measures ANOVA

7. Repeated-measures ANOVA For before and after studies, a paired ttest will suffice. For more than two time periods, you need repeated-measures ANOVA. Serial paired ttests is incorrect, because this strategy will increase your type I error.

8. Repeated-measures ANOVA Answers the following questions, taking into account the fact the correlation within subjects: Are there significant differences across time periods? Are there significant differences between groups (=your categorical predictor)? Are there significant differences between groups in their changes over time?

9. Two groups (e.g., treatment placebo) id group time1 time2 time3 time4 1 A 31 29 15 26 2 A 24 28 20 32 3 A 14 20 28 30 4 B 38 34 30 34 5 B 25 29 25 29 6 B 30 28 16 34 Hypothetical data: measurements of depression scores over time in treatment (A) and placebo (B).

10. Profile plots by group B A

11. Mean plots by group B A Repeated measures ANOVA tells you if and how these two profile plots differ…

12. Possible questions… Overall, are there significant differences between time points? From plots: looks like some differences (time3 and 4 look different) Do the two groups differ at any time points? From plots: certainly at baseline; some difference everywhere Do the two groups differ in their responses over time?** From plots: their response profile looks similar over time, though A and B are closer by the end.

13. repeated-measures ANOVA… Overall, are there significant differences between time points? Time factor Do the two groups differ at any time points? Group factor Do the two groups differ in their responses over time?** Group x time factor

14. From rANOVA analysis… Overall, are there significant differences between time points? No, Time not statistically significant (p=.1743) Do the two groups differ at any time points? No, Group not statistically significant (p=.1408) Do the two groups differ in their responses over time?** No, not even close; Group*Time (p-value>.60)

15. rANOVA Time is significant. Group*time is significant. Group is not significant.

16. rANOVA Time is not significant. Group*time is not significant. Group IS significant.

17. rANOVA Time is significant. Group is not significant. Time*group is not significant.

18. Copyright ©1995 BMJ Publishing Group Ltd. Lokken, P. et al. BMJ 1995;310:1439-1442 Day of surgery Days 1-7 after surgery (morning and evening) Mean pain assessments by visual analogue scales (VAS) Homeopathy vs. placebo in treating pain after surgery p>.05; rANOVA (Group x Time)

19. Copyright ©1997 BMJ Publishing Group Ltd. Cadogan, J. et al. BMJ 1997;315:1255-1260 Mean (SE) percentage increases in total body bone mineral and bone density over 18 months. P values are for the differences between groups by repeated measures analysis of variance Pint of milk vs. control on bone acquisition in adolescent females

20. Copyright ©2000 BMJ Publishing Group Ltd. Hovell, M. F et al. BMJ 2000;321:337-342 Counseling vs. control on smoking in pregnancy P<.05; rANOVA

21. Review Question 1 In a study of depression, I measured depression score (a continuous, normally distributed variable) at baseline; 1 month; 6 months; and 12 months. What statistical test will best tell me whether or not depression improved between baseline and the end of the study? a.Repeated-measures ANOVA. b.One-way ANOVA. c.Two-sample ttest. d.Paired ttest. e.Wilcoxon sum-rank test.

22. Review Question 1 In a study of depression, I measured depression score (a continuous, normally distributed variable) at baseline; 1 month; 6 months; and 12 months. What statistical test will best tell me whether or not depression improved between baseline and the end of the study? a.Repeated-measures ANOVA. b.One-way ANOVA. c.Two-sample ttest. d.Paired ttest. e.Wilcoxon sum-rank test.

23. Review Question 2 In the same depression study, what statistical test will best tell me whether or not two treatments for depression had different effects over time? a.Repeated-measures ANOVA. b.One-way ANOVA. c.Two-sample ttest. d.Paired ttest. e.Wilcoxon sum-rank test.

24. Review Question 2 In the same depression study, what statistical test will best tell me whether or not two treatments for depression had different effects over time? a.Repeated-measures ANOVA. b.One-way ANOVA. c.Two-sample ttest. d.Paired ttest. e.Wilcoxon sum-rank test.

25. Continuous outcome (means) Outcome Variable Are the observations correlated?Alternatives if the normality assumption is violated (and small n): independentcorrelated Continuous (e.g. blood pressure, age, pain score) Ttest: compares means between two independent groups ANOVA: compares means between more than two independent groups Pearson’s correlation coefficient (linear correlation): shows linear correlation between two continuous variables Linear regression: multivariate regression technique when the outcome is continuous; gives slopes or adjusted means Paired ttest: compares means between two related groups (e.g., the same subjects before and after) Repeated-measures ANOVA: compares changes over time in the means of two or more groups (repeated measurements) Mixed models/GEE modeling: multivariate regression techniques to compare changes over time between two or more groups Non-parametric statistics Wilcoxon sign-rank test: non-parametric alternative to paired ttest Wilcoxon sum-rank test (=Mann-Whitney U test): non- parametric alternative to the ttest Kruskal-Wallis test: non- parametric alternative to ANOVA Spearman rank correlation coefficient: non-parametric alternative to Pearson’s correlation coefficient

26. Example: class data Political Leanings and Rating of Obama R=.79, p<.0001

27. Example 2: pain and injection pressure r=.75, p<.0001

28. Correlation coefficient Statistical question: Is injection pressure related to pain? What is the outcome variable? VAS pain score What type of variable is it? Continuous Is it normally distributed? Yes Are the observations correlated? No Are groups being compared? No—the independent variable is also continuous  correlation coefficient

29. New concept: Covariance

30. Covariance between two random variables: cov(X,Y) > 0 X and Y tend to move in the same direction cov(X,Y) < 0 X and Y tend to move in opposite directions cov(X,Y) = 0 X and Y are independent Interpreting Covariance

31. Correlation coefficient Pearson’s Correlation Coefficient is standardized covariance (unitless):

32. Corrrelation Measures the relative strength of the linear relationship between two variables Unit-less Ranges between –1 and 1 The closer to –1, the stronger the negative linear relationship The closer to 1, the stronger the positive linear relationship The closer to 0, the weaker any positive linear relationship

33. Scatter Plots of Data with Various Correlation Coefficients Y X Y X Y X Y X Y X r = -1 r = -.6r = 0 r = +.3 r = +1 Y X r = 0 ** Next 4 slides from “Statistics for Managers”4 th Edition, Prentice-Hall 2004

34. Y X Y X Y Y X X Linear relationshipsCurvilinear relationships Linear Correlation

35. Y X Y X Y Y X X Strong relationshipsWeak relationships Linear Correlation

36. Y X Y X No relationship

37. Review Problem 3 What’s a good guess for the Pearson’s correlation coefficient (r) for this scatter plot? a. –1.0 b. +1.0 c. 0 d. -.5 e. -.1

38. Review Problem 3 What’s a good guess for the Pearson’s correlation coefficient (r) for this scatter plot? a. –1.0 b. +1.0 c. 0 d. -.5 e. -.1

39. Continuous outcome (means) Outcome Variable Are the observations correlated?Alternatives if the normality assumption is violated (and small n): independentcorrelated Continuous (e.g. blood pressure, age, pain score) Ttest: compares means between two independent groups ANOVA: compares means between more than two independent groups Pearson’s correlation coefficient (linear correlation): shows linear correlation between two continuous variables Linear regression: multivariate regression technique when the outcome is continuous; gives slopes or adjusted means Paired ttest: compares means between two related groups (e.g., the same subjects before and after) Repeated-measures ANOVA: compares changes over time in the means of two or more groups (repeated measurements) Mixed models/GEE modeling: multivariate regression techniques to compare changes over time between two or more groups Non-parametric statistics Wilcoxon sign-rank test: non-parametric alternative to paired ttest Wilcoxon sum-rank test (=Mann-Whitney U test): non- parametric alternative to the ttest Kruskal-Wallis test: non- parametric alternative to ANOVA Spearman rank correlation coefficient: non-parametric alternative to Pearson’s correlation coefficient

40. Example: class data Political Leanings and Rating of Obama Expected Obama Rating = 3.0 +.66*political bent

41. Example 2: pain and injection pressure R-squared = correlation coefficient squared. Meaning: the percent of variance in Y that is “explained by” X.

42. Simple linear regression Statistical question: Does injection pressure “predict” pain? What is the outcome variable? VAS pain score What type of variable is it? Continuous Is it normally distributed? Yes Are the observations correlated? No Are groups being compared? No—the independent variable is also continuous  simple linear regression

43. Linear regression In correlation, the two variables are treated as equals. In regression, one variable is considered independent (=predictor) variable (X) and the other the dependent (=outcome) variable Y.

44. What is “Linear”? Remember this: Y=mX+B? B m

45. What’s Slope? A slope of 0.66 means that every 1-unit change in X yields a.66-unit change in Y.

46. Simple linear regression The linear regression model: Ratings of Obama = 3.0 + 0.66 *(political bent) slopeintercept

47. Simple linear regression Expected Sleep = 7.5 - 0.03*Hours of homework/week Every additional hour of weekly homework costs you about 2 minutes of sleep per night (14 minutes of sleep per week. (p=.12) Sleep versus Homework

48. Simple linear regression Expected Wake-up Time = 8:06 - 0:11*Hours of exercise/week Every additional hour of weekly exercise costs you about 11 minutes of sleep in the morning (p=.0015). Wake-up Time versus Exercise

49. More about the model… The distribution of baby weights at Stanford ~ N(3400, 360000) Your “Best guess” at a random baby’s weight, given no information about the baby, is what? 3400 grams But, what if you have relevant information? Can you make a better guess?

50. Predictor variable X=gestation time Assume that babies that gestate for longer are born heavier, all other things being equal. Pretend (at least for the purposes of this example) that this relationship is linear. Example: suppose a one-week increase in gestation, on average, leads to a 100-gram increase in birth-weight

51. Y depends on X Y=birth- weight (g) X=gestation time (weeks) Best fit line is chosen such that the sum of the squared (why squared?) distances of the points (Y i ’s) from the line is minimized:

52. Prediction A new baby is born that had gestated for just 30 weeks. What’s your best guess at the birth-weight? Are you still best off guessing 3400? NO!

53. Y=birth- weight (g) X=gestation time (weeks) At 30 weeks… 3000 30

54. Y=birth weight (g) X=gestation time (weeks) At 30 weeks… (x,y)= (30,3000) 3000 30

55. At 30 weeks… The babies that gestate for 30 weeks appear to center around a weight of 3000 grams. In Math-Speak… E(Y/X=30 weeks)=3000 grams Note the conditional expectation

56. But… Note that not every Y-value (Y i ) sits on the line. There’s variability. Y i =3000 + random error i In fact, babies that gestate for 30 weeks have birth-weights that center at 3000 grams, but vary around 3000 with some variance  2 Approximately what distribution do birth-weights follow? Normal. Y/X=30 weeks ~ N(3000,  2 )

57. Y=birth- weight (g) X=gestation time (weeks) And, if X=20, 30, or 40… 203040

58. Y=baby weights (g) X=gestation times (weeks) If X=20, 30, or 40… 203040 Y/X=40 weeks ~ N(4000,  2 ) Y/X=30 weeks ~ N(3000,  2 ) Y/X=20 weeks ~ N(2000,  2 )

59. Y=baby weights (g) X=gestation times (weeks) 203040 The standard error of Y given X is the average variability around the regression line at any given value of X. It is assumed to be equal at all values of X. S y/x

60. Linear Regression Model Y’s are modeled… Y i = 100*X + random error i Follows a normal distribution Fixed – exactly on the line

61. Review Problem 4 Using the regression equation:  Y/X = 100 grams/week*X weeks What is the expected weight of a baby born at 22 weeks? a. 2000g b. 2100g c. 2200g d. 2300g e. 2400g

62. Review Problem 4 Using the regression equation:  Y/X = 100 grams/week*X weeks What is the expected weight of a baby born at 22 weeks? a. 2000g b. 2100g c. 2200g d. 2300g e. 2400g

63. Review Problem 5 Our model predicts that: a. All babies born at 22 weeks will weigh 2200 grams. b. Babies born at 22 weeks will have a mean weight of 2200 grams with some variation. c. Both of the above. d. None of the above.

64. Review Problem 5 Our model predicts that: a. All babies born at 22 weeks will weigh 2200 grams. b. Babies born at 22 weeks will have a mean weight of 2200 grams with some variation. c. Both of the above. d. None of the above.

65. Assumptions (or the fine print) Linear regression assumes that… 1. The relationship between X and Y is linear 2. Y is distributed normally at each value of X 3. The variance of Y at every value of X is the same (homogeneity of variances)

66. Non-homogenous variance Y=birth- weight (100g) X=gestation time (weeks)

67. Residual Residual = observed value – predicted value At 33.5 weeks gestation, predicted baby weight is 3350 grams 33.5 weeks This baby was actually 3380 grams. His residual is +30 grams: 3350 grams

68. Review Problem 6 A medical journal article reported the following linear regression equation: Cholesterol = 150 + 2*(age past 40) Based on this model, what is the expected cholesterol for a 60 year old? a. 150 b. 370 c. 230 d. 190 e. 200

69. Review Problem 6 A medical journal article reported the following linear regression equation: Cholesterol = 150 + 2*(age past 40) Based on this model, what is the expected cholesterol for a 60 year old? a. 150 b. 370 c. 230 d. 190 e. 200

70. Review Problem 7 If a particular 60 year old in your study sample had a cholesterol of 250, what is his/her residual? a. +50 b. -50 c. +60 d. -60 e. 0

71. Review Problem 7 If a particular 60 year old in your study sample had a cholesterol of 250, what is his/her residual? a. +50 b. -50 c. +60 d. -60 e. 0

72. A ttest is linear regression! In our class the average drinking in the Democrats (politics 6-10, n=17) was 2.4 drinks/week; in Republicans (n=4), this value was 0.3 drinks/week. We can evaluate these data with a ttest *assuming alcohol consumption is normally distributed*:

73. As a linear regression… alcohol = 0.3 + 2.1* (1=Democrat; 0=not)

74. ANOVA is linear regression! A categorical variable with more than two groups: E.g.: very right, middle, very left (mutually exclusive)  =  (=value for very right) +  1 *(1 if middle) +  2 *(1 if very left) This is called “dummy coding”—where multiple binary variables are created to represent being in each category (or not) of a categorical variable

75. Multiple Linear Regression More than one predictor…  =  +  1 *X +  2 *W +  3 *Z Each regression coefficient is the amount of change in the outcome variable that would be expected per one-unit change of the predictor, if all other variables in the model were held constant.

76. Functions of multivariate analysis: Control for confounders Test for interactions between predictors (effect modification) Improve predictions

77. Example: multivariate linear regression What predicts wake-up time? Fit a multivariate model with both sleep and alcohol in the model… Expected Wake-up Time = 7:54 - 0:10*Hours of exercise/week +:04*drinks/week -R 2 =44% (we’ve explained 44% of the variance in wakeup time) -After adjusting for alcohol, you lose 10 minutes of sleep in the morning for each additional hour of exercise (p<.05)... -After adjusting for exercise, you gain 4 minutes of sleep in the morning for every weekly drink (p>.05)...

78. Review Problem 8 A medical journal article reported the following linear regression equation: Cholesterol = 150 + 2*(age past 40) + 10*(gender: 1=male, 0=female) Based on this model, what is the expected cholesterol for a 60 year-old man? a. 150 b. 370 c. 230 d. 190 e. 200

79. Review Problem 8 A medical journal article reported the following linear regression equation: Cholesterol = 150 + 2*(age past 40) + 10*(gender: 1=male, 0=female) Based on this model, what is the expected cholesterol for a 60 year-old man? a. 150 b. 370 c. 230 d. 190 e. 200

80. Linear Regression Coefficient (z Score) VariableSBPDBP Model 1 Total protein, % kcal-0.0346 (-1.10)-0.0568 (-3.17) Cholesterol, mg/1000 kcal0.0039 (2.46)0.0032 (3.51) Saturated fatty acids, % kcal0.0755 (1.45)0.0848 (2.86) Polyunsaturated fatty acids, % kcal0.0100 (0.24)-0.0284 (-1.22) Starch, % kcal0.1366 (4.98)0.0675 (4.34) Other simple carbohydrates, % kcal0.0327 (1.35)0.0006 (0.04) Model 2 Total protein, % kcal-0.0344 (-1.10)-0.0489 (-2.77) Cholesterol, mg/1000 kcal0.0034 (2.14)0.0029 (3.19) Saturated fatty acids, % kcal0.0786 (1.73)0.1051 (4.08) Polyunsaturated fatty acids, % kcal0.0029 (0.08)-0.0230 (-1.07) Starch, % kcal0.1149 (4.65)0.0608 (4.35) Models controlled for baseline age, race (black, nonblack), education, smoking, serum cholesterol. Table 3. Relationship of Combinations of Macronutrients to BP (SBP and DBP) for 11 342 Men, Years 1 Through 6 of MRFIT: Multiple Linear Regression Analyses Circulation. 1996 Nov 15;94(10):2417-23.

81. Total protein, % kcal -0.0346 (-1.10)-0.0568 (-3.17) Linear Regression Coefficient (z Score) VariableSBPDBP Translation: controlled for other variables in the model (as well as baseline age, race, etc.), every 1 % increase in the percent of calories coming from protein correlates with.0346 mmHg decrease in systolic BP. (NS) In math terms: SBP=  -.0346*(% protein) +  age *(Age) …+…. Also (from a separate model), every 1 % increase in the percent of calories coming from protein correlates with a.0568 mmHg decrease in diastolic BP. (significant) DBP=  - 05568*(% protein) +  age *(Age) …+….

82. Other types of multivariate regression Multiple linear regression is for normally distributed outcomes Logistic regression is for binary outcomes Cox proportional hazards regression is used when time-to-event is the outcome

83. Cautions about multivariate modeling… Overfitting Residual confounding

84. Overfitting In multivariate modeling, you can get highly significant but meaningless results if you put too many predictors in the model. The model is fit perfectly to the quirks of your particular sample, but has no predictive ability in a new sample. Example (hypothetical): In a randomized trial of an intervention to speed bone healing after fracture, researchers built a multivariate regression model to predict time to recovery in a subset of women (n=12). An automatic selection procedure came up with a model containing age, weight, use of oral contraceptives, and treatment status; the predictors were all highly significant and the model had a nearly perfect R-square of 99.5%. This is likely an example of overfitting. The researchers have fit a model to exactly their particular sample of data, but it will likely have no predictive ability in a new sample. Rule of thumb: You need at least 10 subjects for each additional predictor variable in the multivariate regression model.

85. Overfitting Pure noise variables still produce good R 2 values if the model is overfitted. The distribution of R 2 values from a series of simulated regression models containing only noise variables. (Figure 1 from: Babyak, MA. What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models. Psychosomatic Medicine 66:411-421 (2004).)

86. Overfitting example, class data… PREDICTORS OF EXERCISE HOURS PER WEEK (multivariate model): Variable Beta p-VALUE Intercept -14.74660 0.0257 Coffee 0.23441 0.0004 wakeup -0.51383 0.0715 engSAT -0.01025 0.0168 mathSAT 0.03064 0.0005 writingLove 0.88753 <.0001 sleep 0.37459 0.0490 R-Square = 0.8192 N=20, 7 parameters in the model!

87. Univariate models… Variable Betap-value Coffee 0.05916 0.3990 Wakeup -0.06587 0.8648 MathSAT -0.00021368 0.9731 EngSAT -0.01019 0.1265 Sleep -0.41185 0.4522 WritingLove 0.38961 0.0279

88. Residual confounding You cannot completely wipe out confounding simply by adjusting for variables in multiple regression unless variables are measured with zero error (which is usually impossible). Residual confounding can lead to significant effect sizes of moderate size if measurement error is high. Hypothetical Example: In a case-control study of lung cancer, researchers identified a link between alcohol drinking and cancer in smokers only. The OR was 1.3 for 1-2 drinks per day (compared with none) and 1.5 for 3+ drinks per day. Though the authors adjusted for number of cigarettes smoked per day in multivariate (logistic) regression, we cannot rule out residual confounding by level of smoking (which may be tightly linked to alcohol drinking). Questions to ask yourself: Is the effect moderate in size? Are there strong confounders in play? Was the exposure, outcome, or strong confounder measured with considerable error/lack of precision?