1. More than two groups: ANOVA and Chi-square

2. ANOVA example
Mean micronutrient intake from the school lunch by school
S1a, n=28
S2b, n=25
S3c, n=21
P-valued
Calcium (mg)
Mean
117.8
158.7
206.5
0.000
SDe
62.4
70.5
86.2
Iron (mg)
2.0
0.854 SD
0.6
Folate (μg)
26.6
38.7
42.6
13.1
14.5
15.1
Zinc (mg)
1.9
1.5
1.3
0.055
1.0
1.2
0.4
a School 1 (most deprived; 40% subsidized lunches). b School 2 (medium deprived; <10% subsidized). c School 3 (least deprived; no subsidization, private school). d ANOVA; significant differences are highlighted in bold (P<0.05).
FROM: Gould R, Russell J, Barker ME. School lunch menus and 11 to 12 year old children's food choice in three secondary schools in England-are the nutritional standards being met? Appetite Jan;46(1):86-92.

3. ANOVA for comparing means between more than 2 groups

4. ANOVA (ANalysis Of VAriance)
Idea: For two or more groups, test difference between means, for normally distributed variables.
Just an extension of the t-test (an ANOVA with only two groups is mathematically equivalent to a t-test).

5. One-Way Analysis of Variance
Assumptions, same as ttest
Normally distributed outcome
Equal variances between the groups
Groups are independent

6. Hypotheses of One-Way ANOVA

7. ANOVA It’s like this: If I have three groups to compare:
I could do three pair-wise ttests, but this would increase my type I error
So, instead I want to look at the pairwise differences “all at once.”
To do this, I can recognize that variance is a statistic that let’s me look at more than one difference at a time…

8. The “F-test” Is the difference in the means of the groups more
than background noise (=variability within groups)?
Summarizes the mean differences between all groups at once.
Analogous to pooled variance from a ttest.

9. The F-distribution A ratio of variances follows an F-distribution:
The F-test tests the hypothesis that two variances are equal.
F will be close to 1 if sample variances are equal.

10. ANOVA example
Randomize 33 subjects to three groups: 800 mg calcium supplement vs mg calcium supplement vs. placebo.
Compare the spine bone density of all 3 groups after 1 year.

11. Spine bone density vs. treatment
1.2
Within group variability
1.1
Between group variation
1.0 S
Within group variability P
Within group variability I N E
0.9
0.8
0.7
PLACEBO
800mg CALCIUM
1500 mg CALCIUM

12. Group means and standard deviations
Placebo group (n=11):
Mean spine BMD = .92 g/cm2
standard deviation = .10 g/cm2
800 mg calcium supplement group (n=11)
Mean spine BMD = .94 g/cm2
standard deviation = .08 g/cm2
1500 mg calcium supplement group (n=11)
Mean spine BMD =1.06 g/cm2
standard deviation = .11 g/cm2

13. The F-Test The size of the groups. Between-group variation.
The difference of each group’s mean from the overall mean.
Each group’s variance.
The average amount of variation within groups.
Large F value indicates that the between group variation exceeds the within group variation (=the background noise).

14. Review Question 1 Which of the following is an assumption of ANOVA?
The outcome variable is normally distributed.
The variance of the outcome variable is the same in all groups.
The groups are independent.
All of the above.
None of the above.

15. Review Question 1 Which of the following is an assumption of ANOVA?
The outcome variable is normally distributed.
The variance of the outcome variable is the same in all groups.
The groups are independent.
All of the above.
None of the above.

16. ANOVA summary
A statistically significant ANOVA (F-test) only tells you that at least two of the groups differ, but not which ones differ.
Determining which groups differ (when it’s unclear) requires more sophisticated analyses to correct for the problem of multiple comparisons…

17. Question: Why not just do 3 pairwise ttests?
Answer: because, at an error rate of 5% each test, this means you have an overall chance of up to 1-(.95)3= 14% of making a type-I error (if all 3 comparisons were independent)
 If you wanted to compare 6 groups, you’d have to do 15 pairwise ttests; which would give you a high chance of finding something significant just by chance.

18. Multiple comparisons

19. Correction for multiple comparisons
How to correct for multiple comparisons post-hoc…
Bonferroni correction (adjusts p by most conservative amount; assuming all tests independent, divide p by the number of tests)
Tukey (adjusts p)
Scheffe (adjusts p)

20. 1. Bonferroni
For example, to make a Bonferroni correction, divide your desired alpha cut-off level (usually .05) by the number of comparisons you are making. Assumes complete independence between comparisons, which is way too conservative.
Obtained P-value
Original Alpha
# tests
New Alpha
Significant? 
.001
.05 5
.010
Yes
.011 4
.013
.019 3
.017 No
.032 2
.025
.048 1
.050

21. 2/3. Tukey and Sheffé
Both methods increase your p-values to account for the fact that you’ve done multiple comparisons, but are less conservative than Bonferroni (let computer calculate for you!).

22. Review Question 2 All of the treatment regimens differ.
I am doing an RCT of 4 treatment regimens for blood pressure. At the end of the day, I compare blood pressures in the 4 groups using ANOVA. My p-value is .03. I conclude:
All of the treatment regimens differ.
I need to use a Bonferroni correction.
One treatment is better than all the rest.
At least one treatment is different from the others.
In pairwise comparisons, no treatment will be different.

23. Review Question 2 All of the treatment regimens differ.
I am doing an RCT of 4 treatment regimens for blood pressure. At the end of the day, I compare blood pressures in the 4 groups using ANOVA. My p-value is .03. I conclude:
All of the treatment regimens differ.
I need to use a Bonferroni correction.
One treatment is better than all the rest.
At least one treatment is different from the others.
In pairwise comparisons, no treatment will be different.

24. Chi-square test of Independence
Chi-square test allows you to compare proportions between 2 or more groups (ANOVA for means; chi-square for proportions).

25. Example
Asch, S.E. (1955). Opinions and social pressure. Scientific American, 193,

26. The Experiment
A Subject volunteers to participate in a “visual perception study.”
Everyone else in the room is actually a conspirator in the study (unbeknownst to the Subject).
The “experimenter” reveals a pair of cards…

27. The Task Cards
Standard line
Comparison lines
A, B, and C

28. The Experiment
Everyone goes around the room and says which comparison line (A, B, or C) is correct; the true Subject always answers last – after hearing all the others’ answers.
The first few times, the 7 “conspirators” give the correct answer.
Then, they start purposely giving the (obviously) wrong answer.
75% of Subjects tested went along with the group’s consensus at least once.

29. Further Results
In a further experiment, group size (number of conspirators) was altered from 2-10.
Does the group size alter the proportion of subjects who conform?

30. Number of group members?
The Chi-Square test
Conformed?
Number of group members? 2 4 6 8 10
Yes 20 50 75 60 30 No 80 25 40 70
Apparently, conformity less likely when less or more group members…

31. = 235 conformed
out of 500 experiments.
Overall likelihood of conforming = 235/500 = .47

32. Number of group members?
Expected frequencies if no association between group size and conformity…
Conformed?
Number of group members? 2 4 6 8 10
Yes 47 No 53

33. Do observed and expected differ more than expected due to chance?
Do observed and expected differ more than expected due to chance?

34. Chi-Square test
Degrees of freedom = (rows-1)*(columns-1)=(2-1)*(5-1)=4

35. Chi-Square test
Degrees of freedom = (rows-1)*(columns-1)=(2-1)*(5-1)=4
Rule of thumb: if the chi-square statistic is much greater than it’s degrees of freedom, indicates statistical significance. Here 85>>4.

36. Interpretation
Group size and conformity are not independent, for at least some categories of group size
The proportion who conform differs between at least two categories of group size
Global test (like ANOVA) doesn’t tell you which categories of group size differ

37. Review Question 8 Repeated-measures ANOVA. One-way ANOVA.
I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare years of schooling (a normally distributed variable) between the three groups. What test should I use?
Repeated-measures ANOVA.
One-way ANOVA.
Difference in proportions test.
Paired ttest.
Chi-square test.

38. Review Question 8 Repeated-measures ANOVA. One-way ANOVA.
I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare years of schooling (a normally distributed variable) between the three groups. What test should I use?
Repeated-measures ANOVA.
One-way ANOVA.
Difference in proportions test.
Paired ttest.
Chi-square test.

39. Review Question 9 Repeated-measures ANOVA. One-way ANOVA.
I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare the proportions of each group that went to graduate school. What test should I use?
Repeated-measures ANOVA.
One-way ANOVA.
Difference in proportions test.
Paired ttest.
Chi-square test.

40. Review Question 9 Repeated-measures ANOVA. One-way ANOVA.
I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare the proportions of each group that went to graduate school. What test should I use?
Repeated-measures ANOVA.
One-way ANOVA.
Difference in proportions test.
Paired ttest.
Chi-square test.

41. Practice Problem:
Suppose the following data were collected on a random (cross-sectional) sample of teenagers (exposure divided into high and low at the median):
Ever tried smoking
Never smoked
Exposed to high occurrence of smoking in films
100
300
Lower exposure to smoking in films 30
370
Is there an association between exposure to smoking in movies and smoking behavior?

42. Chi-square test for a 2x2 table…
Ever Smoked
Never smoked
High exposure smoking in film
100
300
400
Low exposure 30
370
130
670
800
psmoker*phigh exposure=.1625*.50=.08125
Expected in cell a=.08125*800=65
65 in cell c; 335 in cell b; 335 in cell d 65
335
EXPECTED
TABLE:

43. Smoking data, use Chi-square test
Ever Smoked
Never smoked
High exposure smoking in film
100
300
Low exposure 30
370 65
335
EXPECTED TABLE:

44. Caveat
**When the sample size is very small in any cell (<5), Fisher’s exact test is used as an alternative to the chi-square test.

45. Overview of statistical tests
The following table gives the appropriate choice of a statistical test or measure of association for various types of data (outcome variables and predictor variables) by study design.
e.g., blood pressure= pounds + age + treatment (1/0)
Continuous outcome
Binary predictor
Continuous predictors

46. Statistical procedure or measure of association
Types of variables to be analyzed
Outcome variable
Predictor variable/s
Cross-sectional/case-control studies
Binary
Continuous
T-test
Ranks/ordinal
Wilcoxon sum-rank   
Categorical
Continuous
ANOVA
Continuous
Simple linear regression
Multivariate
(categorical and continuous)
Continuous
Multiple linear regression
Multivariate
(categorical and continuous)
Continuous
Multiple linear regression 
Categorical
Chi-square test (or Fisher’s exact) 
Binary
Odds ratio, risk ratio
Cohort Studies/Clinical Trials
Multivariate
Dichotomous
Logistic regression
Multivariate
Dichotomous
Logistic regression 
Binary
Risk ratio
Categorical
Time-to-event
Kaplan-Meier curve/ log-rank test
Multivariate
Time-to-event
Cox-proportional hazards regression, hazard ratio
Multivariate
Time-to-event
Cox-proportional hazards regression, hazard ratio

47. Common statistics for various types of outcome data
Outcome Variable
Are comparison groups independent or correlated?
Assumptions
independent
repeated or paired data
Continuous
(e.g. pain scale, cognitive function)
Ttest
ANOVA
Linear correlation
Linear regression
Paired ttest
Repeated-measures ANOVA
Mixed models/GEE modeling
Outcome is normally distributed (important for small samples).
Outcome and predictor have a linear relationship.
Binary or categorical
(e.g. fracture yes/no)
Difference in proportions
Relative risks
Chi-square test
Logistic regression
McNemar’s test
Conditional logistic regression
GEE modeling
Chi-square test assumes sufficient numbers in each cell (>=5)
Time-to-event
(e.g. time to fracture)
Kaplan-Meier statistics
Cox regression
n/a
Cox regression assumes proportional hazards between groups

48. Continuous outcome (means)
Outcome Variable
Are comparison groups independent or correlated?
Alternatives if the normality assumption is violated (and small sample size):
independent
repeated or paired data
Continuous
(e.g. pain scale, cognitive function)
Ttest: compares means between two independent groups
ANOVA: compares means between more than two independent groups
Pearson’s correlation coefficient (linear correlation): shows linear correlation between two continuous variables
Linear regression: multivariate regression technique used when the outcome is continuous; gives slopes
Paired ttest: compares means between two related groups (e.g., the same subjects before and after)
Repeated-measures ANOVA: compares changes over time in the means of two or more groups (repeated measurements)
Mixed models/GEE modeling: multivariate regression techniques to compare changes over time between two or more groups; gives rate of change over time
Non-parametric statistics
Wilcoxon sum-rank test (=Mann-Whitney U test): non-parametric alternative to the ttest
Kruskal-Wallis test: non-parametric alternative to ANOVA
Spearman rank correlation coefficient: non-parametric alternative to Pearson’s correlation coefficient

49. Binary or categorical outcomes (proportions)
Outcome Variable
Are comparison groups independent or correlated?
Alternative to the chi-square test if sparse cells:
independent
repeated or paired data
Binary or categorical
(e.g. fracture yes/no)
Difference in proportions: compares proportions between two independent groups
Relative risks: odds ratios or risk ratios
Chi-square test: compares proportions between more than two groups
Logistic regression: multivariate technique used when outcome is binary; gives multivariate-adjusted odds ratios
McNemar’s test: compares binary outcome between correlated groups (e.g., before and after)
Conditional logistic regression: multivariate regression technique for a binary outcome when groups are correlated (e.g., matched data)
GEE modeling: multivariate regression technique for a binary outcome when groups are correlated (e.g., repeated measures)
Fisher’s exact test: compares proportions between groups when there are sparse data (some cells <5).

50. Time-to-event outcome (survival data)
Outcome Variable
Are comparison groups independent or correlated?
Modifications to Cox regression if proportional-hazards is violated:
independent
repeated or paired data
Time-to-event (e.g., time to fracture)
Kaplan-Meier statistics: estimates survival functions for each group (usually displayed graphically); compares survival functions with log-rank test
Cox regression: Multivariate technique for time-to-event data; gives multivariate-adjusted hazard ratios
n/a (already over time)
Time-dependent predictors or time-dependent hazard ratios (tricky!)

51. Homework
Continue reading textbook
Problem Set 7
Journal Article