1. ANOVA tomescu.1@sociology.osu.edu
Advanced Statistical Methods: Continuous Variables
ANOVA

2. Total variability = Between-group + Within-group Variability
test uses 2 estimates of variance: variability of the sample means Ỹi about the overall Ỹ (between-groups estimate) to the variability of the sample observations about their separate group means (within-group estimate).
Total variability = Between-group + Within-group Variability
F = Between estimate/Within estimate
= BSS/(g-1)
WSS/(N-g)
where g = no. of groups; N = overall sample size
WSS also error sum of squares
BSS= between sum of squares; sum of squares of the difference between each group mean (sample mean) and the overall mean, Y bar; each squared difference = weighted by the sample size upon which it was based; g = numober of groups in the grouping variable;
WSS= within sum of squares (sum of sqaures are calculated within each group/sample) ; it’s weighted average of the separate sample variances, with greater weight given to larger samples
When H0 = false, the between estimate tends to overestimate the population variance (sigma squared), so it tends to be larger than the within sample estimate  significant result

3. Assumptions
Normal population distributions on the response variable for the g groups;
Equal st. dev. of the population distribution for the g groups;
Independent random samples from the g populations (one-way ANOVA)

4. Confidence Intervals comparing Means
No. of pairwise comparisons = g(g-1)/2
- If g is large (i.e. many groups), some pairs of of means may appear to be different even if all the population means are equal
For 95% CI, error probability of 0.05 = prob. that any particular CI does not contain the true difference in population means; if large no. of Cis  prob. that at least 1 CI = in error is much larger than the error prob. for any particular interval (multiple comparison error rate)
Simultaneous CI – Bonferroni CI
controls the prob. that all CIs contain the true difference
the prob. at least one set of events occurs cannot be greater than the sum of the separate probabilities of the events
Bonferroni 95% CI are wider than the separate 95% CI
For ex: population mean ideology for Republicans vs Dem Ho: (u3 – u1) =0  CI = (0.51; 0.92). We infer that population mean ideology was btw. .5 and.9 units higher for Rep than for Dem. Since the interval contains only positive numbers, we conclude that u3-u1 >0; u3>u1; On average, Rep were more conservative than Democrats
If g = 10  45 pairs of means to compare!!! If we form 95% CI for the difference btw. each pair  error probability of 0.05 applies to each comparison. Hence, for the 45 comparisons 45*0.05 = 2.25 of the CI would not contain the true differences of means
Simultaneous CI: control at the level of 0.05 the probability that at least one CI is in error (i.e. that all 45 CIs contain the pairwise differences ui-uj)
Bonferroni uses more stringent confidence level for each interval, to ensure that the overall confidence level is acceptably high. Ex: if we want a multiple comparison error rate of 0.10 (prob of 90% that all CIs are simultaneously correct) and we plan 4 comparisons of means  Bonferroni uses probability .10/4 =0.025 for each, that is a 97.5% confidence level for each interval
Ensures that that the overall error rate is at most .10 and that the overall confidence is at least .90
Bonf CI are wider because it uses a higher confidence level for each separate interval to achieve simultaneous confidence level

5. Social Class (collapsed)
Factorial Between-Subjects ANOVA
1 response variable (continuous); 2/more qualitative explanatory variables;
compare mean of response variable btw. categories of any of the IVs, controlling for the other(s)
with/without interaction effects
Ex: mean income for gender by social classes without interaction
Gender
Social Class (collapsed)
Privileged
Disadvantaged
Neutral
Male (1)
μ11
μ12
μ13
Female (0)
μ01
μ02
μ03
Social classes: Privileged = Employers, Mangers, Professionals. Disadvantaged = Skilled, Unskilled Manual Workers, Farmers; Neutral = Supervisors, Technicians and Office Workers, Service
In this table: mean income by categories of gender and of social class. We compare 6 means defined for the 2*3 =6 combinations of categories in the 2 IVs (qualitative)
Since here I do not have interaction, if I would carry out this ANOVA test, I would basically test 2 null hypotheses:
Income population means are identical across men and women, controlling for social class; I compare means for males and for females within each social class: is μ11 = μ01; is μ12 = μ02; μ13 = μ03
Income population means are identical for the three social classes, controlling for gender: for males, I compare u11, u12, …u13 are they equal?; for females: is μ01=μ02=μ03.
The effects of individual predictors (IVs) tested in these 2 null hypotheses are called main effects.

6. anova ln_sinc2008 sex2008 class_3grps_2008
corr sex class = **; N= 1295
H0: no diffr. in mean income btw. males and females, controlling for social class: b1=0; F test = Sex mean square/Means square error/residual = 0.627/0.021 with df1=1 and df2 (error;residual) = 739
Ho: no diffr. in mean income for the 3 social classes, controlling for gender b2=b3=0; F = Social Class mean square/Mean square error/residual = 2.232/0.021 = with df1 = 2; df2 (error;residual) = 739
Partial SS: are sum of squares for each IV. Represent the variability in Y explained by these variables, once the other variables are already in the model.
If the predictors are independent (such as when the same no. Of cases occur in each combination of social class and gender), these sum of squares explain separate portions of the variability in Y. They then sum up to the model SS (which is Total SS – Residual/error SS) . For survey research this rarely happens (it is common in experimental designs).
Gender = weakly but sig. Correlated with social class  the partial SS for gender and that fro social class overlap somewhat, and they do not add up exactly to the modelSS

7. Ex: mean income for gender by social classes with interaction
Tests 3 types of H0
a) are means for men & women likely to have come from same sampling distribution of means; income is averaged across social classes to eliminate that source of variation
 compare mean income for males & for females, controlling for social class
b) are means for the 3 social classes likely to have come from same sampling distribution of means, averaged across men and women?
 compare means in income for the 3 social classes, controlling for gender
c) are cell means (the means for women & the means for men within each social class) likely to have come from the same sampling distribution of differences btw. means?
c) That is, the difference between means for 2 categories of one predictor is the same for each category of the other predictor. The difference btw. Males and females in population mean income is the same for each social class. Also the difference btw. each pair of social classes in the population mean income is the same for males and females;

8. anova ln_sinc2008 sex2008 class_3grps_2008 int_cls3gr_ssex
Total 1 9 . 4 8 2 7 6 5 3
Residual
int_cls3g~x
class_~2008
sex2008
Model
Source
Partial SS df MS F Prob > F
Root MSE =
Adj R-squared =
Number of obs =
R-squared =
Remember: F-tests for the 3 primary hypotheses assume that the population distribution for each cell of the cross-classification is normal, and that the standard deviations are identical for each cells (i.e. homogeneity of variance).

9. Ŷ = a +b1sex +b2Privileged +b3Disadvantaged
Two-way ANOVA and Regression
DV = mean income;
IVs: dummy for gender (male=1; female=0)
set of dummies for social class: Privileged (yes=1; 0=else);
Disadvantaged (yes=1; 0=else)
Neutral = reference group
Ŷ = a +b1sex +b2Privileged +b3Disadvantaged
- no interaction
Next table based on Agresti & Finlay 1999, p. 457
To find the correspondence btw. means and regression parameters, we substitute the various combinations of values for the dummy variables.
Hence, for members of the „Neutral” social class who are males, mean income = μ13= a + b1*1 +b2*0 +b3*0
The assumption from regression analysis, that the conditional distributions of Y about the regression eq. are normal, with constant standard deviation implies here that the population distributions for the groups are normal, with the same st. deviation for each group  assumptions of ANOVA F test

10. μ13= a + b1  μ13= μ03 + b1 b1 = μ13 - μ03
H0, no difference between males a& females in mean income, controlling for social class: b1=0
b2: difference btw. Privileged and Neutral, controlling for gender
b3: diffr. btw. Disadvantaged & Neutral, controlling for gender
H0 no difference among social classes in mean income, controlling for gender: b2=b3=0
Gender
Social Class
Dummy Variables
Mean of Income
Sex
Priv
Disad
a+b1sex+b2Pr+b3Dis
Male
Privileged 1
μ11= a + b1 +b2
Disadvantaged
μ12= a + b1 +b3
Neutral
μ13= a + b1
Female
μ01= a + b2
μ02= a + b3
μ03 = a

11. Ho: no diffr. in mean income for the 3 social classes, controlling for gender b2=b3=0
H0: no diffr. In mean income btw. Males and females, controlling for social class: b1=0; 3 ( d r o p e ) 2 - . 6 7 1 5 9 4 8
class_3gr~b
Privileged
Disadvantaged
Neutral
Male
female
_cons
ln_sinc Coef. Std. Err t P>|t| [95% Conf. Interval]
Total
Root MSE =
Adj R-squared =
Residual
R-squared =
Model
Prob > F =
F( 3, 739) =
Source
SS df MS Number of obs =
Model SS = the between sum of square in ANOVA (i.e. the variability explained by differences among means);
Residual SS = sum of squared errors (residual SS) – variability within groups unexplained by including parameters in the model to account for the diffr. Btw. Means
MSResidual =estimate of the variance of the observations within groups
MS Model = between groups estimate of variance
F = MSModel/MSresidual = F test statistic for H0:no differences btw. Any pairs of means (equiv of F test in ‘regular’ multivariate regression)
The coeff for sex = 0.06 is the estimated difference btw. males and females in mean income, controlling for social class
The coeff for Privileged (b2) = 0.18 = estimated difference in mean income between Privileged and Neutral, controlling for gender
The coeff for Disadv (b3): = estimated difference in mean income between Disadvantaged and Neutral, controlling for gender

12. Model with Interaction
Ŷ = a +b1Sex +b2Privileged +b3Disadvantaged + b4(Priv*Sex) + b5(Disadv*Sex)
H0: b4= b5=0  F = Interaction mean square/Mean square error;residual
= 0.049/0.020 = 2.45 w df1 = 2 & df2 = 737
Total 1 9 . 4 8 2 7 6 5 3
Residual
int_cls3g~x
class_~2008
sex2008
Model
Source
Partial SS df MS F Prob > F
Root MSE =
Adj R-squared =
Number of obs =
R-squared =

13. Repeated Measures ANOVA
- when groups are not independent (i.e. have same subjects)
Ex: Opinion of Subjects about 3 Influences on Children (Agresti & Finlay 1999, p. 463)
We cannot use one-way A because the 3 samples for the categories of influence are not independent (same subjects in each sample)  treat rows and columns as different groups, and apply two-way ANOVA: each cell is a combination of a subject with an influence.  construct regression model that expresses the mean response as a function of 2 dummy variables for the 3 influences and 11 dummies for the 12 subjects.
The test for a difference in mean response among the 3 influences is the main effect test for the column variable in the 2-way ANOVA.

14. Mixed Models: Random & Fixed Effects
Ŷ = a +b1Movies +b2TV +b3Subject1 +b4Subject2 +…… b14Subject11
Ŷ = a + bj + gi where bj = effect of influence j;
gi = effect for subject I
- expresses the expected response in the cell in row i and column j additively in terms of a row main effect and a column main effect;
parameter for the last category of each variable =0 (comparison group)
main interest: estimating influence parameters (bs); not subject parameters (gs)
gs = random effects; the categories of this factor (i.e. subjects) are a random sample of all the possible ones;
bs = fixed effects: analyses refer to all the categories of interest (rather than a random sample of them) and provide inferences about differences among means for those categories

15. Often, in mixed models - more than 1 fixed effect
Ex: groups to be compared on the repeated responses ;
generally, groups have independent samples;
each time (wave), however, has the same subjects  at this level, samples are dependent
Next table from Agresti & Finlay 1999, p. 467

16. Treatment: 3 categories; defines 3 groups of girls, represented by 3 independent samples; it is a between-subjects factor: comparisons of its categories use different subjects from each group
Time: 2 main observations: before & after; each time has same subjects, so samples at its level are dependent; it is a within-subjects factor: its categories use repeated measurements within samples (i.e. same persons within respective groups);

17. treatment & time are fixed effects;
the repeated measurements on time creates a 3rd effect, a random effect for subjects
subjects are crossed with the within-subjects factor (time);
subjects are nested within the between-subjects factor (treatment)
Can test for each main effect & for interaction btw. them. Attn: error term has 2 parts (Agresti & Finaly, 1999, p.468):
Each subject is measured at every category of the within-subjects factor (time); subjects are crossed with this factor
Each subject occurs at only 1 category of the between-subjects factor: so subjects are nested within that factor
a) Based on variability among mean scores of subjects: forms error term for testing the between-subjects factor
b) Is based on how the pattern of within-subject scores varies among subjects; it forms error term for any test involving the within-subjects factor (main effects and interaction with other fixed effects)
Attn: within-subjects factor, if more than 2 levels  assumption of shpericity. One component – homogeneity of covariance (subjects line up in scores the same for all pairs of levels of the IVS; if some pairs of levels are close in time (trial 2,3) but others are distant (trial 1 and 10), is often violated. Likely to make type I error. See Tabachnik Ch 9 and especially Ch 10 on methodological problems that follow from violation of this assumption.