1. Multilevel Models 2 Sociology 8811, Class 24
Copyright © 2007 by Evan Schofer
Do not copy or distribute without permission

2. Announcements Paper #2 due in 2 weeks
Come see me ASAP if you don’t have a plan
Unfortunately, I’m unavailable during office hours today
Please send me an to make an appointment at some other time.

3. Multilevel Data Simple example: 2-level data Which can be shown as:
Class
Which can be shown as:
Class 1 S1 S2 S3
Class 2
Class 3
Level 2
Level 1

4. Review: Multilevel Strategies
Problems of multilevel models
Non-independence; correlated error
Standard errors = underestimated
Solutions:
Each has benefits, disadvantages…
1. OLS regression
2. Aggregation (between effects model)
3. Robust Standard Errors
4. Robust Cluster Standard Errors
5. Dummy variables (Fixed Effects Model)
6. Random effects models

5. Example: Pro-environmental values
Source: World Values Survey (27 countries)
Let’s simply try OLS regression
. reg supportenv age male dmar demp educ incomerel ses
Source | SS df MS Number of obs =
F( 7, 27799) =
Model | Prob > F =
Residual | R-squared =
Adj R-squared =
Total | Root MSE =
supportenv | Coef. Std. Err t P>|t| [95% Conf. Interval]
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
_cons |

6. Dummy Variables
Another solution to correlated error within groups/clusters: Add dummy variables
Include a dummy variable for each Level-2 group, to explicitly model variance in means
A simple version of a “fixed effects” model (see below)
Ex: Student achievement; data from 3 classes
Level 1: students; Level 2: classroom
Create dummy variables for each class
Include all but one dummy variable in the model
Or include all dummies and suppress the intercept

7. Dummy Variables
What is the consequence of adding group dummy variables?
A separate intercept is estimated for each group
Correlated error is absorbed into intercept
Groups won’t systematically fall above or below the regression line
In fact, all “between group” variation (not just error) is absorbed into the intercept
Thus, other variables are really just looking at within group effects
This can be good or bad, depending on your goals.

8. Example: Pro-environmental values
Dummy variable model
. reg supportenv age male dmar demp educ incomerel ses _Icountry*
Source | SS df MS Number of obs =
F( 32, 27774) =
Model | Prob > F =
Residual | R-squared =
Adj R-squared =
Total | Root MSE =
supportenv | Coef. Std. Err t P>|t| [95% Conf. Interval]
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
_Icountry_32 |
_Icountry_50 |
_Icountry_70 |
… dummies omitted …
_Icountr~891 |
_cons |

9. Dummy Variables Benefits of the dummy variable approach Weaknesses
It is simple
Just estimate a different intercept for each group
sometimes the dummy interpretations can be of interest
Weaknesses
Cumbersome if you have many groups
Uses up lots of degrees of freedom (not parsimonious)
Makes it hard to look at other kinds of group dummies
Non-varying group variables = collinear with dummies
Can be problematic if your main interest is to study effects of variables across groups
Dummies purge that variation… focus on within-group variation
If you don’t have much within group variation, there isn’t much left to analyze.

10. Dummy Variables
Note: Dummy variables are a simple example of a “fixed effects” model (FEM)
Effect of each group is modeled as a “fixed effect” rather than a random variable
Also can be thought of as the “within-group” estimator
Looks purely at variation within groups
Stata can do a Fixed Effects Model without the effort of using all the dummy variables
Simply request the “fixed effects” estimator in xtreg.

11. Fixed Effects Model (FEM)
For i cases within j groups
Therefore aj is a separate intercept for each group
It is equivalent to solely at within-group variation:
X-bar-sub-j is mean of X for group j, etc
Model is “within group” because all variables are centered around mean of each group.

12. Fixed Effects Model (FEM)
. xtreg supportenv age male dmar demp educ incomerel ses, i(country) fe
Fixed-effects (within) regression Number of obs =
Group variable (i): country Number of groups =
R-sq: within = Obs per group: min =
between = avg =
overall = max =
F(7,27774) =
corr(u_i, Xb) = Prob > F =
supportenv | Coef. Std. Err t P>|t| [95% Conf. Interval]
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
_cons |
sigma_u |
sigma_e |
rho | (fraction of variance due to u_i)
F test that all u_i=0: F(25, 27774) = Prob > F =
Identical to dummy variable model!

13. ANOVA: A Digression
Suppose you wish to model variable Y for j groups (clusters)
Ex: Wages for different racial groups
Definitions:
The grand mean is the mean of all groups
Y-bar
The group mean is the mean of a particular sub-group of the population
Y-bar-sub-j

14. ANOVA: Concepts & Definitions
Y is the dependent variable
We are looking to see if Y depends upon the particular group a person is in
The effect of a group is the difference between a group’s mean & the grand mean
Effect is denoted by alpha (a)
If Y-bar = $8.75, YGroup 1 = $8.90, then aGroup 1= $0.15
Effect of being in group j is:
It is like a deviation, but for a group.

15. ANOVA: Concepts & Definitions
ANOVA is based on partitioning deviation
We initially calculated deviation as the distance of a point from the grand mean:
But, you can also think of deviation from a group mean (called “e”):
Or, for any case i in group j:

16. ANOVA: Concepts & Definitions
The location of any case is determined by:
The Grand Mean, m, common to all cases
The group “effect” a, common to members
The distance between a group and the grand mean
“Between group” variation
The within-group deviation (e): called “error”
The distance from group mean to an case’s value

17. The ANOVA Model This is the basis for a formal model:
For any population with mean m
Comprised of J subgroups, Nj in each group
Each with a group effect a
The location of any individual can be expressed as follows:
Yij refers to the value of case i in group j
eij refers to the “error” (i.e., deviation from group mean) for case i in group j

18. Sum of Squared Deviation
We are most interested in two parts of model
The group effects: aj
Deviation of the group from the grand mean
Individual case error: eij
Deviation of the individual from the group mean
Each are deviations that can be summed up
Remember, we square deviations when summing
Otherwise, they add up to zero
Remember variance is just squared deviation

19. Sum of Squared Deviation
The total deviation can partitioned into aj and eij components:
That is, aj + eij = total deviation:

20. Sum of Squared Deviation
The total deviation can partitioned into aj and eij components:
The total variance (SStotal) is made up of:
aj : between group variance (SSbetween)
eij : within group variance (SSwithin)
SStotal = SSbetween + SSwithin

21. ANOVA & Fixed Effects
Note that the ANOVA model is similar to the fixed effects model
But FEM also includes a bX term to model linear trend
ANOVA
Fixed Effects Model
In fact, if you don’t specify any X variables, they are pretty much the same

22. Within Group & Between Group Models
Group-effect dummy variables in regression model creates a specific estimate of group effects for all cases
Bs & error are based on remaining “within group” variation
We could do the opposite: ignore within-group variation and just look at differences between
Stata’s xtreg command can do this, too
This is essentially just modeling group means!

23. Between Group Model
. xtreg supportenv age male dmar demp educ incomerel ses, i(country) be
Between regression (regression on group means) Number of obs =
Group variable (i): country Number of groups =
R-sq: within = Obs per group: min =
between = avg =
overall = max =
F(7,19) =
sd(u_i + avg(e_i.))= Prob > F =
supportenv | Coef. Std. Err t P>|t| [95% Conf. Interval]
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
_cons |
Note: Results are identical to the aggregated analysis… Note that N is reduced to 27

24. Fixed vs. Random Effects
Dummy variables produce a “fixed” estimate of the intercept for each group
But, models don’t need to be based on fixed effects
Example: The error term (ei)
We could estimate a fixed value for all cases
This would use up lots of degrees of freedom – even more than using group dummies
In fact, we would use up ALL degrees of freedom
Stata output would simply report back the raw data (expressed as deviations from the constant)
Instead, we model e as a random variable
We assume it is normal, with standard deviation sigma.

25. Random Effects
Issue: The dummy variable approach (ANOVA, FEM) treats group differences as a fixed effect
Alternatively, we can treat it as a random effect
Don’t estimate values for each case, but model it
This requires making assumptions
e.g., that group differences are normally distributed with a standard deviation that can be estimated from data

26. Random Effects A simple random intercept model
Notation from Rabe-Hesketh & Skrondal 2005, p. 4-5
Random Intercept Model
Where b is the main intercept
Zeta (z) is a random effect for each group
Allowing each of j groups to have its own intercept
Assumed to be independent & normally distributed
Error (e) is the error term for each case
Also assumed to be independent & normally distributed
Note: Other texts refer to random intercepts as uj or nj.

27. Linear Random Intercepts Model
The random intercept idea can be applied to linear regression
Often called a “random effects” model…
Result is similar to FEM, BUT:
FEM looks only at within group effects
Aggregate models (“between effects”) looks across groups
Random effects models yield a weighted average of between & within group effects
It exploits between & within information, and thus can be more efficient than FEM & aggregate models.
IF distributional assumptions are correct.

28. Linear Random Intercepts Model
. xtreg supportenv age male dmar demp educ incomerel ses, i(country) re
Random-effects GLS regression Number of obs =
Group variable (i): country Number of groups =
R-sq: within = Obs per group: min =
between = avg =
overall = max =
Random effects u_i ~ Gaussian Wald chi2(7) =
corr(u_i, X) = 0 (assumed) Prob > chi =
supportenv | Coef. Std. Err z P>|z| [95% Conf. Interval]
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
_cons |
sigma_u |
sigma_e |
rho | (fraction of variance due to u_i)
Assumes normal uj, uncorrelated with X vars
SD of u (intercepts); SD of e; intra-class correlation

29. Linear Random Intercepts Model
Notes: Model can also be estimated with maximum likelihood estimation (MLE)
Stata: xtreg y x1 x2 x3, i(groupid) mle
Versus “re”, which specifies weighted least squares estimator
Results tend to be similar
But, MLE results include a formal test to see whether intercepts really vary across groups
Significant p-value indicates that intercepts vary
. xtreg supportenv age male dmar demp educ incomerel ses, i(country) mle
Random-effects ML regression Number of obs =
Group variable (i): country Number of groups =
… MODEL RESULTS OMITTED …
/sigma_u |
/sigma_e |
rho |
Likelihood-ratio test of sigma_u=0: chibar2(01)= Prob>=chibar2 = 0.000

30. Choosing Models Which model is best?
There is much discussion (e.g, Halaby 2004)
Fixed effects are most consistent under a wide range of circumstances
Consistent: Estimates approach true parameter values as N grows very large
But, they are less efficient than random effects
In cases with low within-group variation (big between group variation) and small sample size, results can be very poor
Random Effects = more efficient
But, runs into problems if specification is poor
Esp. if X variables correlate with random group effects.

31. Hausman Specification Test
Hausman Specification Test: A tool to help evaluate fit of fixed vs. random effects
Logic: Both fixed & random effects models are consistent if models are properly specified
However, some model violations cause random effects models to be inconsistent
Ex: if X variables are correlated to random error
In short: Models should give the same results… If not, random effects may be biased
If results are similar, use the most efficient model: random effects
If results diverge, odds are that the random effects model is biased. In that case use fixed effects…

32. Hausman Specification Test
Strategy: Estimate both fixed & random effects models
Save the estimates each time
Finally invoke Hausman test
Ex:
streg var1 var2 var3, i(groupid) fe
estimates store fixed
hausman fixed random

33. Hausman Specification Test
Example: Environmental attitudes fe vs re
. hausman fixed random
---- Coefficients ----
| (b) (B) (b-B) sqrt(diag(V_b-V_B))
| fixed random Difference S.E.
age |
male |
dmar |
demp |
educ |
incomerel |
ses |
b = consistent under Ho and Ha; obtained from xtreg
B = inconsistent under Ha, efficient under Ho; obtained from xtreg
Test: Ho: difference in coefficients not systematic
chi2(7) = (b-B)'[(V_b-V_B)^(-1)](b-B) =
Prob>chi2 =
Direct comparison of coefficients…
Non-significant p-value indicates that models yield similar results…