Multilevel Models 2 Sociology 8811, Class 24 Copyright © 2007 by Evan Schofer Do not copy or distribute without permission

Announcements Paper #2 due in 2 weeks Come see me ASAP if you don’t have a plan Unfortunately, I’m unavailable during office hours today Please send me an email to make an appointment at some other time.

Multilevel Data Simple example: 2-level data Which can be shown as: Class Which can be shown as: Class 1 S1 S2 S3 Class 2 Class 3 Level 2 Level 1

Review: Multilevel Strategies Problems of multilevel models Non-independence; correlated error Standard errors = underestimated Solutions: Each has benefits, disadvantages… 1. OLS regression 2. Aggregation (between effects model) 3. Robust Standard Errors 4. Robust Cluster Standard Errors 5. Dummy variables (Fixed Effects Model) 6. Random effects models

Example: Pro-environmental values Source: World Values Survey (27 countries) Let’s simply try OLS regression . reg supportenv age male dmar demp educ incomerel ses Source | SS df MS Number of obs = 27807 -------------+------------------------------ F( 7, 27799) = 104.06 Model | 2761.86228 7 394.551755 Prob > F = 0.0000 Residual | 105404.878 27799 3.79167876 R-squared = 0.0255 -------------+------------------------------ Adj R-squared = 0.0253 Total | 108166.74 27806 3.89005036 Root MSE = 1.9472 ------------------------------------------------------------------------------ supportenv | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | -.0021927 .000803 -2.73 0.006 -.0037666 -.0006187 male | .0960975 .0236758 4.06 0.000 .0496918 .1425032 dmar | .0959759 .02527 3.80 0.000 .0464455 .1455063 demp | -.1226363 .0254293 -4.82 0.000 -.172479 -.0727937 educ | .1117587 .0058261 19.18 0.000 .1003393 .1231781 incomerel | .0131716 .0056011 2.35 0.019 .0021931 .0241501 ses | .0922855 .0134349 6.87 0.000 .0659525 .1186186 _cons | 5.742023 .0518026 110.84 0.000 5.640487 5.843559

Dummy Variables Another solution to correlated error within groups/clusters: Add dummy variables Include a dummy variable for each Level-2 group, to explicitly model variance in means A simple version of a “fixed effects” model (see below) Ex: Student achievement; data from 3 classes Level 1: students; Level 2: classroom Create dummy variables for each class Include all but one dummy variable in the model Or include all dummies and suppress the intercept

Dummy Variables What is the consequence of adding group dummy variables? A separate intercept is estimated for each group Correlated error is absorbed into intercept Groups won’t systematically fall above or below the regression line In fact, all “between group” variation (not just error) is absorbed into the intercept Thus, other variables are really just looking at within group effects This can be good or bad, depending on your goals.

Example: Pro-environmental values Dummy variable model . reg supportenv age male dmar demp educ incomerel ses _Icountry* Source | SS df MS Number of obs = 27807 -------------+------------------------------ F( 32, 27774) = 98.50 Model | 11024.1401 32 344.504377 Prob > F = 0.0000 Residual | 97142.6001 27774 3.49760928 R-squared = 0.1019 -------------+------------------------------ Adj R-squared = 0.1009 Total | 108166.74 27806 3.89005036 Root MSE = 1.8702 ------------------------------------------------------------------------------ supportenv | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | -.0038917 .0008158 -4.77 0.000 -.0054906 -.0022927 male | .0979514 .0229672 4.26 0.000 .0529346 .1429683 dmar | .0024493 .0252179 0.10 0.923 -.046979 .0518777 demp | -.0733992 .0252937 -2.90 0.004 -.1229761 -.0238223 educ | .0856092 .0061574 13.90 0.000 .0735404 .097678 incomerel | .0088841 .0059384 1.50 0.135 -.0027554 .0205237 ses | .1318295 .0134313 9.82 0.000 .1055036 .1581554 _Icountry_32 | -.4775214 .085175 -5.61 0.000 -.6444687 -.3105742 _Icountry_50 | .3943565 .0844248 4.67 0.000 .2288798 .5598332 _Icountry_70 | .1696262 .0865254 1.96 0.050 .0000321 .3392203 … dummies omitted … _Icountr~891 | .243995 .0802556 3.04 0.002 .08669 .4012999 _cons | 5.848789 .082609 70.80 0.000 5.686872 6.010707

Dummy Variables Benefits of the dummy variable approach Weaknesses It is simple Just estimate a different intercept for each group sometimes the dummy interpretations can be of interest Weaknesses Cumbersome if you have many groups Uses up lots of degrees of freedom (not parsimonious) Makes it hard to look at other kinds of group dummies Non-varying group variables = collinear with dummies Can be problematic if your main interest is to study effects of variables across groups Dummies purge that variation… focus on within-group variation If you don’t have much within group variation, there isn’t much left to analyze.

Dummy Variables Note: Dummy variables are a simple example of a “fixed effects” model (FEM) Effect of each group is modeled as a “fixed effect” rather than a random variable Also can be thought of as the “within-group” estimator Looks purely at variation within groups Stata can do a Fixed Effects Model without the effort of using all the dummy variables Simply request the “fixed effects” estimator in xtreg.

Fixed Effects Model (FEM) For i cases within j groups Therefore aj is a separate intercept for each group It is equivalent to solely at within-group variation: X-bar-sub-j is mean of X for group j, etc Model is “within group” because all variables are centered around mean of each group.

Fixed Effects Model (FEM) . xtreg supportenv age male dmar demp educ incomerel ses, i(country) fe Fixed-effects (within) regression Number of obs = 27807 Group variable (i): country Number of groups = 26 R-sq: within = 0.0220 Obs per group: min = 511 between = 0.0368 avg = 1069.5 overall = 0.0239 max = 2154 F(7,27774) = 89.23 corr(u_i, Xb) = 0.0213 Prob > F = 0.0000 ------------------------------------------------------------------------------ supportenv | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | -.0038917 .0008158 -4.77 0.000 -.0054906 -.0022927 male | .0979514 .0229672 4.26 0.000 .0529346 .1429683 dmar | .0024493 .0252179 0.10 0.923 -.046979 .0518777 demp | -.0733992 .0252937 -2.90 0.004 -.1229761 -.0238223 educ | .0856092 .0061574 13.90 0.000 .0735404 .097678 incomerel | .0088841 .0059384 1.50 0.135 -.0027554 .0205237 ses | .1318295 .0134313 9.82 0.000 .1055036 .1581554 _cons | 5.878524 .052746 111.45 0.000 5.775139 5.981908 sigma_u | .55408807 sigma_e | 1.8701896 rho | .08069488 (fraction of variance due to u_i) F test that all u_i=0: F(25, 27774) = 94.49 Prob > F = 0.0000 Identical to dummy variable model!

ANOVA: A Digression Suppose you wish to model variable Y for j groups (clusters) Ex: Wages for different racial groups Definitions: The grand mean is the mean of all groups Y-bar The group mean is the mean of a particular sub-group of the population Y-bar-sub-j

ANOVA: Concepts & Definitions Y is the dependent variable We are looking to see if Y depends upon the particular group a person is in The effect of a group is the difference between a group’s mean & the grand mean Effect is denoted by alpha (a) If Y-bar = $8.75, YGroup 1 = $8.90, then aGroup 1= $0.15 Effect of being in group j is: It is like a deviation, but for a group.

ANOVA: Concepts & Definitions ANOVA is based on partitioning deviation We initially calculated deviation as the distance of a point from the grand mean: But, you can also think of deviation from a group mean (called “e”): Or, for any case i in group j:

ANOVA: Concepts & Definitions The location of any case is determined by: The Grand Mean, m, common to all cases The group “effect” a, common to members The distance between a group and the grand mean “Between group” variation The within-group deviation (e): called “error” The distance from group mean to an case’s value

The ANOVA Model This is the basis for a formal model: For any population with mean m Comprised of J subgroups, Nj in each group Each with a group effect a The location of any individual can be expressed as follows: Yij refers to the value of case i in group j eij refers to the “error” (i.e., deviation from group mean) for case i in group j

Sum of Squared Deviation We are most interested in two parts of model The group effects: aj Deviation of the group from the grand mean Individual case error: eij Deviation of the individual from the group mean Each are deviations that can be summed up Remember, we square deviations when summing Otherwise, they add up to zero Remember variance is just squared deviation

Sum of Squared Deviation The total deviation can partitioned into aj and eij components: That is, aj + eij = total deviation:

Sum of Squared Deviation The total deviation can partitioned into aj and eij components: The total variance (SStotal) is made up of: aj : between group variance (SSbetween) eij : within group variance (SSwithin) SStotal = SSbetween + SSwithin

ANOVA & Fixed Effects Note that the ANOVA model is similar to the fixed effects model But FEM also includes a bX term to model linear trend ANOVA Fixed Effects Model In fact, if you don’t specify any X variables, they are pretty much the same

Within Group & Between Group Models Group-effect dummy variables in regression model creates a specific estimate of group effects for all cases Bs & error are based on remaining “within group” variation We could do the opposite: ignore within-group variation and just look at differences between Stata’s xtreg command can do this, too This is essentially just modeling group means!

Between Group Model . xtreg supportenv age male dmar demp educ incomerel ses, i(country) be Between regression (regression on group means) Number of obs = 27 Group variable (i): country Number of groups = 27 R-sq: within = . Obs per group: min = 1 between = 0.2505 avg = 1.0 overall = 0.2505 max = 1 F(7,19) = 0.91 sd(u_i + avg(e_i.))= .6378002 Prob > F = 0.5216 ------------------------------------------------------------------------------ supportenv | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | .0211517 .0391649 0.54 0.595 -.0608215 .1031248 male | 3.966173 4.479358 0.89 0.387 -5.409232 13.34158 dmar | .8001333 1.127099 0.71 0.486 -1.558913 3.15918 demp | -.0571511 1.165915 -0.05 0.961 -2.497439 2.383137 educ | .3743473 .2098779 1.78 0.090 -.0649321 .8136268 incomerel | .148134 .1687438 0.88 0.391 -.2050508 .5013188 ses | -.4126738 .4916416 -0.84 0.412 -1.441691 .6163439 _cons | 2.031181 3.370978 0.60 0.554 -5.024358 9.08672 Note: Results are identical to the aggregated analysis… Note that N is reduced to 27

Fixed vs. Random Effects Dummy variables produce a “fixed” estimate of the intercept for each group But, models don’t need to be based on fixed effects Example: The error term (ei) We could estimate a fixed value for all cases This would use up lots of degrees of freedom – even more than using group dummies In fact, we would use up ALL degrees of freedom Stata output would simply report back the raw data (expressed as deviations from the constant) Instead, we model e as a random variable We assume it is normal, with standard deviation sigma.

Random Effects Issue: The dummy variable approach (ANOVA, FEM) treats group differences as a fixed effect Alternatively, we can treat it as a random effect Don’t estimate values for each case, but model it This requires making assumptions e.g., that group differences are normally distributed with a standard deviation that can be estimated from data

Random Effects A simple random intercept model Notation from Rabe-Hesketh & Skrondal 2005, p. 4-5 Random Intercept Model Where b is the main intercept Zeta (z) is a random effect for each group Allowing each of j groups to have its own intercept Assumed to be independent & normally distributed Error (e) is the error term for each case Also assumed to be independent & normally distributed Note: Other texts refer to random intercepts as uj or nj.

Linear Random Intercepts Model The random intercept idea can be applied to linear regression Often called a “random effects” model… Result is similar to FEM, BUT: FEM looks only at within group effects Aggregate models (“between effects”) looks across groups Random effects models yield a weighted average of between & within group effects It exploits between & within information, and thus can be more efficient than FEM & aggregate models. IF distributional assumptions are correct.

Linear Random Intercepts Model . xtreg supportenv age male dmar demp educ incomerel ses, i(country) re Random-effects GLS regression Number of obs = 27807 Group variable (i): country Number of groups = 26 R-sq: within = 0.0220 Obs per group: min = 511 between = 0.0371 avg = 1069.5 overall = 0.0240 max = 2154 Random effects u_i ~ Gaussian Wald chi2(7) = 625.50 corr(u_i, X) = 0 (assumed) Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ supportenv | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- age | -.0038709 .0008152 -4.75 0.000 -.0054688 -.0022731 male | .0978732 .0229632 4.26 0.000 .0528661 .1428802 dmar | .0030441 .0252075 0.12 0.904 -.0463618 .05245 demp | -.0737466 .0252831 -2.92 0.004 -.1233007 -.0241926 educ | .0857407 .0061501 13.94 0.000 .0736867 .0977947 incomerel | .0090308 .0059314 1.52 0.128 -.0025945 .0206561 ses | .131528 .0134248 9.80 0.000 .1052158 .1578402 _cons | 5.924611 .1287468 46.02 0.000 5.672272 6.17695 sigma_u | .59876138 sigma_e | 1.8701896 rho | .09297293 (fraction of variance due to u_i) Assumes normal uj, uncorrelated with X vars SD of u (intercepts); SD of e; intra-class correlation

Linear Random Intercepts Model Notes: Model can also be estimated with maximum likelihood estimation (MLE) Stata: xtreg y x1 x2 x3, i(groupid) mle Versus “re”, which specifies weighted least squares estimator Results tend to be similar But, MLE results include a formal test to see whether intercepts really vary across groups Significant p-value indicates that intercepts vary . xtreg supportenv age male dmar demp educ incomerel ses, i(country) mle Random-effects ML regression Number of obs = 27807 Group variable (i): country Number of groups = 26 … MODEL RESULTS OMITTED … /sigma_u | .5397755 .0758087 .4098891 .7108206 /sigma_e | 1.869954 .0079331 1.85447 1.885568 rho | .0769142 .019952 .0448349 .1240176 ------------------------------------------------------------------------------ Likelihood-ratio test of sigma_u=0: chibar2(01)= 2128.07 Prob>=chibar2 = 0.000

Choosing Models Which model is best? There is much discussion (e.g, Halaby 2004) Fixed effects are most consistent under a wide range of circumstances Consistent: Estimates approach true parameter values as N grows very large But, they are less efficient than random effects In cases with low within-group variation (big between group variation) and small sample size, results can be very poor Random Effects = more efficient But, runs into problems if specification is poor Esp. if X variables correlate with random group effects.

Hausman Specification Test Hausman Specification Test: A tool to help evaluate fit of fixed vs. random effects Logic: Both fixed & random effects models are consistent if models are properly specified However, some model violations cause random effects models to be inconsistent Ex: if X variables are correlated to random error In short: Models should give the same results… If not, random effects may be biased If results are similar, use the most efficient model: random effects If results diverge, odds are that the random effects model is biased. In that case use fixed effects…

Hausman Specification Test Strategy: Estimate both fixed & random effects models Save the estimates each time Finally invoke Hausman test Ex: streg var1 var2 var3, i(groupid) fe estimates store fixed hausman fixed random

Hausman Specification Test Example: Environmental attitudes fe vs re . hausman fixed random ---- Coefficients ---- | (b) (B) (b-B) sqrt(diag(V_b-V_B)) | fixed random Difference S.E. -------------+---------------------------------------------------------------- age | -.0038917 -.0038709 -.0000207 .0000297 male | .0979514 .0978732 .0000783 .0004277 dmar | .0024493 .0030441 -.0005948 .0007222 demp | -.0733992 -.0737466 .0003475 .0007303 educ | .0856092 .0857407 -.0001314 .0002993 incomerel | .0088841 .0090308 -.0001467 .0002885 ses | .1318295 .131528 .0003015 .0004153 ------------------------------------------------------------------------------ b = consistent under Ho and Ha; obtained from xtreg B = inconsistent under Ha, efficient under Ho; obtained from xtreg Test: Ho: difference in coefficients not systematic chi2(7) = (b-B)'[(V_b-V_B)^(-1)](b-B) = 2.70 Prob>chi2 = 0.9116 Direct comparison of coefficients… Non-significant p-value indicates that models yield similar results…