Introduction to Multilevel Modeling Stephen R. Porter Associate Professor Dept. of Educational Leadership and Policy Studies Iowa State University Lagomarcino Hall Ames, IA

2 Goals of the workshop Understand why multilevel modeling is important and understand basic 2-level models. Become informed consumer of multilevel research. Know how to estimate some simple models using the software package HLM. Have a thorough grounding in the basics so you can learn more complicated multi-level techniques (3- level, SEM, etc.) on your own.

3 Schedule 1 st day Review and discuss multilevel terminology and theory Begin reviewing choices in model building 2 nd day Estimate simple 2-level models using student version of HLM Discuss in detail model building.

Introduction

5 Why multilevel modeling? Nested data are very common in higher education. Analysis of nested data poses unit of analysis problem – should we analyze the individual or the group? Unfortunately, we often can’t choose one over the other. Traditional linear models offer a simple view of a complex world – generally assume same effects across groups. If effects do differ across groups, we can explain these differences with multilevel modeling.

6 Unit of analysis problem: individual, group or both? Example: studying what affects student retention (1000 students per college) in a group of colleges (n=50). Total dataset N=50,000. We can assign college-level variables to each individual, but … We end up estimating the standard errors for college-level variables using N=50,000. Yet we only have 50 different college observations, so N really equals 50.

7 Unit of analysis problem: individual, group or both? Alternatively, we can average student data for each college so that we have 1 observation per college (N=50). Now we have reduced variance on our student- level variables. We also have variables which measure both individual student characteristics (SAT score=aptitude/preparation) and college environment (average SAT score=selectivity).

8 What are nested data? Simply put, sub-units are grouped (or “nested”) within larger units. Often the data are observations of individuals nested within groups. Key: individuals within groups are more similar to one another than to individuals in other groups. We can empirically verify this. Sometimes data are multiple observations nested within an individual.

9 Students/faculty nested within departments/disciplines Note that this could be one institution, or individuals from several different institutions. Examples: student satisfaction, gains in skills; faculty salaries, research productivity.

10 Students/faculty nested within institutions Examples: student satisfaction, retention

11 Time periods nested within students Example: grade-point average

Terms and theory

13 Terminology: HLM HLM stands for hierarchical linear models. It is both a statistical technique and a software package. People also use the term multilevel models. Economists often refer to these models as random- coefficient regression models

14 Terminology: levels Level-1 variables: These are the variables that are nested within groups. Typically these are individual-level variables. Level-2 variables Typically these are unit-level variables. Note that growth models have time periods at level-1, and individuals at level-2.

15 Terminology: variance Numbers represent people, each number is a person’s question response on a 5-point Likert scale; 6 groups Variance between groups only: Variance within groups only: Variance both between and within groups:

16 Terminology: random and fixed Fixed effects are variable coefficients that are constant across groups, they do not vary. Typical OLS coefficients. Random effects are coefficients that can vary across groups. This means the coefficient can take a different value for each group. E.g., if we allow an intercept for each group, then the intercept is said to be random. It is random because we assume it is stochastic. Yet we can also explain some of this variance with other variables.

17 One way to think about multilevel models: “slopes-as-outcomes” Suppose we estimate 1 regression equation for each group, e.g., for the 1,000 students in school A, the 1,000 students in school B, etc. The result is 50 regression equations. We then take the slope coefficients for each school, as well as information about each school such as private/public status, and make a new dataset. We run a regression model on these 50 observations using the slope coefficients (or intercepts) as the dependent variable and public/private status as the independent variable. The result is a single set of coefficients for the school dataset.

18 Now for some algebra! You must learn some of the basic mathematical notation used in multilevel modeling. As we will see, the program HLM uses this notation to express the models that you estimate. Understanding these basic symbols and expressions will allow you to tackle more complex analyses, and understand other researchers’ more complex analyses.

19 A level-1 model: multiple students in one school (familiar OLS equation) Student i is viewed as having average achievement in the school, plus a positive deviation due to SES, plus a positive or negative deviation due to the unique circumstances of the student.

20 A level-1 model: multiple students in multiple schools Now we’re estimating the equation from before for each school. Each school can have a different average achievement (or intercept), and a different impact of SES on achievement (or slope).

21 Need to make some additional assumptions about the coefficients, because they vary Student-level errors are normally distributed. Gamma’s: we expect the average achievement for school j to equal the average school mean for all j schools, and the slope of SES for school j to equal the average of the slopes for all j schools. Tau’s: these are the variances of the intercepts and slopes, and the covariance between them.

22 Level-2 model: explaining the Level-1 coefficients Since our intercepts and slopes vary by school, we can now model why they vary. Suppose we hypothesize that levels of achievement and impact of SES are related to whether a school is public or Catholic. We need equations for the intercept and slope to describe our hypothesis:

23 Level-2 model: continued

24 So math achievement of an individual student in school j is explained by … mean achievement in public schools, plus impact of a school being Catholic on mean achievement (if j is Catholic) the effect of SES on achievement, plus the impact of a school being Catholic on how SES affects achievement (again, if j is Catholic) student- and school-specific error terms

25 Summary Level-1 Level-2 Explain dependent variable Explain slopes Explain intercepts

Some practical aspects of multilevel modeling

27 Questions to answer Can you use multilevel techniques to study your dependent variable? Should you use multilevel techniques to study your dependent variable? How will you center your level-1 and level-2 predictors? Which of the level-1 coefficients will be explained at level-2? I.e., are they fixed or random? How does my model perform?

28 Can I use HLM? HLM requires a large amount of data. Minimum: number of groups: 30, but most recommend 50+ number of individuals within groups: 5-10, but can have low as 1. average group size: 10, obviously more is better.

29 Should I be using HLM? How much of the variance in your dependent variable is explained by group membership? Intraclass correlation coefficient (ICC) = var between groups (var between groups+var within groups)

30 Centering variables Whether and how you center is a very important decision: interpretation of results depends on your choice. Important because the intercept at level-1 is also a dependent variable. Centering Refers to subtracting a mean from your independent variables. The transformed value for an individual measures how much they deviate (+/-) from the mean.

31 Centering variables Suppose we center verbal SAT scores around a student mean of 500. How would we interpret a regression coefficient if all variables were similarly transformed?

32 Centering variables Why would we want to center? Variable may lack a natural zero point, such as SAT score. Stability of estimates at level-1 affected by location of variables. Location at level-2 is less important.

33 Centering variables Generally two types of centering. For a specific variable: Grand mean centering – subtract the mean for the entire sample from each observation in the sample. Group mean centering – subtract the mean for each group from each member of the group. To fully understand the implications of centering, see the discussion in Bryk and Raudenbush (2002) pp

34 Fixed or random? It would be nice to have everything random; that is, a different set of coefficients for each group. But due to HLM demands on data, usually only the intercept and a few variables can be random. Important: if you randomize gender and you have a group without females, that group will be dropped. Generally you should run parallel models for intercept and slopes, as in our theory example.

35 Model statistics Goodness of fit: Proportion of variance explained at level-1 Variance explained at level-2

36 Some thoughts about building your models Before using HLM, run OLS regressions for sample and for each group. Building the null model: This is should be your first step. Calculate the ICC Building the level-1 models: Should be theory driven Step-up approach Be cautious about what you leave as random – it’s often difficult to leave more than the intercept and one variable as random

37 Some thoughts about building your models Building the level-2 models Rule of thumb: 10 observations/variable Parallel models Many scholars drop insignificant variables at both levels. (I disagree with this.)