Shenyang Guo, Ph.D. University of Tennessee Analyzing Longitudinal Rating Data: A three- level HLM Shenyang Guo, Ph.D. University of Tennessee UNC at Chapel Hill (Starting from Aug. 2002)

For details of this presentation, see Guo, S., & Hussey, D. (1999). Analyzing Longitudinal Rating Data: A three-level hierarchical linear model. Social Work Research 23(4):

You could not step twice into the same river, for other waters are ever flowing onto you. -- Heraclitus -- Heraclitus (B.C ) (B.C )

Hypothetical Data: Two raters ratings on a single subject 1a1b1c1d

A segment of data

The Generalizability Theory -- Cronbach et al. (1972) The theory effectively demonstrates that measurement error is multifaceted. Using the G theory, we may conceptualize the longitudinal rating data collected by multiple raters as a two-facet design (that is, rater and occasion) with study subjects as the object of measurement.

Sources of variability: A two-facet cross design 1.Subjects 2 s 2.Raters 2 r 3.Occasions 2 o 4.Interaction of subject and rater 2 sr 5.Interaction of subject and occasion 2 so 6.Interaction of rater and occasion 2 ro 7.Residual 2 sro,e

Sources of non-random variation associated with raters Some raters may understand the rating rules differently from others -- increased 2 r A rater may be particularly lenient in rating a particular child -- increased 2 sr A rater may give a rating more stringent one day than on other days -- increased 2 ro

A General Measurement Model Developed by Diggle, Liang, & Zeger (1995, pp.79-81) The error term of the linear model Y= X + can be expressed as ij = d' ij U i + W i (t ij ) + Z ij where d ij are r-element vectors of explanatory variables attached to individual measurements, and the U i, the {W i (t ij )} and the Z ij correspond to random effects, serial correlation and measurement error, respectively.

Estimated variance components: A two-facet nested model Variance of children 2 s : 70% Variance associated with raters 2 r,sr : 24% (Non-negligible) Variance of residual 2 o,so,or,sor,e : 6%

A two-level specification (Fixed-rater model): Level 1: Y tj = 0j + 1j (Time) tj + 2j (Residential) tj + e tj Level 2: 0j = (Teacher) j + 02 (Boy) j + u 0j 1j = j (Teacher) j 2j = 20

Adverse consequences (when the 2-level model is used): Biased estimation of coefficients, larger residuals, and poorer fit; Misleading significance tests; Misleading decomposition of variability into various sources;

A three-level specification (Random-rater model): Level 1: Y tij = 0ij + 1ij (Time) tij + 2ij (Residential) tij + e tij Level 2: 0ij = 00j + 01j (Teacher) ij + r 0ij 1ij = 10j + 11j (Teacher) ij 2ij = 20j Level 3: 00j = (Boy) j + u 00j 01j = j = j = j = 200

HLM comparison

Sample mean trajectories

Substantive results The children changed their behavior over time at a rate of decreasing the DSMD total score by.007 per day (p<.01), or in a two-year period. Initially, the average DSMD total for girls placed in a non-residential program and rated by caretakers is (p<.01). Other things being equal, boys were judged to be less behaviorally disturbed than girls at any point in time by (p<.01)

Substantive results continued Children placed in the residential program evidenced greater behavioral disturbance (1.597 higher) than others at any point in time. Teachers rated the children more positively ( lower) than caretakers at any point in time (p<.01). This also indicates that these children presented more disturbed behavior at home or in the residential milieu than in the school setting. Children rated by teachers actually increased DSMD total score at a rate of.001 per day ( =.001, p<.05), or no change over the two year span (i.e. increased.7305 in two years).

Emerging Applicability -- Situations in which the 3-level model may be useful Multi-wave panel data collected by multiple raters Multi-wave panel data collected via two methods: face-to-face and telephone interviews (e.g., AHEAD survey data) Tau-equivalent measures: graphic presentation versus a verbal questionnaire

Recent Progresses in HLM (1) -- HLM and Structural Equation Modeling (SEM) HLM and SEM share common assumptions and may yield same results. Similarities between HLM and Latent Growth Curve Modeling (Willett & Sayer, 1994).

InterceptSlope Y t=0 Y t=1 Y t=2 Y t=3 Y t=4 e0e0 e1e1 e2e2 e3e3 e4e4 didi dsds W Latent Growth Curve Model A Latent Growth Curve Model and its Equivalent HLM -- Hox (2000), in Little et al. (edited) pp HLM Y tj = 0j + 1j (Time) tj + e tj 0j = (W) j + u 0j 1j = j (W) j + u 1j

Recent Progresses in HLM (2) -- Modeling covariance structure within subjects HLM is a special case of Mixed model. In general, a linear mixed- effects model is any model that satisfies the following specifications (Laird & Ware, 1982): Y i = X i + Z i b i + i b i ~ N (0,D), i ~ N (0, i ), b 1 …, b N, 1 …, N independent, SAS Proc MIXED allows specification of the covariance structure within subjects, that is, the covariance structure of i. The choices are compound symmetric, autoregressive order one, and more (Littell et al, 1996, pp ). This is an idea borrowed from econometric time-series models.

Modeling multivariate change: whether two change trajectories (outcome measures) correlate over time? (MacCallum & Kim, 2000, in Little et al. (edited) pp ). Examples: Whether benefits clients gained from an intervention over time negatively correlate with the interventions side effects? Whether clients change in physical health correlates with their change in mental health? Whether a programs designed change in outcome (e.g., abstinence from alcohol or substance abuse) correlates with clients level of depression? Recent Progresses in HLM (3) -- Analyzing more than one dependent variable in the HLM framework

Important References (1) Laird & Ware (1982). Random-effects models for longitudinal data. Biometrics, 38, Bryk & Raudenbush (1992). Hierarchical linear models: Applications and data analysis methods. Sage Publications. Diggle, Liang, & Zeger (1995). Analysis of longitudinal data. Oxford: Clarendon Press. Littell, Milliken, Stroup, Wolfinger (1996). SAS system for mixed models. Cary: SAS Institute, Inc.

Important References (2) Verbeke & Molenberghs (2000). Linear mixed models for longitudinal data. Springer. Little, Schnabel, & Baumert (2000). Modeling longitudinal and multilevel data. Lawrence Erlbaum Associates, Publishers. Sampson, Raudenbush, & Earls (1997). Neighborhoods and violent crime: A multilevel study of collective efficacy. Science 277,