Stephen W. Raudenbush University of Chicago December 11, 2006 Adaptive Centering with Random Effects in Studies of Time-Varying Treatments Stephen W. Raudenbush University of Chicago December 11, 2006

Adaptive Centering with Random Effects in Studies of Time-Varying Treatments by Stephen W. Raudenbush University of Chicago Abstract Of widespread interest in education are observational studies in which children are exposed to interventions as they pass through classrooms and schools. The interventions might include instructional approaches, levels of teacher qualifications, or school organization. As in all observational studies, the non-randomized assignment of treatments poses challenges to valid causal inference. An attractive feature of panel studies with time-varying treatments, however, is that the design makes it possible to remove the influence of unobserved time-invariant confounders in assessing the impact of treatments. The removal of such confounding is typically achieved by including fixed effects of children and/or schools. In this paper, I introduce an alternative procedure: adaptive centering of treatment variables with random effects. I demonstrate how this alternative procedure can be specified to replicate the popular fixed effects approach in any dimension. I then argue that this alternative approach offers a number of important advantages: appropriately incorporating clustering in standard errors, modeling heterogeneity of treatment effects, improved estimation of unit-specific effects, and computational simplicity.

Claims Adaptive centering with random effects can replicate the fixed effects analysis of time-varying treatments in any dimension of clustering. Adaptive centering with random effects has several advantages Incorporating multiple sources of uncertainty Modeling heterogeneity Modeling multi-level treatments Improved estimates of unit-specific effects Computational simplicity

Table 1. Outcome data for 20 hypothetical kids by 9 teachers nested with 3 schools 4 5 6 7 8 9 x -1 w Child -2.4102 2.4628 6.2245 3.6396 4.1441 11.0898 2.1827 10.1339 12.3134 -3170 3.6596 4.8397 -.0727 1.6280 6.0525 -2.7852 1.4795 10.0131 .2350 6.0839 7.5142 -.8803 3.5167 9.7337 -1.5147 5.8636 10.2860 10 2.6814 7.6954 10.0192 11 4.4966 9.5578 11.1152 12 4.7195 8.2204 14.6855 13 4.3609 12.6474 16.8547 14 4.7778 11.9663 18.3998 15 8.5264 12.9066 18.6272 16 8.6820 11.8265 17.0661 17 9.5595 13.8078 16.3071 18 5.6075 12.7943 21.075 19 8.9094 13.5301 20.049 20 6.3465 7.3268 11.5147

Table 2 Correlations w x school id child id teacher id y w = child covariate -.23 .00 .43 .06 58 x = teacher covariate .34 -.48 .57 .14 .97 .67 -.13 -.06 .62 .58

1. “True model” Estimates of Fixed Effects Predictors β Std. Err. T p (Constant) -.415 .302 -1.375 .175 x 2.171 .200 10.866 .000 w 4.799 .278 17.294 schooled-2 3.970 .166 23.912 child id .539 .027 20.001

Methods of Estimation OLS – no control Child random effects Child fixed effects: Child random effects, within-child centering Child and school random effects Child and school fixed effects Child and school random effects, two-way centering Without teacher random effects With teacher random effects*

OLS : No Control Estimates of Fixed Effects Predictor β Std. Err. T p (Constant) 7.963 .748 10.638 .000 X 1.001 .966 1.036 .305

Child random effects “as if randomized” Estimates of Fixed Effects Parameter Estimate Std. Err. df t Sig. Intercept 7.737443 1.274501 16.067 6.071 .000 x 4.381580 .822129 47.768 5.330 Parameter Estimate σ2 12.985122 τ2 28.098578

One-Dimensional Control: OLS Fixed Child Effects Parameter Estimate Std. Error t Sig. Intercept 13.894087 2.217045 6.267 .000 x 5.498095 .865904 6.350 [childid=1.00] -17.299841 3.366029 -5.140 [childid=2.00] -11.268353 3.227033 -3.492 .001 [childid=3.00] -9.349477 -2.897 .006 [childid=4.00] -14.832045 -4.596 [childid=5.00] -11.358169 3.013434 -3.769 [childid=6.00] -12.825538 3.108690 -4.126 [childid=7.00] -11.115732 -3.576 [childid=8.00] -9.770723 -3.242 .002 [childid=9.00] -9.015820 -2.992 .005 [childid=10.00] -12.593491 -3.741 [childid=11.00] -.006149 2.886346 -.002 .998 [childid=12.00] -1.020260 2.900742 -.352 .727 [childid=13.00] -.773729 2.943507 -.263 .794 [childid=14.00] -2.179455 -.723 .474 [childid=15.00] -2.373398 -.763 .450 [childid=16.00] .463474 .157 .876 [childid=17.00] -.669300 -.222 .825 [childid=18.00] 1.097582 .373 .711 [childid=19.00] .268870 .089 .929 [childid=20.00] 0(a) . Estimates of Covariance Parameters Parameter Estimate σ2 12.496491

One-Dimensional Control: Child random effects with person-mean centered x Note this gives the same coefficient, standard error, and residual variance estimate as the student fixed effects model. Estimates of Fixed Effects Parameter Estimate Std. Err. df t Sig. Intercept 8.029549 .927088 19 8.661 .000 5.498095 .865904 39 6.350 Estimate of Covariance Parameters Parameter Estimate σ 2 12.496491 τ 2 13.024353

Table 3. Treatment Received Teacher 1 2 3 4 5 6 7 8 9 x -1 Child .6667 .3333 10 11 -.3333 12 -.6667 13 14 15 16 17 18 19 20 -0.25 0.45

Random child and school effects with x “as if randomized” Estimates of Fixed Effects Parameter Estimate Std. Err. df t Sig. Intercept 7.864998 2.493818 3.034 3.154 .050 x 2.468256 .285074 38.494 8.658 .000 Estimates of Covariance Parameters Parameter Estimate σ2 1.000617 τ2 23.759869 ψ2 15.042292

Two dimensional controls: OLS fixed child and school effects Parameter Estimate Std. Error df T Sig. Intercept 14.642231 .630345 37 23.229 .000 X 2.573106 .287937 8.936 [childid=1.00] -11.449864 .998365 -11.469 [childid=2.00] -6.393372 .946257 -6.756 [childid=3.00] -4.474496 -4.729 [childid=4.00] -9.957064 -10.523 [childid=5.00] -8.433180 .864876 -9.751 [childid=6.00] -8.925554 .901385 -9.902 [childid=7.00] -7.215747 -8.005 [childid=8.00] -6.845734 -7.915 [childid=9.00] -6.090831 -7.042 [childid=10.00] -6.743514 -6.755 [childid=11.00] -.006149 .815539 -.008 .994 [childid=12.00] -.045263 .821167 -.055 .956 [childid=13.00] 1.176263 .837825 1.404 .169 [childid=14.00] .745534 .862 .394 [childid=15.00] 1.526586 1.694 .099 [childid=16.00] 2.413467 2.881 .007 [childid=17.00] 2.255688 2.608 .013 [childid=18.00] 3.047574 3.637 .001 [childid=19.00] 3.193858 3.693 [childid=20.00] 0(a) . [schoolid=1.00] -7.679293 .367143 -20.916 [schoolid=2.00] -3.340106 .347120 -9.622 [schoolid=3.00] Estimates of Covariance Parameters Parameter Estimate σ2 .997655

Two-Dimensional Controls: Random child and school effects with interaction-contrast centering Estimates of Fixed Effects Parameter Estimate Std. Err. t Sig. Intercept 8.029463 2.851520 2.816 .083 2.573106 .287937 8.936 .000 Estimates of Covariance Parameters Parameter Estimate σ2 .997655 τ2 16.857298 ψ2 21.815022

Two-Dimensional Controls: fixed school effects, random kid effects, person-mean centered x. Estimates of Fixed Effects Parameter Estimate Std. Err. df T Sig. Intercept 11.702682 .951278 21.032 12.302 .000 2.573106 .287937 37.000 8.936 [schoolid=1.00] -7.679293 .367143 -20.916 [schoolid=2.00] -3.340106 .347120 -9.622 [schoolid=3.00] 0(a) . Estimates of Covariance Parameters Parameter Estimate σ2 0.997655 τ2 16.857298

Claims For studying time-varying treatments, adaptive centering with random effects replicates fixed effects analysis in any dimension Adaptive centering with random effects is generally the preferable approach

a. A natural way to incorporate uncertainty as a function of clustering Note we are incorporating uncertainty associated with classrooms, which cannot be done using fixed effects if the treatment is at that level.

Two-dimensional controls (kids and schools) random effects of kids, teachers within schools, schools interaction contrast for treatment Estimates of Fixed Effects Parameter Estimate Std. Err. t Sig. Intercept 8.029410 2.436174 3.296 0.170 2.573422 0.284396 9.049 .000 Parameter Estimate σ2 0.97125 τ2  16.73688 2   0.00073 ψ2 15.24546

b. A natural framework for modeling heterogeneity * Heterogeneity is interesting; * A failure to incorporate heterogeneity leads to biased standard errors.

c. We can easily study multilevel treatment and their interaction

d. Improved estimates of unit-specific effects Fixed Effects Approach via OLS

Random Effects Approach Empirical Bayes Step 1: Estimate

Random Effects Approach Step 2: Compute

Results Correlation Mean Squared Error Relative Efficiency

Role of reliability Reliability of OLS Fixed Effects In large samples,efficiency of OLS relative to EB is approximately equal to the reliability (Raudenbush, 1988, Journal of Educational Statistics).

e. Computational Ease We don’t need dummy variables to represent kids, teachers, or schools.