1. Analyzing Data from Small N Designs using Multilevel Models Eden Nagler The Graduate Center, CUNY David Rindskopf, Ph.D The Graduate Center, CUNY

2. 2 Overview/Intro  What is our current work?  Where did we start?  How does HLM fit into this framework?

3. 3 2 Initial Datasets: Stuart, R.B. (1967). Behavioral control of overeating. Behavior Research & Therapy, 5, (357-365). Dicarlo, C.F. & Reid, D.H. (2004). Increasing pretend toy play of toddlers with disabilities in an inclusive setting. Journal of Applied Behavior Analysis, 37(2), (197-207).

4. 4 Stuart (1967):

5. 5 Stuart (1967): Procedures for Getting data into HLM

6. 6

7. 7 Stuart (1967): Level-1 dataset

8. 8 Stuart (1967): Level-2 dataset

9. 9 Stuart (1967): HLM (Linear model) Linear Model: POUNDS = π 0 + π 1 *(MONTHS12) + e

10. 10 Stuart (1967): HLM – Linear Model Estimates Final estimation of fixed effects: Standard Approx. Fixed Effect CoefficientError T-ratiod.f.P-value ---------------------------------------------------------- For INTRCPT1,P0 INTRCPT2, B00 156.439560 5.053645 30.956 7 0.000 For MONTHS12 slope, P1 INTRCPT2, B10 -3.078984 0.233772 13.171 7 0.000 ---------------------------------------------------------- The outcome variable is POUNDS ---------------------------------------------------------- POUNDS ij ≈ 156.4 – 3.1*(MONTHS12) + e ij

11. 11 Stuart (1967): HLM – Quadratic Model Quadratic Model: POUNDS = π 0 + π 1 *(MONTHS12)+ π 2 *(MON12SQ)+e

12. 12 Stuart (1967): HLM – Quadratic Model Estimates Final estimation of fixed effects: Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ----------------------------------------------------------- For INTRCPT1, P0 INTRCPT2, B00 158.833791 5.321806 29.846 7 0.000 For MONTHS12 slope, P1 INTRCPT2, B10 -1.773039 0.358651 -4.944 7 0.001 For MON12SQ slope, P2 INTRCPT2, B20 0.108829 0.021467 5.070 7 0.001 ----------------------------------------------------------- The outcome variable is POUNDS ----------------------------------------------------------- POUNDS ij ≈ 158.8 – 1.8(MONTHS12) + 0.1*(MON12SQ) + e ij

13. 13 Stuart (1967): HLM – Linear vs. Quadratic Model Stuart (1967) – Actual Data Quadratic Model Prediction Linear Model Prediction

14. 14 Dicarlo & Reid (2004):

15. 15 Dicarlo & Reid (2004): Level-1 dataset

16. 16 Dicarlo & Reid (2004): Level-2 dataset

17. 17 Dicarlo & Reid (2004): HLM – Simple Model Simple Model: FREQRND = π 0 + π 1 *(PHASE) + e

18. 18 Dicarlo & Reid (2004): HLM – Simple Model Estimates Level-1 ModelLevel-2 Model log[L] = P0 + P1*(PHASE) P0 = B00 + R0 P1 = B10 + R1 ---------------------------------------------------------- Final estimation of fixed effects: (Unit-specific model) Standard Approx. Fixed Effect Coefficient Error T-ratiod.f.P-value ---------------------------------------------------------- For INTRCPT1,P0 INTRCPT2, B00 -0.7693840.634548 -1.212 4 0.292 For PHASE slope,P1 INTRCPT2, B10 2.516446 0.278095 9.049 4 0.000 ---------------------------------------------------------- LN(FREQRND ij ) = -0.77 + 2.52*(PHASE) + e ij

19. 19 Dicarlo & Reid (2004): HLM – Simple Model Estimates LOG(FREQRND ij ) = B 00 + B 10 *(PHASE) + e ij For PHASE=0 (BASELINE): LOG(FREQRND ij ) = B 00 FREQRND ij = exp(B 00 ) For PHASE=1 (TREATMENT): LOG(FREQRND ij ) = B 00 + B 10 FREQRND ij = exp(B 00 +B 10 ) = exp(B 00 )*exp(B 10 ) Estimates: B 00 = -0.77; B 10 = 2.52 For PHASE=0 (BASELINE): FREQRND ij = exp(B 00 ) = exp(-0.77) = 0.46 For PHASE=1 (TREATMENT): FREQRND ij = exp(B 00 +B 10 ) = exp(-0.77+2.52) = exp(1.75) = 5.75

20. 20 In conclusion… 1. Other issues we’ve encountered and explored 2. Issues we’ve encountered, but not yet explored 3. Issues we’ve not yet encountered nor explored