Statistical Models for the Analysis of Single-Case Intervention Data Introduction to:  Regression Models  Multilevel Models

Why consider statistical models? Can provide effect size estimates and confidence intervals for those estimates. e.g., We are 95% confident the immediate shift in level for Jenny was an increase of between 11 and 14 minutes of time spent reading, or between 1.5 and 2.0 standard deviations

Regression Imagine a scatter plot showing the relationship between motivation and achievement.

Regression allows us to summarize the relationship between the variables.

Often when we think of regression we think of each data point coming from a different individual, but all the observations could come from the same individual.

Rise Run What is the rate of change for Jody?

For single-case studies we expect a discontinuity BaselineIntervention

What is the shift in level for Jody? Effect =

What is the immediate shift in level and the shift in slope for Jody?

Issues to Keep in Mind You may have to choose at what point in time you should calculate the effect size

You may want to standardize the effect size If so, what SD should be use?

There needs to be a match between the trajectory specified in the model and what is seen in the data Effect = b 1 ? This seems incorrect

What is seen may require specification of a complex growth trajectory Do you think specification tends to be easier when there are more or less observations in a phase?

Correct model specification requires more than just correctly specifying the growth trajectory Should you assume the errors (e i ): are independent? have common variance? are normally distributed?

Cindy Lucy George John Imagine we have multiple cases

A separate regression could be obtained for each case Or a multilevel analysis could be run

Multilevel Model Multilevel models allow us to answer additional questions: What is the average treatment effect? What is the average treatment effect? Does the size of the effect vary across participants? Does the size of the effect vary across participants? What factors relate to effect size? What factors relate to effect size?

What is the average effect for the participants? Average Effect = γ 10 Average Baseline Level = γ 00

Does the size of the effect vary across participants?

What factors relate to effect size? ADD Non-ADD

Issues to Keep in Mind There still needs to be a match between the trajectory specified in the model and what is seen in the data Effect = b 1 ? This seems incorrect

Correct model specification requires assumptions about multiple error terms Should you assume the errors (e ij, r 0j, r 1j ) are independent? Normally distributed?

If one standardizes the effect size, what SD should be used for standardization Within case variance? Between case variance?

Imagine we have multiple studies 24

Multilevel models were developed for large sample size conditions, but single-case applications tend to have a very small number of cases. Given small sample sizes the variances (e.g. variance in the treatment effect across participants) will generally be more poorly estimated than the averages (e.g. the average treatment effect).

Example Analysis Summarize results from 5 studies that examined the effect of intervention on autistic children’s speech DV: Percent intervals with child speech IV: Intervention based on increased parent verbalizations Design: Multiple baseline across participants

Laski, K. E., Charlop, M. H., & Schreibman, L. (1988). Training parents to use the natural language paradigm to increase their autistic children’s speech. Journal of Applied Behavior Analysis, 21,

One child from Laski et al. 28

2-Level Model for Laski et al. 29 Parameter EstimatedEstimateSEp Fixed Effects Average Baseline Level ( θ 00 ) Average Treatment Effect ( θ 10 ) Variance Components Variance in Baseline Level ( ) Variance in Treatment Effects ( ) Covariance u 0 & u 1 ( ) Variance Within Person ( ) <.0001

Software Code: SAS, R 30 SAS: proc mixed covtest; class Case; model Y= D / solution ddfm=sat; random intercept D / sub=Case type=un; R: twolevel <- lmer(Y ~ D + (1 + D | Case), data2) summary(twolevel)

3-Level Model for All Five Studies 31

3-Level Model Results 32 Parameter EstimatedEstimate SEp Fixed Effects Average Baseline Level ( γ 000 ) Average Treatment Effect ( γ 100 ) Variance Components Between Study Variance in Baseline Level ( ) Between Study Variance in Treatment Effects ( ) Covariance v 0 & v 1 ( ) Within Study Variance in Baseline Level ( ) Within Study Variance in Treatment Effects ( ) Covariance u 0 & u 1 ( ) Variance Within Person ( ) <.0001

Software Code: SAS, R 33 SAS: proc mixed covtest; class Study Case; model Y= D / solution ddfm=sat; random intercept D / sub=Study type=un; random intercept D / sub=Case(Study) type=un; R: threelevel <- lmer(Y ~ D + (1 + D | Study:Case) + (1 + D | Study), data3) summary(threelevel)

Statistical models (regression and multilevel) provide a flexible approach for estimating treatment effects from single-case data, but care must be taken to ensure the model being used is consistent with the data being analyzed. Conclusion

Applications and Illustrations Baek, E., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across participant variation in autocorrelation and residual variance. Behavior Research Methods, 45, Baek, E. K., Moeyaert, M., Petit-Bois, M., Beretvas, S. N., Van den Noortgate, W., & Ferron, J. M. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24, Ferron, J. M., Moeyaert, M., Van den Noortgate, W., & Beretvas, S. N. (in press). Estimating casual effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods. Moeyaert, M., Ferron, J., Beretvas, S. N., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52, Moeyaert, M., Ugille, M., Ferron, J., Onghena, P., Heyvaert, M., Beretvas, S. N., & Van den Noortgate, W. (in press). Estimating intervention effects across different types of single-subject experimental designs: Empirical illustration. School Psychology Quarterly. Rindskopf, D., & Ferron, J. (2014). Using multilevel models to analyze single-case design data. In T. R. Kratochwill & J. R. Levin (Eds.), Single-Case Intervention Research: Statistical and Methodological Advances (pp ). American Psychological Association. Shadish, W.R., Kyse, E.N., & Rindskopf, D.M. (2013). Analyzing data from single-case designs using multilevel models: new applications and some agenda items for future research. Psychological Methods, 18, Van den Noortgate, W., & Onghena, P. (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, Van den Noortgate, W., Onghena, P. (2007). The aggregation of single-case results using hierarchical linear models. The Behavior Analyst Today, 8(2), Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental designs. Evidence-Based Communication Assessment and Intervention, 2,