IES Project Director’s Meeting June 2010 Rob Horner University of Oregon

Congratulations  Tom Kratochwill ○ Enhancing the scientific credibility of single- case design intervention research: Randomization to the rescue.  Larry Hedges ○ A d-estimator for single-case designs.

Implications  Contribution of single-case research for documenting evidence-based practices in education. ○ Special relevance for low-incidence populations ○ Useful pilot model prior to RCTs ○ Effective strategy for assessing fine-grained effects.

For Single-case Researchers  Adopting a multi-tiered model of analysis. Does the design allow documentation of experimental control? Three demonstrations of effect each at a different time Do the data document a functional relation? Level, trend, variability, immediacy, overlap, consistency What is the size of the effect? Small, medium, large What is the social importance of the effect? Subjective, contextual What are the conceptual contributions of the effect?

Hierarchy of questions Does the design allow assessment of experimental control? Stop no Yes Do data document a functional relation? **Assess Baseline ** Assess Phases ** Assess whole study No Stop Yes What is effect size? What is social importance? What are conceptual implications?

For Single Case Researchers  Improving clarity with which visual analysis rules are defined, taught, used. ○ Reversal designs ○ Multiple baseline designs ○ Alternating treatment designs ○ Changing criterion designs ○ Designs employing multiple design elements Interpreting the whole data set for a study: Not just two phase comparisons

Intervention X

First Demonstration of Effect Second Demonstration of Effect Third Demonstration of Effect Visual Analysis: 1. Change in Level 2. Change in Trend 3. Change in Variability 4. Immediacy of Effect 5. Overlap 6. Consistency of Data Patterns across similar phases Parsonson & Baer, 1978; Kratochwill & Levin, 1992

First Demonstration of Effect Second Demonstration of Effect Third Demonstration of Effect Comparison of actual against projected data (Analysis of Transition States versus Analysis of Steady States)

First Demonstration of Effect Second Demonstration of Effect Third Demonstration of Effect

 Level  Trend  Variability  Overlap  Immediacy of Effect  Consistency across similar phases  Stability in non-intervened series when effect demonstrated in one series Multiple Baseline Design: A 7 th consideration

Assess the magnitude of the intercept gap (Point X, Point Y,) Point Z) X Y Z

Implications for Single Case Researchers  Revise research questions:  from Is there a functional relation between IV and DV? to Is there a functional relation between IV and DIMENSION of DV (level, trend, variability) ○ Implication for the β weights in an HLM analysis

Baseline Intervention

Implications for Single Case Researchers  Number of data points per phase ○ Fewer than 5 data points will create challenges  Randomization ○ Timing of phase introduction ○ Number of data points needed

Implications for the field  Single case research is an effective design method for documenting causal relations.  Single case designs are needed for us to understand what works, where, and how for low-incidence populations.

Implications for the field  Meta-analysis of single-case research is essential for broad adoption of the single case contributions.  Defining multiple design/statistical models for defining interpretable effect size measures (d) will be important for useful meta-analyses.

Multiple Options for Assessing Effect Size  d – estimator  Hierarchical Linear Modeling  Generalized Least Squares  Randomization  Percentage of Non-overlapping Data.

Summary: Using Single Case Designs  Understand the behavior of individuals  Understand change across time  Multi-tiered interpretation ○ Design ○ Visual Analysis ○ Statistical Analysis ○ Social and Conceptual Importance