1. Single-Case Effect Size and Meta-Analytic Measures
Part I (today):
Overview
Non-Overlap Measures of Effect Size
Part II (tomorrow):
Parametric Measures of Effect Size
Meta-Analytic Methods

2. Why consider meta-analysis of single-case intervention studies?
We would like to synthesize results across studies to strengthen arguments about:
whether an intervention is evidence based
which interventions are most effective
for which cases interventions tend to be most effective

3. Steps in Meta-Analysis
Identification (search strategies)
Selection (inclusion/exclusion criteria)
Abstraction
extracting effect sizes (or raw data)
extracting case or study characteristics
Analysis
averaging effect sizes
moderation analyses

4. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ

5. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ PND

6. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Precision

7. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision

8. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision How effect sizes analyzed? Dependency Considered

9. Difference between Group Meta-analysis & Typical Single-Case Meta-analysis
Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision How effect sizes analyzed? Dependency Dependency Considered Ignored

10. Percent Non-overlapping Data (PND)
If anticipating an increase, find the highest data point in the A phase, and then find the percent of the B phase data points that exceed it.

11. PND - Concerns 1. Instability Sensitivity to Outliers
Sensitivity to Number of Baseline Observations

12. PND - Concerns
2. Ignores Baseline Trend

13. PND - Concerns
3. Ceiling Effect

14. PND - Concerns 4. No known sampling distribution
Cannot weight effect sizes based on precision
5. Not comparable to group effect sizes
Limits audience

15. Alternative Effect Sizes: Nonparametric
A series of other nonparametric effect sizes developed to overcome noted concerns with PND

16. Percent Exceeding Median(PEM)
If anticipating an increase, find the median of the A phase, and then find the percent of the B phase data points that exceed it.

17. Percent Exceeding Median(PEM)
PND
PEM
Stable - +
2. Account for Trends
3. Sensitive to Size of Effect --
4. Known Sampling Distribution
5. Comparability

18. Extended Celeration Line (ECL or PEM-T)
If anticipating an increase, find the celebration line of the A phase, extend it, and then find the percent of the B phase data points that exceed it.

19. ECL Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect
PND
PEM
ECL
Stable - +
2. Account for Trends +a
3. Sensitive to Size of Effect --
4. Known Sampling Distribution
5. Comparability
aAssuming trend is linear and can be extrapolated

20. NAP
Each baseline observation can be paired with each intervention phase observation to make n pairs (i.e., n = nA*nB). Count the number of Positive (P), Negative (N), and Tied (T) pairs.

21. ECL Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect
PND
PEM
ECL
NAP
Stable - +
2. Account for Trends +a
3. Sensitive to Size of Effect --
4. Known Sampling Distribution
5. Comparability
aAssuming trend is linear and can be extrapolated
bAssuming independece

22. TauU TauU is closely related to NAP If no ties then
TauU is scaled from -1 to 1

23. TauUadj
Each baseline observation can be paired with all later baseline observations (nA*(nA-1)/2). Then compute baseline trend:

25. TauU Stable - + 2. Account for Trends +a +c
PND
PEM
ECL
NAP
TauU
Stable - +
2. Account for Trends +a +c
3. Sensitive to Size of Effect --
4. Known Sampling Distribution +b
5. Comparability
aAssuming trend is linear and can be extrapolated
bAssuming independence
cSome technical reservations

26. References
Ma, H.-H., (2006). An alternative method for quantitative synthesis of single-subject research: Percentage of data points exceeding the median. Behavior Modification, 30,
Parker, R. I., Vannest, K. J., & Davis, J. L. (2014). Non-overlap analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.) Single-case intervention research: Methodological and statistical advances. Washington DC: APA.
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Nine non-overlap techniques for single case research. Behavior Modification, 35,
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S (2011). Combining non-overlap and trend for single-case research: Tau-U. Behavior Therapy, 42, Scruggs, T. E., Mastopieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8,
Scruggs, T. E., Mastopieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8,
Wolery, M., Busick, M., Reichow, B., & Barton, E. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44,