Single-Case Effect Size and Meta-Analytic Measures Part I (today): Overview Non-Overlap Measures of Effect Size Part II (tomorrow): Parametric Measures of Effect Size Meta-Analytic Methods

Why consider meta-analysis of single-case intervention studies? We would like to synthesize results across studies to strengthen arguments about: whether an intervention is evidence based which interventions are most effective for which cases interventions tend to be most effective

Steps in Meta-Analysis Identification (search strategies) Selection (inclusion/exclusion criteria) Abstraction extracting effect sizes (or raw data) extracting case or study characteristics Analysis averaging effect sizes moderation analyses

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ PND

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Precision

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision How effect sizes analyzed? Dependency Considered

Difference between Group Meta-analysis & Typical Single-Case Meta-analysis Group Single-Case What effect size is used? δ PND How effect sizes averaged? Weighted by Not Weighted Precision How effect sizes analyzed? Dependency Dependency Considered Ignored

Percent Non-overlapping Data (PND) If anticipating an increase, find the highest data point in the A phase, and then find the percent of the B phase data points that exceed it.

PND - Concerns 1. Instability Sensitivity to Outliers Sensitivity to Number of Baseline Observations

PND - Concerns 2. Ignores Baseline Trend

PND - Concerns 3. Ceiling Effect

PND - Concerns 4. No known sampling distribution Cannot weight effect sizes based on precision 5. Not comparable to group effect sizes Limits audience

Alternative Effect Sizes: Nonparametric A series of other nonparametric effect sizes developed to overcome noted concerns with PND

Percent Exceeding Median(PEM) If anticipating an increase, find the median of the A phase, and then find the percent of the B phase data points that exceed it.

Percent Exceeding Median(PEM) PND PEM Stable - + 2. Account for Trends 3. Sensitive to Size of Effect -- 4. Known Sampling Distribution 5. Comparability

Extended Celeration Line (ECL or PEM-T) If anticipating an increase, find the celebration line of the A phase, extend it, and then find the percent of the B phase data points that exceed it.

ECL Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect PND PEM ECL Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect -- 4. Known Sampling Distribution 5. Comparability aAssuming trend is linear and can be extrapolated

NAP Each baseline observation can be paired with each intervention phase observation to make n pairs (i.e., n = nA*nB). Count the number of Positive (P), Negative (N), and Tied (T) pairs.

ECL Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect PND PEM ECL NAP Stable - + 2. Account for Trends +a 3. Sensitive to Size of Effect -- 4. Known Sampling Distribution 5. Comparability aAssuming trend is linear and can be extrapolated bAssuming independece

TauU TauU is closely related to NAP If no ties then TauU is scaled from -1 to 1

TauUadj Each baseline observation can be paired with all later baseline observations (nA*(nA-1)/2). Then compute baseline trend:

http://www.singlecaseresearch.org/calculators/tau-u

TauU Stable - + 2. Account for Trends +a +c PND PEM ECL NAP TauU Stable - + 2. Account for Trends +a +c 3. Sensitive to Size of Effect -- 4. Known Sampling Distribution +b 5. Comparability aAssuming trend is linear and can be extrapolated bAssuming independence cSome technical reservations

References Ma, H.-H., (2006). An alternative method for quantitative synthesis of single-subject research: Percentage of data points exceeding the median. Behavior Modification, 30, 598-617. Parker, R. I., Vannest, K. J., & Davis, J. L. (2014). Non-overlap analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.) Single-case intervention research: Methodological and statistical advances. Washington DC: APA. Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Nine non-overlap techniques for single case research. Behavior Modification, 35, 303-322. Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S (2011). Combining non-overlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299. Scruggs, T. E., Mastopieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8, 24-33. Scruggs, T. E., Mastopieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8, 24-33. Wolery, M., Busick, M., Reichow, B., & Barton, E. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44, 18-28.