1. Institute of Education Sciences Summer Research Training Institute: Single-Case Intervention Design and Analysis June 19-23, The Concourse Hotel Madison, Wisconsin

2. Introductions and General Overview (Tom Kratochwill, Joel Levin, and Rob Horner)
Welcome and Introductions
Goals and Objectives
Schedule and Logistics of the Meeting
Group Activity Sessions and Individual Consultation Options
Resources on Single-Case Design and Analysis

3. Single-Case Design Institute Faculty
Thomas R. Kratochwill, PhD (University of Wisconsin-Madison) Joel R. Levin, PhD (University of Arizona) Robert Horner (University of Oregon) Wendy Machalicek, PhD John Ferron, PhD (University of South Florida) Ann Kaiser, PhD (Vanderbilt University) Carnac the Magnificent Statistician, PhD, PsyD, JD, MD, DDS, and NCM (Psychic University)

4. A Special Tribute to our Colleague and Friend
William R. Shadish ( )

5. Welcome Participants!
Welcome to Wisconsin

6. Assumptions for Participants:
Scholars with some knowledge and expertise in traditional and single-case research methods (some basic and many advanced);
Some with less experience, and maybe some skepticism about utility of single-case research methods;
Commitment to the role of science in society and the importance of intervention research, and evidence-based practices in psychology, education, and related fields.

7. Goals and Objectives of the Single-Case Design Institute
Review Logic and Foundations of Single-Case Designs in Intervention Research;
Review “pilot” Single-Case Design Standards
WWC design standards (Single-Case Designs)
WWC evidence criteria (Visual Analysis);
Provide an overview of approaches to visual and statistical analysis and effect size measures within single-case intervention research;
Review criteria for documenting evidence-based practices using single-case intervention research methods;
Each participant will leave the Institute with a logic model (and more) for a single-case research study (or program of study) that fits their grant/research agenda.

8. Schedule and Logistics of the Meeting
Agenda for the Institute
Breaks, Lunch, and Dinner
Break-out Sessions/Small Group Activities
Individual Project Consultations

9. Group Activity Sessions and Individual Project Consultation Options
Format for Small Group Activities
Small Group Break-Out Rooms:
(Rooms are reserved for the week)
Schedule for Individual Project Consultations

10. Some Goals Group Activity: Personal Research Consultation:
Participants are assigned to a collaborative group to build a logic model for a single-case study, design a study, and apply visual analysis, statistical analysis, and effect size measures.
Personal Research Consultation:
Each participant can meet with a faculty member(s) and develop a logic model (and more) for a single-case research study (or program of study) that fits their grant/research agenda.

11. Resources on Single-Case Design and Analysis
Required Readings
Recommended Readings
Additional Resources
Books, Monographs, and Journals
Faculty as a Resource and Follow Up

12. Follow up to Institute
Thomas R. Kratochwill, PhD Wisconsin Center for Education Research 1025 West Johnson Street University of Wisconsin-Madison Madison, Wisconsin Robert Horner, PhD University of Oregon 1235 University of Oregon Eugene, OR Joel R. Levin, PhD College of Education University of Arizona Tucson, Arizona

13. Institute of Educational Sciences (IES)
Dr. Sarah Brasiel, National Center for Special Education Research

14. Logic and Foundations of Single-Case Intervention Research
Joel R. Levin and Thomas R. Kratochwill University of Wisconsin-Madison

15. Brief Review of the Principles of Scientifically Credible Intervention Research and Transition to Single-Case Intervention Research Joel Levin

16. Logic and Foundations of Single-Case Intervention Research Tom Kratochwill

17. Options for single-case research in development of effective interventions:
Useful in the iterative development of interventions (e.g., replication research).
Documentation of experimental effects that can help understand and define the mechanism for change, not just the occurrence of change.
Analysis of interventions targeting low-incidence populations (e.g., individual as unit of analysis, individuals with disabilities).
Useful for pilot research to assess the effect size needed for other research methods (e.g., RCTs).

18. Options for single-case research in development of effective interventions (Continued):
Useful for fine-grained analysis of “weak and non-responders” (Negative Results; to be discussed later in the Institute).
Useful in RCT/group research when unique participants (e.g., non-responders) are further assessed to determine what modifications in the intervention may need to occur.

19. Purposes and Fundamental Assumptions of Single-Case Intervention Research Methods
Defining features of SCDs
Core design types
Internal validity and the role of replication
“True” Single-Case Applications and the WWC Pilot Standards (design standards and evidence criteria)
Classroom-Based Applications (design and evidence credibility)

20. Features of Single-Case Research Methods
Single-Case Research will have Four Features:
Independent variable
Dependent variable
Focus is on functional relation (causal effect)
Dimension(s) of predicted change (e.g., level, trend, variability, score overlap)

21. Design Examples Reversal/Withdrawal Designs Multiple Baseline Designs
Alternating Treatment Designs
Others:
Changing Criterion
Non-Concurrent Multiple Baseline
Multiple Probe

22. Additional Considerations
Operational definition of dependent variable (DV)
Measure of DV is valid, reliable, and addresses the dimension(s) of concern.
Operational definition of independent variable (IV)
Core features of IV are defined and measured to document treatment integrity (Sanetti & Kratochwill, 2014).
Unit of IV implementation
Group versus individual unit.

23. Background on Single-Case Designs: Defining Features
Design: Repeated measurement of an outcome before, during, and/or after active manipulation of independent variable
Often Used in Applied and Clinical Fields
Allows study of low prevalence disorders where otherwise would need large sample for statistical power (Odom, et al., 2005).
Sometimes more palatable to service providers because SCDs do not include a no-treatment comparison group.
What are researchers doing in literature reviews and studies?

24. Literature Reviews
Wong and colleagues (2015) conducted a review of intervention practices for children with autism. Of 456 studies identified in their review, 89% used SCDs and only 11% used between-groups research designs.
Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S., … Schultz, T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. doi: /s z

25. Literature Reviews
Conroy, Dunlap, Clarke, and Alter (2005) conducted a systematic review of positive behavioral interventions for children with challenging behavior. They identified 62 single-case studies and 1 between-group design.
Conroy, M. A., Dunlap, G., Clarke, S., & Alter, P. J. (2005). A descriptive analysis of positive behavioral intervention research With young children with challenging behavior. Topics in Early Childhood Special Education, 25(3), 157–166. doi: /

26. Descriptive Analysis
Hammond and Gast (2010) reviewed 196 randomly identified journal issues (from ) containing 1,936 articles (a total of 556 single-case designs were coded). Multiple baseline designs were reported more often than withdrawal designs and these were more often reported across individuals and groups.
Hammond, D., & Gast, D. L. (2010). Descriptive analysis of single subject research designs: Education and Training in Autism and Developmental Disabilities, 45(2), 187–202.

27. Literature Reviews that Include Single-Case Design
Kiuhara, Kratochwill, and Pullen (in press) reported over 70 literature reviews (e.g., meta-analysis) of single-case research studies from More recent reviews have adopted various standards for critique of single-case design research.
Kiuhara, S. A., Kratochwill, T. R., & Pullen, P. C. (in press). Single-case design intervention research: Applications in special education. In J. M. Kauffman, D. P. Hallahan, and P. C. Pullen (Eds.). Handbook of special education (2nd ed.). New York: Routledge.

28. Types of Research Questions that Can be Answered with Single-Case Design Types
Evaluate Intervention Effects Relative to Baseline
For example: Does Multi-systemic Therapy reduce the level of problem behavior for students with emotional behavior disorders?
Compare Relative Effectiveness of Interventions
For example: Is “function-based behavior support” more effective than “non-function-base support” at reducing the level and variability of problem behavior for this participant?
Compare Single- and Multi-Component Interventions
For example: Does adding Performance Feedback to Basic Teacher Training improve the fidelity with which instructional skills are used by new teachers in the classroom?

29. Some Further Examples of SCD Research Questions that Might be Addressed
Is a certain teaching procedure functionally related to an increase in the level of social initiations by young children with autism?
Is time delay prompting or least-to-most prompting more effective in increasing the level of self-help skills performed by young children with severe intellectual disabilities?
Is the pacing of reading instruction functionally related to increased level and slope of reading performance (as measured by ORF) for third graders?
Is Adderal (at clinically prescribed dosage) functionally related to increased level of attention performance on the Attention Network Test for elementary age students with Attention Deficit Disorder?

30. Single-Case Designs are Experimental Designs
Like RCTs, purpose is to document causal relationships
Control for major threats to internal validity
Document effects for specific individuals / settings
Replication (across studies) required to enhance external validity
Can be distinguished from case studies

31. Ambiguous Temporal Precedence Selection History Maturation Testing
Single-Case Design Standards were Developed to Address Threats to Internal Validity (when the unit of analysis is the individual)
Ambiguous Temporal Precedence
Selection
History
Maturation
Testing
Instrumentation
Additive and Interactive Effects of Threats 31

32. Additional Threats to Validity May Occur When Clusters are the Unit of Analysis
Examples:
Cluster Selection
Cluster Composition
Within-Cluster Variability
Attrition of Within-Cluster Participants and of Clusters
Within-Cluster Extraneous Variables
Across-Cluster Contagion Effects

33. Distinctions Between Experimental Single-Case Design and Clinical Case Study Research

34. Some Characteristics of Traditional Case Study Research
Often (and traditionally) characterized by narrative description of case, treatment, and outcome variables
Typically lack a formal design with replication but can involve a “basic effect” design format (e.g., A/B)
Methods have been suggested to improve drawing valid inferences from case study research [e.g., Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings(2nd ed.). New York: Oxford University Press]

35. Some Considerations That May Improve Case Study Research
Type of data
Assessment occasions
Planned vs. ex post facto
Projections of performance
Treatment effect size
Treatment effect impact
Number of participants/replication
Standardization of treatment
Integrity of treatment
Randomization

36. Questions and Discussion