Institute of Education Sciences (IES) Connecting Research, Policy and Practice The Role of Single-Case Research at the Institute of Education Sciences (IES) Sarah Brasiel, Ph.D., Program Officer National Center for Special Education Research June 19, 2017 Madison, WI Sarah.Brasiel@ed.gov ies.ed.gov

Organizational Structure of IES National Board for Education Sciences Standards & Review Office Office of the Director National Center for Education Evaluation National Center for Education Statistics National Center for Education Research National Center for Special Education Research

Missions of the Research Centers NCER supports rigorous research that addresses the nation’s most pressing education needs, from early childhood to adult education NCSER sponsors a rigorous and comprehensive program of special education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school

IES Grant Programs: Research Objectives Develop or identify education interventions (practices, programs, policies, and approaches) that enhance academic achievement that can be widely deployed Identify what does not work and thereby encourage innovation and further research Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness

What kinds of research are we funding in NCSER? FY2018 Funding Opportunities

RFA Research Goals Exploration Development & Innovation (NCER & NCSER) Efficacy & Replication (NCSER only) Effectiveness Measurement 1) DEVELOPMENT GRANTS: Develop an innovative intervention (e.g., curriculum, instructional approach, program, or policy) OR improve existing education interventions AND collect data on its feasibility, usability, and fidelity of implementation in actual education settings AND collect pilot data on student outcomes. Single Case can be used during the iterative development process to get the Intervention “right” as well as in the pilot study required for each development grant to show it has promise. 2) Efficacy Grants: Evaluate whether or not a fully developed intervention is efficacious under limited or ideal conditions OR Replicate an efficacious intervention varying the original conditions Gather follow-up data examining the longer term effects of an intervention with demonstrated efficacy Analyze retrospective (historical) secondary data to test an intervention implemented in the past Single Case design can be used as the primary research design or as a complementary method. to further understand the results of randomized controlled trials in efficacy studies. For example, you could implement a complementary study using single-case designs to determine how manipulation of intervention components may affect outcomes for children who were nonresponsive to the intervention tested in the randomized controlled trial.

2 Early Career Development & Mentoring 8 Post-doc Training 52 NCSER Grants Use SCD 1 Exploration 37 Development 2 Efficacy 2 Measurement 2 Early Career Development & Mentoring 8 Post-doc Training FY17 Awards to be announced soon

Examples of Single-Case Design in NCSER Grants

USE of SCD in Exploration of Malleable Factors Goal 1

Example 2016 Goal 1 Project Gregory Fabiano, State University of New York (SUNY) Comprehensive Meta-analysis of Malleable Factors to Support Youth with ADHD (Topic Area: Social and Behavioral) Researchers are conducting a study to understand which intervention practices are associated with positive outcomes for students with ADHD. Meta-analysis of School-based intervention treatment studies The outcomes in this meta-analysis are the effect sizes for each eligible study. The researchers calculate effect size estimates from single-case design studies

USE of SCD in Intervention Development Goal 2 Development grants – use SCD in the iterative process of development, as well as in the pilot study required to show progress

Example 2016 Goal 2 Projects Thomas Keating, Eugene Research Institute Goal Guide: A Web-Based Application to Improve Goal Self-Management for Students with Autism (Topic Area: Autism) Researchers are developing a web application, Goal Guide, which enables students with mild to moderate Autism Spectrum Disorders (ASD) to effectively set goals in various life domains (e.g., academic and behavioral outcomes in school, personal behavior, life skills). A multiple baseline SCD study is used with 6 high school students with ASD to understand the usability and feasibility of the intervention (Students randomly assigned to the order of starting the program.) Data is analyzed using visual analysis, trend analysis, and effect sizes.

Example 2016 Goal 2 Project Peggy Hester, Old Dominion University A Model of Professional Development that Focuses on the Centrality of Teacher-Child Interactions in the Learning, Behavior, and School Readiness of Preschool Children with Disabilities or At-Risk for Disabilities (Topic: Early Intervention) Researchers are developing and piloting a model of professional development that focuses on the teacher-child interactions of preschool teachers and children with or at risk for disabilities in inclusive classrooms. In Year 3, the intervention will be pilot tested using a single-case multiple baseline design across ABC strategies within each intervention component (i.e., simulations and cyber-coaching) to examine teacher and child outcomes. Single-case design studies will be analyzed using visual analysis and calculation of effect sizes.

Use of SCD IN Evaluating EFFICACY Goal 3

Example 2012 Goal 3 Project Howard Wills, University of Kansas Center for Research A Multi-Site Efficacy Trial of the Class-wide Function-related Intervention Teams "CW-FIT": A Research to Practice Agency for Students With and At Risk for EBD (Topic: Social & Behavioral) Researchers examined the impact of the “CW-FIT” intervention on outcomes measured at the teacher, class, and individual student levels for students diagnosed with or at risk for EBD in general and special education settings. A multi-site randomized trial was used with teachers/classrooms within each school randomly assigned to treatment or comparison groups. For the students who required the individualized program, single-case reversal and multiple baseline designs were used. Data Analysis included visual analysis of the single-case studies

Use of SCD in Measurement Studies Goal 5

Example of 2015 Goal 5 Project Robert Volpe, Northeastern University Development and Validation of a Web-Based System for Monitoring Social Behavior (Topic: Social and Behavioral) Researchers are developing tools for progress monitoring of social behavior, particularly for use in monitoring the impact of Tier 2 and 3 school-based interventions targeting academic enablers (study skills, interpersonal skills, motivation, and academic engagement) and externalizing behavior (disruptive and oppositional behavior) for students in kindergarten through third grade. Treatment sensitivity of the multi-item Direct Behavior Rating (DBR) scales will be investigated through a series of multiple-baseline single-case design intervention studies. Visual analysis and non-overlap effect size techniques will be used to evaluate the multiple-baseline single-case design studies.

Other Funded Projects Using SCD

Search Funded Research Grants

SEARCH FUNDED RESEARCH GRANTS

Funding opportunities FY2018

Identifying IES Funding Opportunities Begin at the IES website (http://ies.ed.gov/funding) Sign up for the IES Newsflash All are announced in The Federal Register Review current Requests for Applications (RFAs) Contact relevant Program Officer(s) for the topic(s) of interest in the relevant Center

https://ies.ed.gov/ Funding Opportunities

Special Education Research Topics (84.324A) - NCSER Autism Spectrum Disorders Cognition and Student Learning in Special Education Early Intervention and Early Learning in Special Education Families of Children with Disabilities Mathematics and Science Education Professional Development for Teachers/Related Service Providers Reading, Writing, and Language Development Social and Behavioral Outcomes to Support Learning Special Education Policy, Finance, and Systems Technology for Special Education Transition Outcomes for Secondary Students with Disabilities

Education Research Topics (84.305A) - NCER Cognition and Student Learning Early Learning Programs and Policies Education Technology Effective Teachers & Effective Teaching English Learners Improving Education Systems: Policies, Organization, Management, and Leadership Mathematics and Science Education Postsecondary and Adult Education Reading and Writing Social and Behavioral Context for Academic Learning

Research Training Programs in Special Education 84.324B The Postdoctoral Training program Funds programs at doctoral-granting institutions to further prepare researchers who have obtained their doctorate to become researchers capable of conducting high-quality, independent special education or early intervention research The Early Career program Provides support for an integrated research and career development plan for investigators in the early stages of their academic careers who have established an interest in special education research, with the ultimate aim of launching independent research careers focused on infants, toddlers, children, and youth with or at risk for disabilities. The SMART Design Training program Provides funding to a program to help current education researchers enhance their methodology and data analysis skills related to using SMART designs as part of rigorous and relevant research on adaptive interventions focused on children with or at risk for disabilities.

Low-Cost, Short-Duration Evaluation Grants The Institute views Low-Cost Evaluation projects as a means to obtain rigorous evidence of impact that state and local agencies can use in making timely decisions regarding the scaling-up or revision of education interventions. Low-Cost, Short-Duration Evaluation of Education Interventions– 84.305L Low-Cost, Short-Duration Evaluation of Special Education Interventions– 84.324L Project carried out by a partnership between a research institution and a state or local agency. Evaluation uses a randomized controlled trial, regression discontinuity design, or a single case experimental design. Evaluation relies on administrative data or other sources of secondary data

Partnerships and Collaborations Focused on Problems of Practice or Policy– 84.305H Researcher Practitioner Partnerships in Education New or Existing Partnership Joint Activities to build or strengthen the partnership to carry out initial research, maintain a longer-term collaboration, and increase capacity to take part in and use research Explore a problem or issue linked to student outcomes Evaluation of State and Local Education Programs & Policies Joint activities to carry out the proposed evaluation and to increase capacity to take part in and use research Causal evaluation of an important agency program or policy intended to improve student outcomes

New Requests for Applications The Research Networks Focused on Critical Problems of Policy and Practice in Special Education: Multi-Tiered Systems of Support National Research and Development Centers The Improving Education Outcomes for Disadvantaged Students in Choice Schools (can include students with or at risk for disabilities) The Improving Rural Education The Writing in Secondary Schools The Exploring Science Teaching in Elementary School Classrooms

When you are ready to discuss your SCD grant ….. Contact the program officer that oversees the topic area of interest to you. The contact information is in the Request for Applications Sarah.Brasiel@ed.gov

Focus on SCD in Request for Applications SCD Efforts at IES What Works Clearinghouse SCD Study Design Standards (http://ies.ed.gov/ncee/wwc/) Focus on SCD in Request for Applications

Conclusion IES is committed to improving the national capacity for conducting high quality studies using SCD. A number of projects addressing important questions are currently being funded using SCD. There is a need for standards for SCD, which are still evolving We need to continue to make progress in the use and reporting of SCD. We expect to see high quality applications from you!