Response to Intervention: Using Data to Enhance Outcomes for all Students Amanda VanDerHeyden Education Research and Consulting, Inc.

16 x 3 = 48 hours

Data allow us to Provide faster, more effective services for ALL children Work “smarter” not harder, better utilize the talents of the school psychologist and school- based assessment and intervention teams. Make implementation SIMPLE and EASY for teachers (low cost, few errors) Prevent diagnosis

What is RTI? A science of decision making and way of thinking about how educational resources can be allocated (or reallocated) to best help all children learn Major premium on child outcomes

STEEP Model Screening to Enhance Educational Progress

Tier 1: Screening Screening –Math Screening 2 minutes. Scored for Digits Correct –Writing Screening 3 Minutes. Scored for Words Written Correctly –Reading Screening 1 Minute. Scored for Words Read Correctly

Class-wide Screening

Feedback to Teachers

Tier 2: Class-wide Intervention

No Class-wide Problem Detected

Tier 2: Can’t Do/Won’t Do Assessment “Can’t Do/Won’t Do” Individually-administered Materials –Academic material that student performed poorly during class assessment. –Treasure chest: plastic box filled with tangible items. 3-7 minutes per child

Can’t Do/Won’t Do Assessment

Decision Rule Following Can’t Do/Won’t Do Assessment

Tier 3: Individual Intervention

Response to Intervention Before Intervention During Intervention Avg. for his Class Intervention in Reading #Correct Intervention Sessions Each Dot is one Day of Intervention

Before Intervention During Intervention #Correct Avg. for his Class Response to Intervention

Instructional range Frustrational range Vehicle for System Change: System-wide Math Problem Each bar is a student’s performance

Re-screening Indicates No Systemic Problem Fourth Grade

Effect on SAT-9 Performance VanDerHeyden & Burns, 2005

Effect on CBM Scores VanDerHeyden & Burns, 2005

Computation Gains Generalized to High Stakes Test Improvements (Gains within Multiple Baseline shown as pre-post data)

Gains within Multiple Baseline (shown as pre-post data)

District-wide Implementation Data Vail Unified School District – Three years, system-wide implementation of STEEP grades 1-8

System Outcomes Referrals reduced greater than half % who qualify improved at 4 of 5 schools SLD down from 6% of children in district in (with baseline upward trend) to 3.5% in school year Corresponding gains on high-stakes tests (VanDerHeyden & Burns, 2005) Intervention successful for about 95 to 98% of children screened VanDerHeyden, Witt, & Gilbertson, 2007

Cost Reduction VanDerHeyden, Witt, & Gilbertson, 2007

Findings Diverse settings, psychologists of diverse backgrounds and no prior experience with CBM or functional academic assessment Disproportionate representation of males positively affected VanDerHeyden, Witt, & Gilbertson, 2007

Team Decision-Making Agreement RTI + and Evaluated RTI- and Did Not Evaluate (3 schools) 100%41% (5 schools) 100%87% VanDerHeyden, Witt, & Gilbertson, 2007

Team Decision-Making VanDerHeyden, Witt, & Gilbertson, 2007

Identification Accuracy CBA + RTI CriterionITBSWJ-R STEEP Sensitivity Specificity Positive Predictive Power Negative Predictive Power Teacher Referral Sensitivity Specificity Positive Predictive Power Negative Predictive Power VanDerHeyden, et al., 2003

Percent Identified at each Tier Identified CBM (Classwide Assessment) 55 (15%) CBM + Reward (Performance/skill Deficit Assessment) 40 (11%) CBM + Reward + Instruction (STEEP +) 22 (6%) Teacher Referral 32 (19%) CIBS-R 64 (18%) DRA 17 (9%) RTI Criterion Assessment 17 (5%) WJ-R 12 ITBS deficit 3 (4%) VanDerHeyden, et al., 2003

What Proportion of Ethnicity Represented Before and After Intervention in Risk Category? VanDerHeyden & Witt, 2005

Screening tells you How is the core instruction working? What problems might exist that could be addressed? Most bang-for-the-buck activity Next most high-yield activity is classwide intervention at Tier 2.

Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Any Curriculum Area Students Dave Tilly, 2005

“Weighing a cow doesn’t make it fatter.”

Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Any Curriculum Area Students Dave Tilly, 2005

Intervention Plan- 15 Min per Day Protocol-based classwide peer tutoring, randomized integrity checks by direct observation Model, Guide Practice, Independent Timed Practice with delayed error correction Group performance contingency Teachers encouraged to –Scan papers for high error rates –Do 5-min re-teach for those with high-error rates –Provide applied practice using mastery-level computational skill

Math Sample Sequence

Class-wide Math Intervention

Class 1 at Screening

Class 1: Following 10 Days Intervention

Class 1: Following 15 Days Intervention

Class 2 at Screening

Class 2: Following 5 Days Intervention

Class 2: Following 10 days Intervention

Class 3 at Screening

Class 3: Following 5 days Intervention

Following 10 Days Intervention

Academic SystemsBehavioral Systems 1-5% 5-10% 80-90% Intensive, Individual Interventions Individual Students Assessment-based High Intensity Of longer duration Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive Universal Interventions All settings, all students Preventive, proactive Any Curriculum Area Students Dave Tilly, 2005

Tier 3 Assessment Data –Instructional level performance –Error analysis (high errors, low errors, pattern) –Effect of incentives, practice, easier task –Verify intervention effect Same implementation support as Tier 2 Instructional-level materials; Criterion-level materials

Tier 3 Implement for 5-15 consecutive sessions with 100% integrity Link to referral decision Weekly graphs to teacher and weekly generalization probes outside of classroom, supply new materials Troubleshoot implementation weekly

Tier 3 Intervention >5% of children screened (total population) IF solid Tier 1 Possibly as low as 2% IF solid Tier 1 and Tier 2 About 1-2% failed RTI; 10% of most at-risk VanDerHeyden et al., 2007

Principal FILTER-- How much time allocated to instruction? Children actively engaged? Standards introduced? Effective instruction occurring? DATA on Learning Goal Setting Teacher Evaluation Allocation of Instructional Resources Upset parent Check on health dept Check on police interview Etc.

Great Implementers Follow the aimline and attend to implementation integrity Understand the variables of effective instruction and engage in contextualized assessment that is technically valid for the purposes needed AND has treatment utility Minimize meeting time and avoid “the science of strange behavior…” Provide adequate resources and space for principals to be effective instructional leaders and hold them accountable for results Evaluate quality of all programs locally and make decisions about continued use based on DATA.

For More Information Thank you to the US Dept of Education for providing all film clips shown in this presentation