National Center on Response to Intervention OSPI & National Center on Response to Intervention November 1, 2011 Implementer Series Module 3: Multi-level Prevention System

National Center on Response to Intervention Session Agenda  Welcome and Introductions  Review  What Is Multi-level Instruction?  Selecting Evidence-Based Practices  Wrap-Up: Review, Questions, and Resources 2

National Center on Response to Intervention Upon Completion Participants Will Be Able To:  Use screening and progress monitoring data to make decisions at all levels of the multi- level prevention system, including movement between levels.  Develop a multi-level prevention system.  Select evidence-based interventions and practices. 3

National Center on Response to Intervention REVIEW: SCREENING AND PROGRESS MONITORING 4

National Center on Response to Intervention 5  Response to intervention (RTI) integrates assessment and intervention within a schoolwide, multi ‑ level prevention system to maximize student achievement and reduce behavior problems. Defining RTI (NCRTI)

National Center on Response to Intervention 6  With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions based on a student’s responsiveness; and  RTI may be used as part of the determination process for identifying students with specific learning disabilities or other disabilities. Defining RTI (NCRTI)

National Center on Response to Intervention RTI as a Preventive Framework  RTI is a multi-level instructional framework aimed at improving outcomes for ALL students.  RTI is preventive and provides immediate support to students who are at risk for poor learning outcomes.  RTI may be a component of a comprehensive evaluation for students with learning disabilities. 7

National Center on Response to Intervention Essential Components of RTI  Screening  Progress Monitoring  Schoolwide, Multi-level Prevention System Primary Level Secondary Level Tertiary Level  Data-Based Decision Making for Instruction Evaluating Effectiveness Movement within the multi-level system Disability identification (in accordance with state law) 8

National Center on Response to Intervention Essential Components of RTI 9

National Center on Response to Intervention Screening  PURPOSE: Identify students who are at risk for poor learning outcomes  FOCUS: ALL students  TOOLS: Brief assessments that are valid, reliable, and demonstrate diagnostic accuracy for predicting learning or behavioral problems  TIMEFRAME: Administered more than one time per year (e.g., fall, winter, spring) 10

National Center on Response to Intervention Progress Monitoring  PURPOSE: Monitor students’ response to primary, secondary, or tertiary instruction in order to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction  FOCUS: Students identified through screening as at risk for poor learning outcomes  TOOLS: Brief assessments that are valid, reliable, and evidence based  TIMEFRAME: Students are assessed at regular intervals (e.g., weekly, biweekly, or monthly) 11

National Center on Response to Intervention Data-Based Decision Making  Analyze data at all levels of RTI implementation (e.g., state, district, school, grade level) and all levels of prevention (e.g., primary, secondary, or tertiary).  Establish routines and procedures for making decisions.  Set explicit decision rules for assessing student progress (e.g., state and district benchmarks, level and/or rate).  Use data to compare and contrast the adequacy of the core curriculum and the effectiveness of different instructional and behavioral strategies. 12

National Center on Response to Intervention Review Activity  What is the difference between a mastery measure and a general outcome measure?  T or F: All progress monitoring tools are created equal.  Where can I find evidence of the reliability and the validity of progress monitoring tools?  Name three uses of progress monitoring data.  What is a trend line?  What are three ways to establish progress monitoring goals?  Describe two ways to analyze progress monitoring data. 13

National Center on Response to Intervention WHAT IS A MULTI-LEVEL PREVENTION SYSTEM? 14

National Center on Response to Intervention Essential Components of RTI 15

National Center on Response to Intervention Levels, Tiers, and Interventions FRAMEWORK: 3 levels of intensity Primary Secondary (Targeted/Strategic) Tertiary (Intensive) Secondary Level of Prevention (~15% of students) Tertiary Level of Prevention (~ 5 % of students) Primary Level of Prevention (~80% of students) 16

National Center on Response to Intervention Levels, Tiers, and Interventions MODEL: Minimum of 3 tiers representing each level of intensity Tier II (secondary) Tier I (primary) 17 Tier III (tertiary)

National Center on Response to Intervention Levels, Tiers, and Interventions Interventions are provided at each level and within each tier. Secondary Level of Prevention Tertiary Level of Prevention Primary Level of Prevention 18

National Center on Response to Intervention NCRTI recommends different evidence standards across intervention levels. Research-based curricula Recommended for primary prevention across subjects. Components have been researched and found to be generally effective. Curriculum has not been rigorously evaluated. Evidence-based intervention Recommended for secondary and tertiary prevention Evaluated using rigorous research design Evidence of positive effects for students who received the intervention 19

National Center on Response to Intervention PRIMARY PREVENTION LEVEL 20

National Center on Response to Intervention Primary Prevention Level  FOCUS: ALL students  INSTRUCTION: District curriculum and instructional practices that are research based; aligned with state or district standards; and incorporate differentiated instruction  SETTING: General education environment  ASSESSMENTS: Screening, continuous progress monitoring, and outcome measures or summative assessments 21

National Center on Response to Intervention Primary Prevention Focus  ALL students  Includes students with disabilities, learning differences, or language barriers  Increase access through Differentiated instruction. Practices that are linguistically and culturally responsive. Accommodations. Modifications. 22

National Center on Response to Intervention Primary Level Instruction  Research-based curriculum materials for students (including sub groups)  Implementation fidelity  Articulation of teaching and learning within and across grades  Differentiation of instruction based on data  Ongoing professional development See NCRTI Integrity Rubric 23

National Center on Response to Intervention What Is Core Curriculum in RTI?  Grade level standards  Usually mandatory for all students of a school or a school system  Often instituted at the elementary and secondary levels by local school boards, departments of education, or other administrative agencies charged with overseeing education  May or may not include instructional materials 24

National Center on Response to Intervention What Are Differentiated Learning Activities?  Offers students in the same class different teaching and learning strategies based on Student assessment data and knowledge of student readiness Learning preferences, Language and culture 25

National Center on Response to Intervention What Are Differentiated Learning Activities?  Involves Mixed instructional groupings, Team teaching, Peer tutoring, Learning centers, and Accommodations to ensure that all students have access to the instructional program  Is NOT the same as providing more intensive interventions to students with low achievement or learning disabilities 26

National Center on Response to Intervention Primary Prevention Setting  General education environment  Various grouping strategies Whole class Cooperative learning groups 27

National Center on Response to Intervention Primary Prevention Assessment  Universal screening to determine students’ current level of performance  Continuous progress monitoring to confirm risk status and monitor progress of at-risk students  Outcome measures or summative assessments for accountability 28

National Center on Response to Intervention Progress Monitoring and Screening Data Within Primary Prevention  Screening data Identify students who need additional assessment or instruction. Evaluate the effectiveness of primary prevention for all students.  Progress monitoring data Confirm and disconfirm risk. 29

National Center on Response to Intervention Screening: Identify Students Who Need Additional Assessment and Instruction Fall Above average Average Below average Student Benchmark Scores for Grade 2 Screening Measure Score 30

National Center on Response to Intervention Screening: Evaluate Effectiveness of Primary Prevention FallSpringWinter Target score General population. Title I Special education Score 31

National Center on Response to Intervention Progress Monitoring: Confirming Risk Status X X 32

National Center on Response to Intervention XX Progress Monitoring: Confirming Risk Status 33

National Center on Response to Intervention Team Activity: Primary Prevention  Complete Primary prevention (column 2) in Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 34

National Center on Response to Intervention SECONDARY PREVENTION LEVEL 35

National Center on Response to Intervention Secondary Prevention Level  FOCUS: Students identified through screening as at risk for poor learning outcomes  INSTRUCTION: Targeted, supplemental instruction delivered to small groups  SETTING: General education classroom or other general education location within the school  ASSESSMENTS: Progress monitoring, diagnostic 36

National Center on Response to Intervention Secondary Prevention Focus  Students identified through screening as at risk for poor learning outcomes  Typically 15% of entire population 37

National Center on Response to Intervention Secondary Level Instruction  Evidence based  Aligns with and supports core instruction  Implementation fidelity based on developer guidelines.  Delivered by well-trained staff in optimal group sizes  Decisions are based on valid and reliable data and criteria are implemented accurately.  Supplements core instruction See NCRTI Integrity Rubric 38

National Center on Response to Intervention Secondary Prevention Setting  General education classroom or similar setting  Adult-led instruction  Small group rather than whole class 39

National Center on Response to Intervention Secondary Prevention Assessment  Decisions about responsiveness to intervention Are based on reliable and valid progress monitoring data. Reflect judgment based on the slope of improvement or final status at the end of the intervention period.  Decision-making rules are applied accurately and consistently 40

National Center on Response to Intervention Secondary Prevention Assessment  Progress monitoring Monitor student response to secondary instruction. Evaluate the efficacy of the secondary system. Conduct at least monthly.  Diagnostic assessment Match students needs to interventions. 41

National Center on Response to Intervention Secondary Prevention Goal Setting  End-of-year benchmarking  National norms for weekly rate of improvement (slope) 42

National Center on Response to Intervention Progress Monitoring Data Within Secondary Prevention  Progress monitoring data Determine response to secondary interventions using  The four-point rule.  Trend-line analysis. Compare efficacy of secondary interventions. 43

National Center on Response to Intervention Progress Monitoring: Determining Response Using the Four-Point Rule 44 Goal line

National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis X Goal line Trend line 45

National Center on Response to Intervention Growth by Intervention Type Progress Monitoring: Compare Efficacy of Secondary Interventions 46 Words Read Correctly

National Center on Response to Intervention Progress Monitoring: Evaluate Efficacy of Secondary System  Data should indicate the following: 70%-80% of students in secondary interventions demonstrating adequate progress Implementation fidelity for interventions and data- based decision rules 47

National Center on Response to Intervention Team Activity: Secondary Prevention  Complete secondary prevention (column 3) in Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 48

National Center on Response to Intervention TERTIARY PREVENTION LEVEL 49

National Center on Response to Intervention Tertiary Prevention Level  FOCUS: Students who have not responded to primary or secondary level prevention  INSTRUCTION: Intensive, supplemental instruction delivered to small groups or individually  SETTING: General education classroom or other general education location within the school  ASSESSMENTS: Progress monitoring, diagnostic 50

National Center on Response to Intervention Tertiary Prevention Focus  Students who have not responded to primary or secondary level prevention  Typically 3%-5% of the entire population 51

National Center on Response to Intervention Tertiary Level Instruction  Evidence based or based on validated progress monitoring methods for individualizing instruction  More intense than secondary  Implementation fidelity  Delivered by well-trained staff in optimal group sizes  Decisions are based on valid and reliable data, and criteria are implemented accurately.  Address general education curriculum in appropriate manner for students. See NCRTI Integrity Rubric 52

National Center on Response to Intervention Tertiary Prevention Setting  General education setting or other appropriate setting  Decisions are made on a case-by-case basis.  Optimal group size is chosen for ages and needs of students. 53

National Center on Response to Intervention Tertiary Prevention Assessment  Decisions about responsiveness to intervention Are based on reliable and valid progress monitoring data. Reflect judgment based on the slope of improvement or final status at the end of the intervention period.  Decision-making rules are applied accurately. 54

National Center on Response to Intervention Tertiary Prevention Assessment  Progress monitoring Frequent progress monitoring (at least bi-weekly) is recommended. Regularly monitor based on established learning trajectories indicated by the goal line.  Diagnostic Match instruction to needs. Inform individualized instructional planning. 55

National Center on Response to Intervention Tertiary Prevention: Goal Setting Three goal-setting options:  End-of-year benchmarking  National norms for weekly rate of improvement (slope)  Intra-individual 56

National Center on Response to Intervention Progress Monitoring Data Within Tertiary Prevention  Progress monitoring data Determine response to secondary interventions using  The four-point rule.  Trend line analysis.  Trend line analysis and slope. Compare efficacy of tertiary interventions. 57

National Center on Response to Intervention Progress Monitoring: Determining Response Using Four-Point Rule 58 Goal line

National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis 59 XX Goal line Trend line

National Center on Response to Intervention Progress Monitoring: Determining Response Using Trend Line Analysis and Slope 60 Goal line Trend line

National Center on Response to Intervention Progress Monitoring: Evaluate Efficacy of Tertiary System  Data should indicate the following: Majority of students in tertiary prevention are demonstrating adequate progress (e.g., eventually move from tertiary to secondary) Implementation fidelity for interventions and data decision rules 61

National Center on Response to Intervention Team Activity: Tertiary Prevention  Complete tertiary prevention (column 4) Appendix C “Developing an RTI Model”: Tier(s) Focus Instruction Setting Assessment Data-based decision making Other 62

National Center on Response to Intervention Changing the Intensity and Nature of Instruction  Intervention  Duration  Frequency  Interventionist  Group size 63

National Center on Response to Intervention IDEA AND MULTI-LEVEL PREVENTION SYSTEM 64

National Center on Response to Intervention What About Special Education? Two groups to consider:  Students with disabilities who are currently receiving special education  Students being referred for special education eligibility consideration 65

National Center on Response to Intervention What About Students with Disabilities? 66 ~15% ~5% Tier III (tertiary) Specialized individualized systems for students with intensive needs Tier II (secondary) Supplemental group systems for students with at- risk response to primary level Tier I (primary) Schoolwide instruction for all Students, including differentiated instruction ~80%

National Center on Response to Intervention Disability Identification To ensure that underachievement in a child suspected of having a specific learning disability is not due to a lack of appropriate instruction in reading or math, the group must consider, as part of the evaluation, what is described in 34 CFR through :  Data demonstrate that prior to, or as a part of, the referral process, the child was provided appropriate instruction in regular education settings, delivered by qualified personnel.  Data-based documentation of repeated assessments of achievement at reasonable intervals reflect formal assessment of student progress during instruction, which was provided to the child’s parents. 67 ( )

National Center on Response to Intervention 68 When Do We Refer to Special Education? ~15% ~5% Tier III (tertiary) Tier II (secondary) Tier I (primary) ~80% of students Example 1: After nonresponsiveness to two secondary interventions

National Center on Response to Intervention 69 When Do We Refer to Special Education? ~15% ~5% Tier III (tertiary) Tier II (secondary) Tier I (primary) ~80% of Students Example 2: After nonresponsiveness to one secondary and one tertiary intervention

National Center on Response to Intervention Recommendations  Collaborate with special education to develop an inclusive multi-level instruction model  Create written guidance on the following: How do special education services “fit” in the model? What are the roles and responsibilities of special education staff? When should students be referred for eligibility consideration? 70

National Center on Response to Intervention Prereferral Model Versus RTI 71

National Center on Response to Intervention SELECTING EVIDENCE-BASED PRACTICES 72

National Center on Response to Intervention Selecting Evidence Based Practices: Definitions 73  Evidence-based intervention Data demonstrates (or empirically validates) efficacy through scientific, rigorous research designs.  Research-based curricula May incorporate design features that have been researched generally. The curriculum or program as a whole has not been studied using a rigorous research design. See Training Manual Page 9

National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. See Training Manual Page 9 74

National Center on Response to Intervention 1) Identifying Needs and Priorities 1.Gather a team. 2.Conduct a “needs assessment.” Gather information from multiple sources. Compile and summarize data. 3.Determine priorities. See Training Manual Page 9 and Instructional Intervention Tools Chart User’s Guide 75

National Center on Response to Intervention 1) Identifying Needs and Priorities  Evaluate existing data to determine baseline performance on indicators of interest. Academic achievement Discipline referrals Attendance/truancy data 76

National Center on Response to Intervention 1) Identifying Needs: Interventions  For what skills do we need a secondary intervention instructional program? Is there a specific academic outcome or measure we are interested in providing supplemental instruction for?  For what grades do we need an instructional program?  Will this program be used with all students who are not progressing in the core curriculum or only with specific sub groups of students? Tier I? Tier II? Tier III?  Which sub groups? English language learners (ELLs)? Special education? 77 See Instructional Intervention Tools Chart User’s Guide

National Center on Response to Intervention 1) Identifying Priorities: Interventions  Is it a program that can be purchased for a reasonable cost?  Is it a program with a reasonable implementation time?  Is it a program that does not require specialized expertise or lengthy training to administer?  Is it a program that offers ready access to training and technical support for staff?  Is it a program that has documented evidence of efficacy through the most rigorous research?  Is it a program whose effectiveness has been studied and demonstrated in our district or state? 78 See Instructional Intervention Tools Chart User’s Guide

National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 79

National Center on Response to Intervention 2) Selecting Evidence-Based Practices  Screening and progress monitoring  Data-based decision making  Instruction/interventions 80

National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Screening and Progress Monitoring  NCRTI: Screening Tools Chart  NCRTI: Progress Monitoring Tools Chart  IES practice guide: RTI for mathematics and reading  IRIS Center webinars and RTI/assessment training modules 81

National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Data-Based Decision Making  IES Practice Guide: Using Student Achievement Data  U.S. Department of Education: Doing What Works and the American Recovery and Reinvestment Act  IRIS Center Case Study Unit RTI: Data-based Decision Making 82

National Center on Response to Intervention 2) Selecting Evidence-Based Practices: Instruction/Intervention  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tool Chart  Florida Center for Reading Research  IES Practice Guide: RTI for Mathematics and Reading  IRIS Center Learning Strategies Modules 83

National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 84

National Center on Response to Intervention 3) Evaluating Evidence  Where can I find evidence?  What type of evidence exists?  What is the quality of the evidence?  What were the desired outcomes?  What are the effects of the intervention?  Is the sample population similar? 85 See Training Manual Page 10

National Center on Response to Intervention 3a) Evaluating Evidence: Where?  Curriculum websites (use with caution)  Peer-reviewed journals (various) ERIC ( Google Scholar ( Education Abstracts (Education Full Text) Psychological Abstracts See Training Manual Page 10 86

National Center on Response to Intervention 3a) Evaluating Evidence: Where?  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tools Chart  Florida Center for Reading Research 87 See Training Manual Appendix B

National Center on Response to Intervention 3b) Evaluating Evidence: Type?  What type of evidence is available? Research study Summary of existing research Technical report Peer/publisher claims Other methods? See Training Manual Page 11 88

National Center on Response to Intervention 3b) Evaluating Evidence: Type? 89  Randomized control trial*  Quasi-experiment  Single-case design  Quantitative research synthesis * Most rigorous form of research See Training Manual Page 11

National Center on Response to Intervention 3c) Evaluating Evidence: Quality?  How was the program implemented? Is it realistic under normal circumstances? Who implemented the intervention? Was the intervention described? Was there a manual or a script? How often did the intervention occur? Was fidelity evaluated in the study? If so, how? What phase of research? (exploratory, pilot, efficacy, scale-up) See Training Manual Page 11 90

National Center on Response to Intervention 3d) Evaluating Evidence: Desired Outcomes?  Were the outcomes assessed relevant to your outcomes? What outcome measures were used to evaluate the intervention? Do the outcome measures seem reasonable? Are they relevant to your concerns? Are they reliable and valid? Is it feasible to implement with your population? See Training Manual Page 12 91

National Center on Response to Intervention 3e) Evaluating Evidence: Effects?  Are the effects large enough to be meaningful? Significance of differences Interpretation of effect sizes 92 Practitioners can use this information to compare programs and identify those most likely to meet their specific needs. See Training Manual Page 12

National Center on Response to Intervention 3f) Evaluating Evidence: Population?  For which population does the evidence show an effect? Is the sample described? Can you tell who was studied? Is the participant similar to or representative of your student population? Are there different effects for different population groups? See Training Manual Page 12 93

National Center on Response to Intervention Demonstration  What Works Clearinghouse  Best Evidence Encyclopedia  NCRTI Instructional Intervention Tools Chart 94

National Center on Response to Intervention Click here to select a topic area What Works Clearinghouse

National Center on Response to Intervention You can use search terms to search for interventions here You can also click on the Find what works link for a more extensive search feature What Works Clearinghouse 96

National Center on Response to Intervention 97 What Works Clearinghouse What outcome are you focused on? What grade level are you looking for? Is there a specific focus or population?

National Center on Response to Intervention What Works Clearinghouse 98 Click here to create a report What level of effectiveness are you looking for? What is the extent of evidence you are looking for? How will the intervention be delivered? Are you looking for a curriculum, a supplemental intervention, or a practice?

National Center on Response to Intervention 99 Use the filter to select your interest Look at the results provided for the student outcome, the improvement index, the effectiveness rating, and the extent of evidence. Click the link to learn more about the intervention and research

National Center on Response to Intervention Evaluating Evidence 100 Click here to learn more about the interventions

National Center on Response to Intervention Evaluating Evidence 101

National Center on Response to Intervention Evaluating Evidence Accelerated Reader™ (Reviewed for Beginning Reading) 102

National Center on Response to Intervention 103 Evaluating Evidence Accelerated Reader™ (Reviewed for Beginning Reading)

National Center on Response to Intervention Evaluating Evidence 104 Accelerated Reader™ (Reviewed for Beginning Reading)

National Center on Response to Intervention Evaluating Evidence 105

National Center on Response to Intervention Program Information 106

National Center on Response to Intervention Team Activity: What Works Clearinghouse 1.What effective supplemental (Tier II/III), small group, and reading comprehension programs are available for eighth- grade students? 2.How many programs show a positive improvement index for mathematics achievement? 3.Identify a program used in your district/school What is the improvement index, the evidence rating, and the extent of evidence? What additional information is available about the evidence in the program report? 107

National Center on Response to Intervention Best Evidence Encyclopedia Click to select the topic: subject matter grade level focus area (e.g., ELLs, struggling readers)

National Center on Response to Intervention Finding an Evidence-Based Program Find programs that show high or moderate levels of effectiveness Find programs that show limited evidence of effectiveness Find programs with insufficient evidence or no qualifying studies See the key findings, the summary and, the methods

National Center on Response to Intervention Best Evidence Encyclopedia 110

National Center on Response to Intervention 111 Review criteria Studies are rated based on the effect size, the type of study, and the size of study

National Center on Response to Intervention 112 Selecting an Evidence- Based Program

National Center on Response to Intervention Evaluating Evidence 113

National Center on Response to Intervention Determine the Type of Program 114 The type of program is defined as

National Center on Response to Intervention Learn About the Program 115 The program name, a brief description, and contact information for more information are provided.

National Center on Response to Intervention Team Activity: Best Evidence Encyclopedia 1.What programs show strong or moderate levels of effectiveness for eighth-grade reading? 2.Were any small group tutorials (SGT in Column 3) shown to have a strong level of effectiveness for struggling readers? 3.Identify a program used in your district/school: What level of evidence is reported for the program? 116

National Center on Response to Intervention NCRTI Instructional Intervention Tools Chart

National Center on Response to Intervention NCRTI Instructional Interventions Tools Chart 118  The tools chart lists instructional programs that can be used as secondary interventions within an RTI context.  The technical review committee (TRC) does not rate instructional programs; instead the TRC rates studies of program efficacy.

National Center on Response to Intervention Instructional Intervention Tools Chart NCRTI definition of instruction:  Additional or alternative intervention programs to the core curriculum conducted in small groups or individually with evidence of efficacy for improving academic outcomes for students whose performance is unsatisfactory in the core program. 119

National Center on Response to Intervention Instructional Intervention Tools Chart  Purpose: help consumers identify secondary programs that Have been evaluated through rigorous design. Have shown positive, meaningful treatment effects. 120

National Center on Response to Intervention Search by Content: Reading, Math, or Writing Search by Grade: Elementary or Secondary

National Center on Response to Intervention Comparing Tools Narrow search selection to programs that fit your needs

National Center on Response to Intervention 123 Comparing Tools Compare them side by side

National Center on Response to Intervention Instructional Intervention Tools Chart Click on the name of the program to see the implementation requirements.

National Center on Response to Intervention Implementation Requirements

National Center on Response to Intervention Technical Rigor of the Study 126 Study Quality

National Center on Response to Intervention Participants 127

National Center on Response to Intervention Design 128

National Center on Response to Intervention Fidelity of Implementation 129

National Center on Response to Intervention Measures 130

National Center on Response to Intervention Technical Rigor of the Study 131 Effect Size Adjusted posttest: corrects for differences in groups on pretest

National Center on Response to Intervention Technical Rigor of the Study 132 Effect Size Unadjusted posttest: does not account for differences in groups on pretest

National Center on Response to Intervention Effect Size: Sample Developer was unable to provide necessary data for NCRTI to calculate effect sizes.

National Center on Response to Intervention Effect Size: Sample Mean for proximal and distal measures

National Center on Response to Intervention Effect Sizes 135

National Center on Response to Intervention Disaggregated Data  The column reports effect size data that have been disaggregated for sub groups, if available. Students with disabilities ELLs Students from diverse racial-ethnic groups 136

National Center on Response to Intervention Team Activity: Instructional Intervention Tools Chart 1.What secondary instruction programs are available for elementary mathematics students? 2.Which study has the highest average adjusted effect size? 3.What are the average unadjusted effect sizes for SSRD Writing Strategies? 4.Identify a program used in your district/school What level of evidence is reported for the program? 137

National Center on Response to Intervention Team Activity Appendix D: Selecting Evidence-Based Practices  As a table group, choose one of the following examples  Think of intervention materials you have or may considering adopting to support the student  Using the three websites, investigate the evidence for that program (or programs)  We’ll reconvene in 20 minutes to report decisions 138

National Center on Response to Intervention Appendix D, Example 1: Bell Top Elementary School Ms. Jones, a third-grade math teacher, has noticed that Jack, Bobby, and Jane seem to be struggling with word problems despite the strong research-based core curriculum. In looking at their scores on the screening measure, Ms. Jones noticed that they are all below the cut score. After monitoring their progress over a number of weeks she continued to see no progress. Can you identify an intervention that might help Jack, Bobby, and Jane improve? Why did you select that intervention? 139

National Center on Response to Intervention Appendix D, Example 2: Lake Ridge Middle School Mr. Morris has identified that Jessica does not have a strong grasp on the basic skills of reading and is therefore struggling in his class. After looking at her screening data and monitoring her progress, he has decided to implement an intervention to help her with basic reading skills. Is there an intervention available that Mr. Morris might use to help Jessica? What questions might he ask in selecting an appropriate intervention? 140

National Center on Response to Intervention Selecting Evidence-Based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 141

National Center on Response to Intervention 4) Implement Practices  Provide initial recommended training and professional development.  Plan for initial implementation (e.g., scheduling, materials).  Provide ongoing coaching and professional development.  Monitor and evaluate fidelity of implementation. 142

National Center on Response to Intervention What is Fidelity of Implementation? 143

National Center on Response to Intervention 4) Implement Practices: Fidelity The best way to monitor fidelity is to measure it.  Consider adherence to schedule, protocols, and intervention plans  Self-Report Data  Observation  Logs/Lesson Plans Training Manual Page

National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Self-Report Data Questionnaires, surveys, interviews May provide an indicator of teacher knowledge, context of implementation Often unreliable when used alone

National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Evaluating fidelity through observation Develop checklists of critical implementation components Record and listen to sessions at random Spot checks Peer observations Peer coaching

National Center on Response to Intervention Evaluating Implementation: Monitoring Fidelity Logs/lesson plans Student work Allows for evaluation of what was done Content covered Student progress Provides less information about: Delivery Dosage Adherence to scripts (if applicable)

National Center on Response to Intervention 6) Evaluating Implementation Using Fidelity Data Distinguish curriculum/intervention vs. quality of implementation when problems occur Identify implementation strengths (people, potential coaches, parts of intervention) Target areas in need of improvement Coaching Professional Development Retraining

National Center on Response to Intervention Selecting Evidence-based Practices 1.Identify needs and priorities. 2.Select practices to address needs. Resources for identifying evidence-based practices 3.Evaluate evidence claims. Standards of evidence across intervention levels/tiers within RTI 4.Implement practices. 5.Evaluate effectiveness. 149

National Center on Response to Intervention 5) Evaluate Effectiveness  Evaluate general intervention effectiveness.  Evaluate effectiveness for individual students. Ongoing, graphed progress monitoring data Reviewed at least every 4-6 weeks Evidence-based decision-making criteria General outcome measure versus mastery measure 150

National Center on Response to Intervention Growth by Intervention Type Compare Efficacy of Interventions 151 Words Read Correctly

National Center on Response to Intervention Intervention Effectiveness 152 Intervention A Goal line Trend line Intervention B Indicates Change

National Center on Response to Intervention CLOSING 153

National Center on Response to Intervention Things to Remember  Good data IN … Good data OUT Know where your data came from and the validity of that data  Focus on the big picture or ALL students Are most students making progress?  ALL instructional and curriculum decisions should be based on DATA.  Keep it SIMPLE and EFFICIENT! 154

National Center on Response to Intervention Implementing the RTI Framework  Select and implement research and evidence-based practices and procedures.  Implement essential components with integrity.  Ensure cultural, linguistic, and socioeconomic factors students bring to the classroom are reflected in tool selection and implementation. 155

National Center on Response to Intervention Review Activity  Why does the NCRTI use levels instead of tiers?  Where can districts and schools find evidence of an intervention’s effectiveness?  How can schools monitor the fidelity of intervention implementation? 156

National Center on Response to Intervention Next Steps  Develop a district/school multi-level prevention framework model (e.g., guidance document).  Develop an RTI district implementation plan. 157

National Center on Response to Intervention 158 National Center on Response to Intervention RTI Action Network IDEA Partnership Need More Information?

National Center on Response to Intervention Questions? 159

National Center on Response to Intervention 160 This document was produced under U.S. Department of Education, Office of Special Education Programs Grant No. H326E Grace Zamora Durán and Tina Diamond served as the OSEP project officers. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this publication is intended or should be inferred. This product is public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: National Center on Response to Intervention

161 DISCLAIMER The opinions and positions expressed herein are not intended to ensure compliance with any particular law or regulation pertaining to the provision of educational services for eligible students. This presentation and/or materials should be viewed and applied by users according to their specific needs. This presentation and/or materials represent the views of the presenter(s) regarding what constitutes preferred practice based on research available at the time of this publication. The presentation and/or materials should be used as guidance. Any references specific to any particular education product are illustrative, and do not imply endorsement of these products by OSPI, or to the exclusion of other products that are not referenced in the presentation materials. OSPI, Special Education, is not responsible for the content of those educational product(s) referenced in this presentation. Douglas H. Gill, Ed.D., Director, Special Education WA OSPI