1. Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities Sarah Arden, Laura Berry Kuchle, Christopher Lemons, T. Chris Riley-Tillman This document was produced under U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005. Celia Rosenquist serves as the project officer. The views expressed herein do not necessarily represent the positions or policies of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this document is intended or should be inferred.

2. A note about questions… Please type questions related to technical issues in the Chat box. Please type questions related to webinar content in the Q&A box.

3. Panelists Sarah V. Arden, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research Laura Berry Kuchle, Ph.D. – Researcher and Technical Assistance Liaison at American Institutes for Research Chris Lemons, Ph.D. – Senior Advisor to the National Center on Intensive Intervention and Assistant Professor in the Department of Special Education at Peabody College of Vanderbilt University T. Chris Riley-Tillman, Ph.D. – Trainer for the National Center on Intensive Intervention and Professor at the University of Missouri College of Education

4. Session Overview  What is the vision of RDA  How is intensive intervention part of RDA  Overview of National Center on Systemic Improvement  Introduction to DBI  Why Do We Need to Assess DBI Implementation?  DBI Implementation Rubric  Lessons Learned: Strategies for Improving Technical Assistance and DBI Implementation  NCII Resources to Support Implementation

5. 5 Results-Driven Accountability: Vision All components of an accountability system will be aligned in a manner that best support states in improving results for infants, toddlers, children, and youth with disabilities and their families. Shift from Compliance to Results + Compliance Slide adapted from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/ osep/rda/index.html

6. 6 SSIP Conduct root cause analysis (including infrastructure) to identify contributing factorsConduct root cause analysis (including infrastructure) to identify contributing factors For each contributing factor, identify both barriers and leverage points for improvementFor each contributing factor, identify both barriers and leverage points for improvement Conduct root cause analysis (including infrastructure) to identify contributing factorsConduct root cause analysis (including infrastructure) to identify contributing factors For each contributing factor, identify both barriers and leverage points for improvementFor each contributing factor, identify both barriers and leverage points for improvement Search/evaluate evidence- based solutions (Exploration Phase)Search/evaluate evidence- based solutions (Exploration Phase) Develop action steps (address barriers/use leverage points)Develop action steps (address barriers/use leverage points) Develop Theory of ActionDevelop Theory of Action Develop Plan for Improvement (Implementation Framework)Develop Plan for Improvement (Implementation Framework) Search/evaluate evidence- based solutions (Exploration Phase)Search/evaluate evidence- based solutions (Exploration Phase) Develop action steps (address barriers/use leverage points)Develop action steps (address barriers/use leverage points) Develop Theory of ActionDevelop Theory of Action Develop Plan for Improvement (Implementation Framework)Develop Plan for Improvement (Implementation Framework) Initiate Data AnalysisInitiate Data Analysis Conduct broad Infrastructure AnalysisConduct broad Infrastructure Analysis Identify problem areaIdentify problem area Initiate Data AnalysisInitiate Data Analysis Conduct broad Infrastructure AnalysisConduct broad Infrastructure Analysis Identify problem areaIdentify problem area Evaluation of progress annuallyEvaluation of progress annually Adjust plan as neededAdjust plan as needed Evaluation of progress annuallyEvaluation of progress annually Adjust plan as neededAdjust plan as needed How well is the solution working? What is the problem ? Why is it happening? What shall we do about it? SSIP Phase I SSIP Phase I and II SSIP Phase III SSIP Phase I Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

7. 7 State Systemic Improvement Plan Year 1—FFY 2013 Delivered by April 2015 Year 2—FFY 2014 Delivered by Feb. 2016 Years 3 – 6—FFY 2015 – 18 Feb. 2017 – Feb. 2020 Phase I AnalysisPhase II PlanPhase III Evaluation  Data analysis  Infrastructure analysis  State-identified measureable result  Coherent improvement strategies  Theory of action  Multiyear plan addressing: Infrastructure development Support early intervening services program and local education agencies in implementing evidence- based practices Evaluation plan  Reporting on progress including: Results of ongoing evaluation Extent of progress  Revisions to the State Performance Plan Slide from: OSEP Slides to Explain Results Driven Accountability (RDA) Retrieved from http://www2.ed.gov/about/offices/list/oser s/osep/rda/index.html

8. 8 State-identified Measurable Result(s)  State-identified Measurable Result(s) (SiMR) A child-level (or family-level, for Part C) outcome Not a process or system result May be a single result or a cluster of related results  Identified based on analysis of data

9. 9 On What Are States Focusing? Graduation: AK, DC, FL, GA, MN, MT, NC, ND, NJ, PA, RMI, VA, WV Reading/ELA: AR, AS, AZ, CNMI, CO, CT, DE, FSM, GU, HI, IA, ID, IL, IN, KS, LA, MI, MS, NE, NV, NM, NY, OH, OK, OR, Palau, SC, SD, TN, TX, VI, WA, WI, WY Math: KY, MD, ME, PR, RI, UT, VT Reading and Math: CA, MO Early Childhood Outcomes: MA, NH Post-school Outcomes: AL, BIE Variations: Disability category; race/ethnicity; gender; grades; English learner; poverty status; subset of districts

10. 10 Collaboration between Centers  Led by WestEd  Partners: AIR NASDSE SRI CCSSO Parent Center Network Meadows Center for Preventing Educational Risk (evaluator) Learn more at http://ncsi.wested.org/http://ncsi.wested.org/ Ask NCSI: http://ncsi.wested.org/contact/http://ncsi.wested.org/contact/ NCSI Resources: http://ncsi.wested.org/resources /http://ncsi.wested.org/resources /

11. 11 Targeted Outcomes of NCSI 1.Increase the capacity of state education agencies (SEAs) and lead agencies (LAs) to develop, implement, and evaluate their State Systemic Improvement Plans (SSIPs) 2.Increase SEAs’ and LAs’ knowledge, selection, and utilization of evidence-based practices (EBPs) 3.Improve SEA and LA infrastructure and coordination within SEAs and LAs for delivering effective technical assistance (TA) on implementing and scaling-up effective strategies, stakeholder engagement, resource mapping and allocation, and instructional collaboration

12. 12 Targeted Outcomes of NCSI Cont. 4.Increase the use of effective dissemination strategies by SEAs and LAs to ensure that local education agencies (LEAs) and early intervention service (EIS) programs and providers have access to EBPs and can select and implement those EBPs in a sustainable manner 5.Increase the effectiveness of SEAs and LAs to meaningfully engage State and local stakeholders in the development and implementation of the SSIP

13. 13 NCSI Approach to TA

14. 14 Why Is This Important?  Meeting SiMR goals will require a focus on improving instruction.  States will be in need of support on how to provide intensive intervention for the kids who need it the most, including: Evidence-based intervention strategies Overcoming implementation barriers Making connections to other state, district, and school initiatives

15. 15 What Does This Mean for the Field?  Data Use Knowing what data to look at…accessing those data…root cause analysis to make sense of the data  Knowledge Utilization Selecting coherent/aligned evidence-based practices…implementing them with fidelity…scaling up and sustaining them  Systems Change Infrastructure considerations…resource mapping and alignment…policies that support the goals  Communication & Collaboration Problem solving and planning with diverse stakeholders, in the right settings, at the right time…and working together for implementation to achieve results

16. 16 What is Intensive Intervention and Why Do We Need to Assess Fidelity of Implementation?

17. 17 What is Intensive Intervention? Intensive intervention addresses severe and persistent learning or behavior difficulties. Intensive intervention should be  Driven by data  Characterized by increased intensity (e.g., smaller group, expanded time) and individualization of academic instruction and/or behavioral supports

18. 18 What Intensive Intervention… Is…  Individualized based on student needs  More intense, often with substantively different content AND pedagogy  Comprised of more frequent and precise progress monitoring Is Not…  A single approach  A manual  A preset program  More of the same Tier 1 instruction  More of the same Tier 2 instruction

19. 19 Why Do We Need Intensive Intervention? Low academic achievement Dropout rates Arrest rates

20. 20 What is NCII’s Approach to Intensive Intervention? Data-Based Individualization (DBI): A systematic method for using data to determine when and how to provide more intensive intervention:  Origins in data-based program modification/experimental teaching were first developed at the University of Minnesota (Deno & Mirkin, 1977).  It is a process, not a single intervention program or strategy.  It is not a one-time fix, but an ongoing process comprising intervention and assessment adjusted over time.

21. 21 DBI: Integrating data-based decision- making across academics and social behavior

22. 22 Is DBI the same as MTSS? Is DBI special education?  Many components of DBI are consistent with elements of special education and tiered service delivery systems Tiered Interventions (RTI, MTSS, PBIS) Universal, secondary, and tertiary interventions Progress monitoring Team-based decisions based on data Special Education Individualized program Progress monitoring Team-based decisions based on data

23. 23 Who Needs DBI?  Students with disabilities who are not making adequate progress in their current instructional program  Students who present with very low academic achievement and/or high-intensity or high-frequency behavior problems (typically those with disabilities)  Students in a tiered intervention system who have not responded to secondary intervention programs delivered with fidelity

24. 24 What Is Fidelity?  Extent to which an intervention’s core components have been implemented as planned (Nelson et al., 2012)  For DBI, this includes the instructional platform, adapted iterations of intervention, ongoing progress monitoring, and decision-making procedures.  See IRIS module for additional information: http://iris.peabody.vanderbilt.edu/module/fid/ http://iris.peabody.vanderbilt.edu/module/fid/

25. 25 Levels of Fidelity  Student level For a given student, are assessment and intervention being carried out as planned? Student-level fidelity may be assessed through teacher logs, observations, or record review  Systems level (school or team) Are essential components of DBI being implemented consistently? Are there systems-level problems that hinder DBI implementation? A systems-level rubric or interview might examine team meeting checklists, logs, or other records

26. 26 Why Does Fidelity Matter?  Fidelity assessment provides evidence that DBI is being implemented as intended Assessment, intervention, and decision-making processes Student and systems levels  Provides guidance on how to improve DBI implementation Does the interventionist need additional training or support? Is an intervention adaptation needed? Is there a systems-level problem? (e.g., scheduling prevents sufficient intervention time, staff do not have access to evidence-based instructional platforms)

27. 27 Developing a Rubric to Measure Implementation Fidelity

28. 28 Developing a Systems-Level Rubric  Based on the structure of the Center on Response to Intervention’s RTI Essential Components Integrity Rubric http://www.rti4success.org/resource/essential-components- rti-integrity-rubric-and-worksheet http://www.rti4success.org/resource/essential-components- rti-integrity-rubric-and-worksheet  Identify structures, resources, and practices needed for quality school-level implementation of DBI  Reflect lessons learned from NCII knowledge development sites

29. 29 Measuring School-Level Implementation  NCII’s DBI Implementation Rubric and Interview: http://www.intensiveintervention.org/resource/dbi- implementation-rubric-and-interview http://www.intensiveintervention.org/resource/dbi- implementation-rubric-and-interview  Aligned with the essential components of DBI and the infrastructure that is necessary for successful implementation in Grades K–6

30. 30 Key Findings From Knowledge Development Sites  Purpose: to learn about strategies for implementing intensive intervention from sites that have demonstrated positive outcomes for students with disabilities (SWDs)  Implementing Intensive Intervention: Lessons Learned From the Field (NCII, 2013) http://www.intensiveintervention.org/resource/implementing -intensive-intervention-lessons-learned-field http://www.intensiveintervention.org/resource/implementing -intensive-intervention-lessons-learned-field

31. 31 Lessons Learned 1.Intensive intervention is most likely to be facilitated when implemented as a component of a multi-tiered system of support (MTSS) 2.Family engagement supports implementation 3.Intensive behavioral intervention is often more challenging because of limited progress monitoring 4.Inconsistent decision rules for intensifying supports can hinder intervention planning and resource allocation 5.Hidden inefficiencies may overtax staff unnecessarily

32. 32 Components/Infrastructure Required for DBI Implementation 1.System Features 2.Data and Decision Making 3.Intervention 4.DBI Process 5.DBI Evaluation

33. 33 1. System Features a)Leadership b)School Schedules c)Resources d)Cultural and Linguistic Responsiveness e)Communication With and Involvement of Parents f)Communication With and Involvement of All Staff

34. 34 2. Data and Decision Making a)Process to Identify Students in Need of Intensive Intervention b)Academic Progress-Monitoring Tools c)Behavior Progress-Monitoring Tools d)Progress-Monitoring Procedures e)Diagnostic Assessment f)Fidelity of Assessment Implementation

35. 35 3. Intervention a)Intensive Intervention b)Adaptation of Intervention c)Fidelity of Implementation of Intervention d)Interventionist Characteristics e)Relationship to Core Instruction

36. 36 4. DBI Process a)Team or Appropriate Support Mechanism b)Regular Meetings c)Data Review d)Documentation e)Decision Rules f)Overall Implementation of DBI Process

37. 37 5. DBI Evaluation a)Evaluation

38. 38 Rating Scale 1–5 points with anchors for  1 point = little or no implementation  3 points = partial or inconsistent implementation  5 points = complete and consistent implementation

39. 39 DBI Implementation Interview  Script and note-taking template for gathering information to evaluate rubric  Sample questions for each rubric item

40. 40 Lessons Learned: Initial use in NCII Intensive Sites Strategies for Improving DBI Implementation

41. 41 Highest Rated Items  Trained, experienced interventionists  Teams to support decision making for DBI, with a regular meeting schedule  Aligning intervention to student need and core expectations, addressing prerequisite skills as appropriate  District and school leadership support for DBI  Technically rigorous academic progress-monitoring tools that are sensitive to student improvement

42. 42 Lowest Rated Items Evaluation  Schools may informally review implementation without setting specific goals  Evaluation may be difficult or a low priority if other pieces are not clearly articulated and in place Behavior progress monitoring  Fewer validated tools available compared to academics  New to many schools

43. 43 Lowest Rated Items Continued…. Fidelity  Few schools systematically monitor Decision rules  May not be clearly articulated in one or more areas  Application may not be consistent Overall implementation  May be inconsistent or poorly documented

44. 44 Essential Elements of DBI Implementation  Staff commitment  Student plans  Student meetings  Valid, reliable data  Inclusion of students with disabilities

45. Staff Commitment Key Element Flexibility Within Implementation Commitment of:  Principal  Intervention staff  Special educators  Specific intervention staff involved including staff who work with students with intensive needs in the area(s) of concern. (e.g., reading specialists, social workers)

46. 46 Student Plans Key Element Flexibility Within Implementation Student plans are developed and reflect:  Accurate and timely student data  Goal(s) for the intervention based on valid, reliable assessment tools  Timeline for executing and revisiting the intervention plan  Content area(s)  Number of student plans  Grade level(s)

47. 47 Student Meetings Key Element Flexibility Within Implementation  Student meetings are data driven  There is a regularly scheduled time to meet  Meetings are structured to maximize efficiency and focused problem solving  Frequency  Schedule  Team members

48. 48 Progress Monitoring Key Element Flexibility Within Implementation  Valid, reliable progress monitoring tools are used.  Data are graphed.  Data are collected at regular intervals.  Choice of tool  Use of progress- monitoring data at other tiers

49. 49 Students With Disabilities Key Element Flexibility Within Implementation  Students with disabilities must have access to intensive intervention.  Who delivers intervention for students with disabilities  Inclusion of students with and without IEPs

50. 50 NCII & NCSI Resources to Support Implementation

51. 51 Addressing Common Barriers to Implementation  Systems level considerations—infrastructure and staff commitment Module to assess and support readiness to implement DBI  Identifying intervention and assessment tools for mathematics and behavior Tools charts Sample lessons and activities  Consistent procedures and documentation Student meeting tools

52. 52 Getting Ready to Implement Intensive Intervention: Infrastructure for DBI  Module introducing schools to DBI and infrastructure needed for implementation: http://www.intensiveintervention.org/resource/getting- ready-implement-intensive-intervention-infrastructure-data- based-individualization http://www.intensiveintervention.org/resource/getting- ready-implement-intensive-intervention-infrastructure-data- based-individualization  Handouts and activities to assess readiness and begin action planning

53. 53 Fidelity Resources http://www.intensiveintervention.org/fidelity-resources

54. 54 Fidelity Resources Continued http://www.rti4success.org/resource/essential-components- rti-integrity-rubric-and-worksheet

55. Tools Charts Academic Progress Monitoring http://www.intensiveintervention.org/char t/progress-monitoring http://www.intensiveintervention.org/char t/progress-monitoring Academic Intervention http://www.intensiveintervention.org/char t/instructional-intervention-tools http://www.intensiveintervention.org/char t/instructional-intervention-tools Behavioral Progress Monitoring http://www.intensiveintervention.org/char t/behavioral-progress-monitoring-tools http://www.intensiveintervention.org/char t/behavioral-progress-monitoring-tools Behavioral Intervention http://www.intensiveintervention.org/char t/behavioral-intervention-chart http://www.intensiveintervention.org/char t/behavioral-intervention-chart

56. 56 Mathematics: Sample Lessons & Activities http://www.intensiveintervention.org/resources/sample-lessons- activities/mathematics

57. 57 Implementing Behavioral Strategies: Considerations and Sample Resources http://www.intensiveintervention.org/implementing-behavioral-strategies- considerations-and-sample-resources

58. 58 Individual Student Meeting Tools http://www.intensiveintervention.org/tools-support-intensive-intervention- data-meetings

59. 59 Learn More About DBI  NCII website  Examples of Standards-Aligned Instruction Across Tiers  DBI Training Series  Webinars  Connect to NCII

60. 60 Find Out What NCII Has to Offer www.intensiveintervention.org

61. 61 Examples of Standards-Aligned Instruction Across Tiers http://www.intensiveintervention.org/illustration-standards-relevant-instruction- across-levels-tiered-system

62. 62 DBI Training Series  Eight training modules focusing on components of DBI for academics and behavior  One module focused on readiness and action planning  Include  Slides and speaker notes  Activities  Coaching guides http://www.intensiveintervention.org/content/dbi-training-series

63. 63 Webinars View archived webinars and look for announcements about the next live webinar: www.intensiveinterven tion.org

64. 64 Connect to NCII  Sign up on our website to receive our newsletter and announcements  Follow us on YouTube and Twitter YouTube Channel: National Center on Intensive Intervention National Center on Intensive Intervention Twitter handle: @TheNCII@TheNCII

65. 65 Connect to NCSI  This website provides immediate, easy access to foundational NCSI information, previously developed technical assistance resources, a snapshot of the NCSI team, and contact information.  www.ncsi.wested.org/contact/ www.ncsi.wested.org/contact/

66. 66 Ask the NCSI  Ask the NCSI is intended to support states to (1) obtain information about evidence-based practices;  (2) develop, implement and evaluate State Systemic Improvement Plans (SSIPs);  (3) learn about practices being implemented in other states; and  (4) find out what current research says about “what works” to improve results for children with disabilities.  www.ncsi.wested.org/contact/ www.ncsi.wested.org/contact/

67. 67 Questions

68. 68 References National Center on Intensive Intervention. (2013). Implementing intensive intervention: Lessons learned from the field. Washington, DC: U.S. Department of Education, Office of Special Education Programs. http://www.intensiveintervention.org/sites/default/files/Lessons_Learned_Fro m_Field_0.pdf http://www.intensiveintervention.org/sites/default/files/Lessons_Learned_Fro m_Field_0.pdf Nelson, M. C., Cordray, D. S., Hulleman, C. S., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374–396.

69. 69 NCII & NCSI Disclaimers  This presentation was produced under the U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005 for which Celia Rosenquist serves as the project officer and No. H326R140006 for which Perry Williams and Shedeh Hajghassemali serve as project officers. The views expressed herein do not necessarily represent the positions or polices of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this website is intended or should be inferred.

70. Although permission to redistribute this webinar is not necessary, the citation should be: National Center on Intensive Intervention and National Center on Systemic Improvement (2015). Results Driven Accountability and Intensive Intervention: Using MTSS to Improve Outcomes for Students with Disabilities. Washington, DC: U.S. Department of Education, Office of Special Education Programs, National Center on Intensive Intervention.

71. National Center on Intensive Intervention 1000 Thomas Jefferson Street NW Washington, DC 20007-3835 www.intensiveintervention.org ncii@air.org @TheNCII www.intensiveintervention.org ncii@air.org National Center on Systemic Improvement 730 Harrison St San Francisco, CA 94107 http://ncsi.wested.org/ NCSI@wested.org http://ncsi.wested.org/ NCSI@wested.org