1. Improving MTSS Implementation and Outcomes through Problem Solving

2. Session Outcomes
Identify areas of concerns and develop a plan of action to improve system level implementation.

3. Intervention Levels/Tiers
Primary (T1)
Secondary (T2)
Intensive (T3)
Instruction/
Intervention Approach
Comprehensive research-based curriculum
Standardized, targeted small-group instruction
Individualized, based on student data
Group Size
Class-wide (with some small group instruction)
3–7 students
No more than 3 students
Monitor Progress
1x per term
At least 1x per month
Weekly
Population Served
All students
At-risk students
Significant and persistent learning needs
Review table. Highlight the differences between the tiers in terms of:
Increasingly focused/tailored approach
Decreasing group size
Increasing frequency of progress monitoring
Increasing student need

4. Critical Features of Tier I Instruction
Section objectives:
Explain the critical features of a high-quality Tier I system
Delivery time: 40 min with activities

5. Critical Features of Tier I Instruction
Uses research-based curriculum materials
Articulation of teaching and learning (in and across grade levels)
Consistent use of differentiated instruction
Instruction aligned with state standards
Inclusion of students with disabilities and those exceeding benchmark
Refer participants to Handout 4.2.
This is an animated slide. Alignment of behavior and academics is not described on the WY MTSS Fidelity of Implementation Rubric, but it is a critical feature of Tier 1 Instruction that will be highlighted in this module.
As we talk about Tier I, we are referring to the alignment of core instruction and expectations for both behavior and academics. Each of these features is described in detail in Handout 4.2, which is an excerpt from the Wyoming MTSS Fidelity of Implementation Rubric.
Briefly describe each component by reviewing the contents of the last column in the handout.
Handout

6. Use Research-Based Curriculum Materials
Across all subjects and school-wide behavior supports
Components have been researched and found to be generally effective
Curriculum materials have not been rigorously evaluated as a package
NCII
Encourage participants to follow along using Handout 4.2. This slide has animation.
Tier I requires research-based curricula across subjects and in the design of the school-wide behavior supports. Research-based curricula have been researched and found to be generally effective, although the curriculum materials have not been rigorously evaluated as a package.
If you’re wondering about resources available for determining the extent of the research-base for curriculum materials available to teachers, there are online resources available, or you can contact the vendor or publisher of the curricula available to you in your school or classroom.
Click for animation. The What Works Clearinghouse is consider the “Gold Standard”. It review of studies conducted on curricula and instructional practices and provides Intervention Reports and Practice Guides,
Click for animation. The Best Evidence Encyclopedia also provide research reviews of commonly used curriculum material,
Click for animation. Some curricula may not have been reviewed by external resources such as these. In these cases, it is recommended to contact the vendor/publisher of curricula to inquire about available research reports.

7. Ensure Articulation of Teaching and Learning (In and Across Grade Levels)
Teaching and learning objectives are well articulated from one grade to another
Teaching and learning are well articulated within grade levels so that students have highly similar experiences, regardless of their assigned teacher
Review slide.
How can you determine the effectiveness of strategies (e.g., curriculum mapping, pacing guides) to ensure articulation of teaching and learning? We will take a look at some screening data together on the next slide.

8. Consistent Use of Differentiated Instruction
Teachers in the school differentiate instruction for students on, below, or above grade level
Most teachers in the school use student data to identify and address the needs of students
Review participants to the final column for differentiated instruction in Handout 4.2.
Review criteria.
How do we know if we are consistently using differentiated instruction? The best way to assess this is to interview and observe staff. Lack of differentiated instruction is often attributed to inconsistent understanding of what is differentiated instruction and how do it look in practice.

9. Aligned with State Standards
The core curriculum (reading and mathematics) is aligned with the Common Core or other state standards.
v/in-the- classroom/wyoming- standards/
Refer participants to the descriptor found in the final column for aligned with state standards in Handout 4.2. Note that there are guidance, resources, lesson plans, student writing samples, etc. on the Common Core State Standards site as well as the Wyoming standards page (screenshot here).
In June 2012, Wyoming adopted the Common Core State Standards as its standards in English/Language Arts and Math. No amendments were made to the original Common Core State Standards.
Wyoming standards:
CCSS:

10. Inclusion of Students Exceeding Benchmark
Schools provide enrichment opportunities for students exceeding benchmarks
Educators implement those opportunities consistently in all classes and grade levels
Refer participants to the final row of Handout 4.2. Review the description in the last column.

11. Inclusion of Struggling Students and Students with Disabilities
Schools provide differentiation of instruction, accommodations, and modifications to ensure struggling students, including those with disabilities, benefit from core instruction
These supports are consistently provided across all classes and grade levels
Struggling students, including students with disabilities, must be provided appropriate supports to ensure they have access to core instruction.
Review slide.

12. Activity: Self-Evaluation of Tier I System
Ask participants to locate Handout 6.1. and review the directions on Handout 6.1.
Clipart: Creative Commons license;
Handout

13. Critical Features of Tier II System
Section objective: Explain the critical features of a braided academic and behavior Tier II system
Key ideas:
Feature 1: Uses evidence-based intervention that support academic and behavior needs.
Feature 2: Complements core academic and behavior instruction/program.
Feature 3: Uses standardized interventions with appropriate dosage and grouping size delivered by trained personnel and implemented with fidelity.
Feature 4: Scheduled in addition to Tier I support
The HLPs introduced in Module 4: Tier I are intensified as the foundations for more targeted instruction and interventions
Delivery time: 20 – 30 minutes (Slides 7 – 11)

14. Four Critical Features of Tier II
Uses evidence-based interventions that support academic and behavior needs
Complements core academic and behavior instruction/program
In previous modules, you explored the WY MTSS implementation rubric. Item 4b describes four critical features of secondary-level, or Tier II. Let’s discuss those features now.
The WY MTSS implementation rubric is excerpted over the next few slides. The full rubric is available for download here:
All Tier II interventions are evidence-based in content areas and grade levels where they are available and they support academic and behavior needs. Tools for identifying evidence-based interventions will be discussed later in the presentation.
Tier II complements core instruction; specifically, it is well aligned with core instruction and incorporates foundational skills that support the learning objectives of core academic instruction and supports behavior programs.
Clipart retrieved from: Creative Commons license

15. Four Critical Features of Tier II
Uses standardized interventions with appropriate dosage and grouping size delivered by trained personnel with fidelity
Standardized intervention
uses research-based instructional programs
provided in a specific manner to students
typically includes a step-by-step sequence
(Vaughn et al., 2012)
Scheduled in addition to Tier I
The third critical feature pertains to the use of instructional characteristics. Specifically, all three of the following conditions are met: (1) interventions are standardized; (2) secondary-level interventions are led by staff trained in the intervention according to developer requirements; and (3) group size and dosage are optimal (according to research) for the age and needs of students.
The fourth critical feature is that secondary-level interventions supplement core instruction is scheduled in addition to Tier I.
Invite participants to discuss how they approach scheduling Tier II interventions in addition to Tier I
Clipart retrieved from: Creative Commons license

16. Activity: Self-Evaluation of Tier II System
Ask participants to locate Handout 6.1. and review the directions on Handout 6.1.
Clipart: Creative Commons license;
Handout

17. Essential Components of Tier III
Section Objective: Explain the critical features of an MTSS Tier III system.
Estimated time: minutes

18. Essential Tier III Criteria: Data-based Individualization
Tier III or intensive intervention
Is more intensive than Tier II interventions.
Is adapted to address individual student needs in a number of ways (e.g., increased duration or frequency, change in interventionist, decreased group size, change in instructional delivery, and change in type of intervention).
Uses an iterative process based on student data.
Read slide.

19. Essential Tier III Criteria: Instructional Characteristics
Tier III or intensive intervention
Is individualized.
Led by well-trained staff experienced in individualizing instruction based on student data.
Uses optimal group size (according to research) for the age and needs of students.
Read slide.

20. Essential Tier III Criteria: Relationship to Core
Decisions regarding student participation in both core instruction and intensive intervention are made on a case-by-case basis, according to student need.
Intensive interventions address the general education curriculum in an appropriate manner for students.
Read slide.
Resources regarding alignment of standards-relevant instruction and multi-tiered systems of support (MTSS):

21. What’s the difference between Tier 2 and Tier 3?
Tier 3 or DBI
This graphic provides a closer look at the DBI process, and notes the places in which secondary interventions fit into the process.
Before a team decides that a student requires intensive intervention in a certain content area, the team should consider whether or not the student has received a secondary intervention program with consistency and fidelity.
The DBI process begins when data show that a student is making insufficient progress in response to a secondary intervention program that is evidence based and delivered with fidelity. The first step in this process is to implement the intervention with greater intensity (e.g., smaller group size, more time), and to progress monitor frequently. If the student continues to be non-responsive, diagnostic data are collected and analyzed to identify the specific skill deficits that need to be targeted. The results of the diagnostic assessment, in combination with the teacher’s analysis of what features of the intervention need to be modified to better support the student, help staff determine how to individualize the secondary intervention program to meet the individual student’s unique needs. Upon implementing the intervention adaptation, the teacher continues to progress monitor at regular intervals to help determine whether additional changes to the individualized intervention are required to support adequate student response.
As you can see, secondary interventions play a significant role in the DBI process. A student’s responsiveness to secondary intervention determines whether or not intensive intervention is needed. The secondary intervention program is also the platform or starting point for intensive intervention, which is intensified and/or adapted to meet a student’s unique needs.
For more information about the DBI process, please review or direct participants to the Introduction to Data-Based Individualization (DBI): Considerations for Implementation in Academics and Behavior module.

22. Activity: Self-Evaluation of Tier III System
Ask participants to locate Handout 7.1 and review the directions For the activity described in Handout 7.1.
Clipart: Creative Commons license;
Handout

23. Data Review Considerations
Are 80% of students benefiting from core instruction?
Are 80% of students responding to Tier 2 instruction?
Are students making adequate growth?

24. Share Out What is the impact of MTSS on student outcomes?
What are your top 2-3 areas of improvement?

25. Understanding Evidence-Base Instruction and Intervention

26. Intervention and Instruction
Evidence-Based Intervention
Fidelity
Adherence
Student Engagement
Program Specificity
Quality of Delivery
Exposure
Tier II interventions should be evidence-based and conducted with fidelity.
Click for animation.
First, let’s take a look at what it means for a secondary intervention to be evidence-based.

27. EBPs: What Are They?
In simple terms, EBPs are practices that are supported by research
But what is considered a practice?
And how do I know if it is supported by research?

28. Challenge: Inconsistent Terminology
Authors and organizations refer to EBPs in a variety of ways:
Evidence-based practice
Research-based practice
Scientifically-based practice
Promising practice
Best practice
Possible poll/comment request: ask people to throw out other terms they have heard

29. Varying Evidence Standards
Research-Based Curricula
Evidence-Based Intervention
Recommended for Tier I across subjects
Components have been researched and found to be generally effective
Curriculum materials have not been rigorously evaluated as a package
Recommended for Tier II and Tier III
Materials evaluated using rigorous research design
Evidence of positive effects for students who received the intervention
There are varying levels of evidence for instruction and interventions delivered within an MTSS framework.
On the left, you’ll see research-based curricula. This is recommended for Tier I, or core instruction, across subjects. This means that the components have been researched and found to be generally effective, although the curriculum materials have not been rigorously evaluated as a package.
On the right side of your screen you’ll see evidence-based intervention. Evidence-based interventions are recommended for Tier II and Tier III. These materials are evaluated using rigorous research design, and there is evidence of positive effects for students who received the intervention. This means that programs have been rigorously evaluated as a whole, and that the specific intervention program was found to have positive effects for students receiving the intervention program.

30. Challenge: Inconsistent Criteria
Authors and organizations define criteria for EBPs in different ways:
Quality of evidence (study design)
Direction and magnitude of results
Quantity of evidence
Source of evidence
Some combination of all these things

31. Every Student Succeeds Act
Requires use of “evidence-based interventions” in low performing schools
Four categories of evidence:
“strong evidence”: supported by at least one randomized study
“moderate evidence”: supported by at least one quasi-experimental study
“promising evidence”: at least one correlational study with pretests as covariates
programs with a rationale based on high-quality research or a positive evaluation that are likely to improve student or other relevant outcomes
Must have a statistically significant positive effect on relevant outcomes

32. Limitations of Gold Standard Evidence…
Time and fiscal resources that delay movement from the research lab into schools may mean few options in certain areas.
Generalizability to other populations
Why isn’t this working?
Misalignment of “practice” with research design
Generally effective isn’t universally effective… Some kids need more.

33. Is MTSS/RTI an EBP?
In MTSS/RTI, the “practice” in “evidence-based practice” is different in each tier!
Wouldn’t this mean that criteria for evidence should differ too?

34. Varying Evidence Standards (Examples)
This graphic illustrates an example of what a school’s reading instruction may look like at each tier. As you’ll see, the evidence standards for the instruction vary by tier. At the primary level, of Tier I, reading instruction consists of research-based curricula. This means that the core curriculums are not evidence-based as a whole, but typically include research-based strategies and practices such as explicit instruction or partner reading.
At the Tier II or secondary level, interventions should be evidence-based interventions, researched as a whole for their effect on student learning outcomes.
Finally, at the Tier III or intensive level, schools should use adapted evidence-based interventions. In other words, Tier III depends on the adaptation of Tier II evidence-based interventions. Adaptations are be based on data to meet the unique needs of that student.

35. What to Look For When Examining the Published Evidence Base
Type/Source Population Desired Outcomes Effects
In order to ensure that a Tier II intervention program is evidence-based, teams should convene to review the existing research on that particular program.
You can focus your efforts by looking at these areas when examining the evidence base:
First, consider the type of information and source from which you are gathering this information. Is the information coming from the intervention vendor or a reputable website? Also ask yourself, what type of evidence is available? Did the study involve experimental design, where the intervention group was compared to an equivalent control group?
Next, consider the population. For which populations has the program been researched and found effective? Is the sample described? Is the population similar to or representative of your student population? Are there different effects for different population groups?
It is important to consider whether or not the desired outcomes assessed in a study are relevant to the outcomes you hope to achieve with an intervention.
Finally, consider the effects. Were the effects of the study large enough to be meaningful? Consider established guidelines for effect sizes when reviewing evidence on effectiveness. Sites including NCII’s Interventions Tools Chart, the What Works Clearinghouse, and the Best Evidence Encyclopedia all offer guidance when interpreting effect sizes.

36. Resources for Evaluating Evidence Base of Published Tier II Interventions
NCII Interventions Tools Chart
What Works Clearinghouse
Best Evidence Encyclopedia
ce.org/
Model how to navigate at least the first two resources. If you are unable to model the site during the presentation, review the content of the resources on the next couple of slides. Participants may also wish to locate Handout 6.2, which outlines the features of the resources modeled on this slide.
After modeling the resources, ask participants to share out at their tables, with a partner, or as a whole-group using prompts such as the following:
Have you accessed any of these resources to identify of evaluate the Tier II interventions used at your site?
If you haven’t accessed any of these resources, identify one that you will access and describe how you might use it.
Handout

37. What If Evidence-Based Interventions Aren’t Available?
Use them when available and consider augmenting current offerings, if feasible.
Also consider:
Remediation materials that came with your core program materials
Expert recommendations (if evidence-based programs are not available)
Standards-aligned materials
In some content areas and grade levels, published evidence-based intervention programs may not be available.
Schools should use evidence-based intervention programs when available, and consider augmenting current offerings if feasible. If evidence-based interventions aren’t available, consider using remediation materials that came with your core program materials, expert recommendations, or standards-aligned materials.

38. What Works Clearing House Scavenger Hunt
Identifying and evaluating evidence base for Tier II interventions
The purpose of this OPTIONAL activity is to provide individuals or teams an opportunity to explore a commonly used resource for identifying and evaluating the evidence base of Tier II interventions.
Website link:
Estimated time: minutes
Grouping: Partners or school teams. This activity is not recommended for individuals to complete during the session.
If not completed during the session, encourage participants to complete the activity or utilize the tools to evaluate current interventions.
A Find What Works Video Tour is available on the WWC website here: The link is included on Handout 6.4.
Clip art: Creative Commons
Handout

39. Elements of Tier II Interventions
Evidence-Based Intervention
Fidelity
Adherence
Student Engagement
Program Specificity
Quality of Delivery
Exposure
Now let’s discuss what it means to deliver Tier II interventions with fidelity.

40. What Is Fidelity?
Degree to which the program is implemented the way intended by program developer.
Fidelity = Consistency and Accuracy
Fidelity = Integrity
Fidelity refers to how closely prescribed procedures are followed, and in the context of schools, the degree to which teachers implement programs the way they were intended by the program developers. It also relates to the quality of the implementation.
This means that teachers are implementing the intervention with consistency and accuracy, and are adhering to the instructional plan with integrity.
Note: Throughout discussions of fidelity it is important to ensure that teachers believe that they work in an open, non-threatening environment that values their skills and expertise and where they can learn from their colleagues. With a system of open communication and productive feedback, fidelity checks of classroom techniques and the essential components of multi-tiered systems of support can be a useful and supportive way for teachers to collaborate and become a stronger teaching network. This may be a useful discussion point for some groups.
(Gersten et al., 2005; Mellard & Johnson, 2007; Sanetti & Kratochwill, 2009)

41. Why Is Fidelity Important?
Ensures that instruction has been implemented as intended Allows us to link student outcomes to instruction Helps in the determination of intervention effectiveness and instructional decision-making Positive student outcomes depend on level of fidelity of intervention implementation
Why is fidelity important? If teachers aren't consistent and accurate in delivering Tier II interventions, they aren’t able to confidently explain a student's lack of response to an intervention. Did the student make insufficient progress because they require more intensive intervention? Or, did the student make insufficient progress because the secondary intervention wasn’t delivered with fidelity? Without practicing consistency and integrity in intervention delivery, we can't link, or attribute, student outcomes to the instruction provided. Fidelity allows us to evaluate the effectiveness of the secondary intervention and tells us when a student may require a more intensive level of intervention.
Furthermore, Pierangelo and Giuliani (2008) concluded that positive student outcomes are particularly dependent on aspects of fidelity within the framework of a tiered support system. One of these aspects is fidelity of implementation at the classroom or teacher level.
(Pierangelo & Giuliani, 2008)

42. Five Elements of Fidelity
Student Engagement: How engaged and involved are the students in this intervention or activity?
Adherence: How well do we stick to the plan, curriculum, or assessment?
Program Specificity: How well is the intervention defined and different from other interventions?
Exposure/Duration: How often does a student receive an intervention? How long does an intervention last?
Refer participants to Handout 6.3. Encourage participants to record examples of what this looks like in the third column of the handout. After presenting the slide content, provide participants 3-5 minutes to talk with their neighbors about other examples of these five elements in action. Have them record their responses.
This graphic provides one example of a way to think about fidelity, and includes the elements of adherence, exposure, quality of delivery, program specificity, and student engagement.
Schools should have procedures in place to monitor the fidelity of their implementation of secondary interventions. While these don’t have to be formal, it is important to consider whether or not they’re implementing programs the way that they are intended to be delivered. In the midst of all of the responsibilities of educators, small checks can make a big difference in keeping services for students on track.
Note: The notes on each element on fidelity are animated to pop up with each click. Click ahead each time you discuss a new element of fidelity, and click again to close that element.
(Click) When we discuss adherence we are focused on how well we stick to the plan/curriculum/assessment, or implementing the plan/curriculum/assessment as it was intended to be implemented based on research. For a secondary intervention, this may mean how well teachers implement all pieces of an intervention, in the way they were intended to be implemented. This doesn’t necessarily mean that teachers should follow a script word for word, but that covering certain content with appropriate pacing and relevant language and techniques are important. (Click)
2. (Click) Duration/Exposure refers to how often a student receives an intervention and how long an intervention lasts. When thinking about fidelity we are considering whether the exposure/duration being used with a student matches the recommendation by the author/publisher of the curriculum.
In the case of secondary interventions, developers and researchers typically specify the required exposure/duration that is needed for the intervention to be effective for most students. If the intervention developer calls for the intervention 3 days a week for 45 minutes each day, is the student receiving this dosage? (Click)
3. (Click) Not only is it important to adhere to the plan/curriculum/assessment, but it is also import to look at the quality of delivery. This refers to how well the intervention, assessment, or instruction is delivered. For example, do you use good teaching practices? Quality instructional delivery also means that teachers are engaged in what they’re teaching, and animated in their delivery, not simply reading from a script. Providing teachers with constructive feedback on their instructional delivery is one way to improve the quality of delivery for secondary interventions. (Click)
4. (Click) Another component is program specificity, or how well the intervention is defined and how different it is from other interventions. Having clearly defined interventions/assessments allows teachers to more easily adhere to the program as defined. Is the intervention a good match for the student’s needs? Or does every low reader get the same intervention? (Click)
5. (Click) Just as quality of delivery is critical, it also is important to also focus on student engagement, or how engaged and involved the students are in the intervention or activity. Following a prescribed program alone is often not enough. Consider whether or not competing behaviors make it difficult for students to take part in the intervention as designed. During the delivery of secondary interventions, teachers may need to use behavior management strategies to manage student behaviors, including providing choice, adding elements of competition, and offering frequent opportunities to respond. (Click)
Quality of Delivery: How well is the intervention, assessment, or instruction delivered? Do you deliver instruction using high leverage practices?
(Dane & Schneider, 1998; Gresham et al., 1993; O’Donnell, 2008)

43. Monitoring Fidelity of Implementation – Examples
Element
Data Source Examples
Adherence
Self-report, observation checklist
Duration/exposure
Self-report, observation
Quality of delivery
Observation, reflection, self-report on techniques used
Program specificity
Intervention component checklist
Student engagement
Student progress, student survey
Refer participants to the last column of Handout 6.3. Encourage participants to record examples of possible data sources presented on the slide. After presenting the slide content, provide participants 3-5 minutes to identify additional examples of what evidence could be used to assess if these elements of fidelity were addressed in the delivery of the Tier II intervention.
Review slide.
Sources: Dane & Schneider, 1998; Mellard & Johnson, 2008; O’Donnell, 2008

44. Discussion
To what extent are we using evidence based practices at all levels of instruction?
What changes do we need to make to improve students access to evidence based practices?

45. Next Steps

46. Communication Plan
Purpose
Audience
Strategies

47. Professional Development Plan
Content
Schedule
Audience
Format