“Need to Knows” for Leading High Quality Reading MTSS Implementation Kim St. Martin October 22, 2015

Agenda 1.0 Unpacking “Leadership:” Its Effects & the Who, What, and How 2.0 Reading Tiered Fidelity Inventory (R- TFI) Overview 3.0 Using the R-TFI to Shape Leadership Practices and Distribute Leadership

1.0 Unpacking “Leadership:” Its Effect on Student Outcomes & the “Who, What and How”

John Hattie’s work has changed the conversation about what works. John Hattie, author of Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement

“We need a barometer of what works best, and such a barometer can also establish guidelines as to what is excellent…excellence is attainable: there are many instances of excellence, some of it fleeting, some of it aplenty…” - John Hattie

Why a Barometer? Idea was try to take almost everything that has happened in education that relates to student achievement and put it along a continuum to answer the following questions: What are the things that enhance student achievement? What are the things that had some effect on student achievement? What are the thing that have a negative effect on student achievement?

Domains The 138 innovations were categorized into domains: Child Curricula Home Teaching Teacher School

Most of the domains are within the realm of our control. Some are not Most of the domains are within the realm of our control. Some are not. The innovations that fall in the home and student domains are not really in our realm of control.

The Effect Size Hattie took data from thousands of studies that focused on student achievement and converted that data into a common metric (effect size) so they can be put along a scale (barometer) An effect size is a measure of strength (of a program, practice, intervention, phenomena, etc.)

Overall Findings (2009) The average effect size for all innovations attempted = 0.4 Coincidentally looks like the bell shaped curve (half of the things teachers are doing are above .4 and half that are below 0.4) Almost everything “works” (or has above a zero effect size) 90% of all effect sizes in education are positive The 10% that are negative, half are “expected” (e.g. effects of disruptive students) Consequently, 95% of all the things we do in education have positive effects (influences) in achievement

Implications “When teachers claim they are having a positive effect on achievement or when a policy improves achievement this is almost a trivial claim: virtually everything works…” (Hattie, 2009, p. 16) Setting the bar at zero is a very low bar Any innovation that has an effect size greater than .15 – but less than .4 is in need of more consideration Any innovation less than .15 is disastrous

Where do principals / school leaders fall along the barometer?

Unpacking Principal Leadership (ES = .36) Moderators: Instructional leadership (ES: .42 - .77) Transformational leadership (ES: .09-.11) Some key distinctions between the two: Relational aspects are emphasized over evaluation of impact of the practices on student outcomes Leaders may feel uncomfortable rocking the boat Leaders place being liked by their staff ahead of challenging the status quo Robinson, Lloyd, Rowe (2008)

Defining “Instructional” Leadership Establishing goals and priorities (ES = .42) Ensuring high quality teaching (ES = .42) Leading teacher learning (ES = .84) Strategic resourcing (ES = .31) Ensuring a safe and orderly environment (ES = .27) Robinson, 2011

Leadership: Literature Review

Activity Read the Leadership White Paper. Use at least one of the following sentence frames to facilitate your discussion of the Leaders White Paper: “After leading previous EI implementations, I focused more on the _____ from the list of 8 critical features.” “I think an increased emphasis on ______ is a good strategy for me to use. Share with your table mates! 16

2.0 Reading Tiered Fidelity Inventory (R-TFI) Elementary and Secondary Editions

R-TFI Rationale Only widely known reading fidelity assessment is the Planning Evaluation Tool-Revised (PET-R) that was revised in 2003 The PET-R is designed to address elements of an elementary reading model Focus is primarily on Tier 1

R-TFI Rationale (cont.) Participating MiBLSi schools were reporting high PET-R data (at or above 80%) yet, their student outcome data (DIBELS) was not improving as expected

R-TFI Rationale (cont.) There is no known reading fidelity tool that addresses: Secondary reading Tiers 2 & 3 The R-FTI (elementary and secondary editions) also includes a scoring guide (rubric) to assist School Leadership Teams in their efforts to accurately self-assess current status and provide suggestions for how to improve The R-TFI editions also integrate some behavior elements into the items

R-TFI (Elementary Edition)

R-TFI (Elementary Edition) Constructs Construct: School-Wide Reading Model Definition: Multi-tiered structures encompassing: Evidence-based practices focused on the Big Ideas of Reading designed to improve reading outcomes for all students Systems to address the continuum of needs across the student body Data use and analysis

R-TFI (Elementary Edition) Domains Tier 1: system that ensures all students have access to and benefit from the core reading curriculum which includes: High quality, evidence-based instruction that is differentiated to address the continuum of reading needs across all students Universal screening on a periodic basis to measure the impact of the core reading curriculum and instruction; and to assess student reading performance

R-TFI (Elementary Edition) Domains (cont.) Tier 2: System that ensures students who are not making adequate progress in the core reading curriculum are provided with evidence-based, supplemental instruction matched to their needs on the basis of levels of performance and rates of progress

R-TFI (Elementary Edition) Domains (cont.) Tier 3: System that ensures students who have the most intensive needs in reading have access to and benefit from individualized, intensive interventions that that targets students’ skill deficits for the remediation of existing problems and the prevention of more severe problems

R-TFI (Elementary Edition) Subscales Subscales for each Tier (Tiers 1-3) Teams Implementation Resources Evaluation

R-TFI (Elementary Edition) Structure Tier 1: 27 items Tier 2: 14 items Tier 3: 12 items

R-TFI (Elementary Edition) Scoring Guide

R-TFI (Elementary Edition) Scoring Guide: Behavior Elements

R-TFI (Elementary Edition) Scoring Guide: Behavior Elements (cont.)

R-TFI (Elementary Edition) Scoring Guide: Behavior Elements (cont.)

R-TFI (Elementary Edition) Scoring Guide: Behavior Elements (cont.)

R-TFI (Secondary Edition)

R-TFI (Secondary Edition) Construct Construct: School-Wide Content Area Reading Model Definition: Multi-tiered model across a variety of content areas encompassing: Practices designed to improve reading outcomes for all students that involve active participation by all school staff Systems to address the continuum of reading needs across the student body Data use and analysis

R-TFI (Secondary Edition) Domains Tier 1: System that ensures all students have access to and benefit from content area reading strategies implemented across all core content areas, including: High quality, evidence-based instruction that is differentiated to address the continuum of reading needs across all students Universal screening at least 3 times per year to measure the impact of the content area reading strategies.

R-TFI Secondary Edition Domains (cont.) Tiers 2 & 3: System that ensures students with reading deficits who are not making adequate progress in core subject areas have access to and benefit from interventions that target students’ skill deficits for the remediation of existing problems and the prevention of more severe problems

R-TFI (Secondary Edition) Subscales Subscales for Tier 1 and Tiers 2 &3 Teams Implementation Resources Evaluation

R-TFI (Secondary Edition) Structure Tier 1: 25 items Tiers 2 & 3: 19 items

There is overlap between the R-TFI elementary and secondary editions in the items that include behavior elements.

R-TFI Administration Recommendations All members of the School Leadership team actively participation in the completion of the R-TFI Options for first administration: School Leadership Team can choose to only assess Tier 1 or all three tiers It is not recommended a team skip Tier 1 and assess Tiers 2 & 3 Team can also choose to complete the entire R-TFI (all three tiers) in order to establish baseline data

R-TFI Administration Recommendations (cont.) After first administration, the R-TFI should be completed at least once per year (typically in the spring) The outcome of administering the R-TFI is action planning for continuous improvement Anticipated timeframe is approximately 1-2 hours

Typical R-TFI Administration Process Facilitator reads item Facilitator reads 2 point column and summarizes big ideas in the 2 point column (how the 1and 0 are different from the 2 point column Respondents are asked to read scoring rubric (2-0) Facilitator asks if any clarification is needed and provides local contextualization Facilitator calls for the simultaneous vote Respondents vote using hand signals If there is not consensus, facilitator: Invites participants to share rationales & facilitates discussion Calls for a revote till consensus is achieved After introductions to the purpose and process of the R-TFI, for each item: Consensus Defined: Can the minority live the majority vote score and publicly support it?

R-TFI Administration Recommendations (cont.)

R-TFI Next Steps Begin collecting R-TFI data with elementary and secondary schools R-TFI is a part of our MiBLSi project data collection requirements Continue our work to validate the measures Collaborate with University of Oregon to place the R-TFI (elementary and secondary editions) on the field test site of PBIS Assessments so other schools across the country can access and if willing, participate in the validation process

Activity Access the R-TFI Scoring Form (elementary or secondary based on what level you lead / support leadership). Flip to the 8 critical features of leadership in the Leadership White Paper. Categorize the Tier 1 items (as best you can) based on those that, address the “Who” critical features and the critical features for the “What” of leadership. Share with your table mates! 47

3.0 Using the R-TFI to Shape the “What” and “Who” of Leadership

Pathway for the Reading Components of an Elementary MTSS Model

MiBLSi Elementary Scope and Sequence DIBELS Next Data Interpretation (1 Day) Tier 1 Elementary Reading Systems (1 Day) Tier 1 Elementary Grade Level Problem Solving 2 Days combined for Reading & Behavior Data Review Fall, winter, spring combined for reading & behavior Tier 2/3 Managing Reading Interventions (2 Days) Tier 1 Elementary Strengthening Reading Implementation Optional: 2 Days for Kindergarten/1st grade & 2 Days for 2nd/3rd grade Trainer Notes: Listing of reading series trainings along with the number of days required for each. We still need to know who is expected to attend the trainings (Team vs All Staff). – SHOULD WE JUST ADD THIS TO THE TRAINER NOTES??? This is Terri – is this the right order for now??

Proposed Pathway for the Content Reading Components of a Secondary MTSS Model

MiBLSi Secondary Scope and Sequence Tier 1 Secondary Content Area Reading (2 days for School Leadership Team Tier 1 Secondary Content Area Reading Strategies (1 day for content area teacher representation) Data Review Fall, winter, spring combined for reading & behavior – Early Warning Indicators are a part of the data review process Secondary Data-Based Problem Solving Teams (2 Days) Tier 2/3 Managing Reading Interventions (2 Days) Trainer Notes: Listing of reading series trainings along with the number of days required for each. We still need to know who is expected to attend the trainings (Team vs All Staff). – SHOULD WE JUST ADD THIS TO THE TRAINER NOTES??? This is Terri – is this the right order for now??

Promoting Adolescent Reading Success Model Demonstration Project Awarded the grant by the U.S. Department of Education in August, 2015

Michigan Context 800+ districts 54 Intermediate School Districts (ISDs) who support local districts within a region MiBLSi is a state-supported MTSS model with implementation emphasis at the school and district levels (K-12) MiBLSi focuses on developing capacity at the ISD and district levels to implement an integrated behavior and reading MTSS model

Michigan Context (cont.) MiBLSi has more experience with elementary implementation sites (2003-present). Secondary support began in 2006-2007 with Middle Schools High School support began in 2011-2012 starting with PBIS and now, with high school content (2014-2015) http://miblsi.cenmi.org/MiBLSiModel/ProfessionalDevelopment/DistrictTrainingScopeSequence.aspx

Model Demonstration Grant Advisory Panel Ed Keme’enui Michael Kamil Deborah Reed Paula Lancaster (IRB support in addition to secondary expertise through her university (Grand Valley State University)

Consultants to Support our Efforts Anita Archer Nancy Marchand-Martella

Targeted Population Middle school (ages 12-14) SLD: Demonstrate significant reading problems (reading at least at the third grade level) Have identified disability Reading goals on IEP

Additional Supports Beyond the Scope and Sequence Dates in bold = scheduled dates that cannot be changed due to presenter schedule. There is also a School Leadership Team component for all 3 Tiers to ensure this work is embedded into the systems of support for the school that is not included on these slides. There is also a “Teaming” component for departments and cross departments to analyze data and develop a plan for

Coaching Component 2 FTE (currently MiBLSi staff with secondary expertise) to provide instructional coaching We are currently working on standardizing the coaching model that will be used Forms, process (e.g., use of video clips, feedback protocol, debriefing lesson segments), etc.

Coaching Component (cont.) Sites we are considering partnership with have identified coaches and also have support from their Intermediate School District in secondary reading We want to think about sustainability beyond our project coaching support

Activity Access the R-TFI Scoring Guide. Focus your attention on Tier 1 Implementation Elementary & Secondary: Items 1.6-1.11 Read the item and the Scoring Guide and conduct an informal self-assessment of your school. Based on the strengths and needs of your school’s Tier 1 reading MTSS model: What local resources and / or other resources can you access to help guide your next steps? Who should you share this information with? Share with your table mates! 64

Session Evaluation Link http://www.surveygizmo.com/s3/2393314/Thursday-AM-1D

Thank you! Kim St. Martin, MiBLSi Assistant Director kstmartin@miblsimtss.org