Investing in local implementation capacity: Statewide scale up a multi-tiered school reform model for fidelity and sustainability Kim St. Martin Assistant Director, MiBLSi kstmartin@miblsimtss.org Steve Goodman Director, MiBLSi sgoodman@miblsimtss.org OSEP Project Directors’ Conference July 2013 Investing in local implementation capacity: Statewide scale up a multi-tiered school reform model for fidelity and sustainability miblsi.cenmi.org

Session Description This session describes a scale-able and durable statewide model that emphasizes data-based decision making around Student outcomes, Implementation fidelity of effective reading and behavior support practices, and Organizational structures of supports. Strategic planning focuses on developing intermediate and local school district capacity to implement with durability in a self-modifying system. Suggestions for scaling-up will be provided.

What is MiBLSi? A statewide structure to create capacity for an integrated Behavior and Reading Multi-Tiered System of Support (MTSS) that can be implemented with fidelity, is sustainable over time and utilizes data-based decision making at all levels of implementation support

Participating Schools The Early Years 2009 Schools (150) 2006 Schools (50) 2007 Schools (165) 2008 Schools (95) 2004 Schools (21) 2005 Schools (31) Participating Schools 2000 Model Demonstrations (5) This slides illustrates moving from model demonstration schools to replication schools. Note that it is more likely to replicate in areas of the state where earlier demonstrations have previously been established. 4

Emphasis of MiBLSi Work Over Time Focus on providing direct PD and TA to Implementers (Team Training, Focus Training) Focus on providing direct PD and TA to Implementers (Team Training, Focus Training) MiBLSi develop capacity for project implementation Focus on providing direct PD and TA to Implementers (Team Training, Focus Training) MiBLSi develop regional capacity for MiBLSi regional implementation Focus on providing direct PD and TA to Implementers and cascading levels of support MiBLSi develop local capacity for LEA/ISD implementation Focus on providing direct PD and TA to cascading levels of support LEA/ISD develop local capacity for LEA/ISD implementation 2003 2005 2007 2010 2017

Replications Replications Note: the ducks at the end of the row are less clear that the closest ducks (most recent) as we continue each iteration in replications re become clearer in the work we do Through each replication, we become more clear in our implementation efforts

Why we moved from school to district model… Schools do not always get to criteria for fidelity of implementation. Schools do not always maintain implementation efforts Supporting a school based participation model cannot be scaled and is not durable as district based

Implementation Science Longitudinal Studies of a Variety of Comprehensive School Reforms Effective Interventions Actual Supports Years 1-3 Outcomes Years 4-5 Every Teacher Trained Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of those teachers received support Vast majority of students did not benefit Aladjem, D. K., & Borman, K. M. (2006, April). Summary of Findings from the National Longitudinal Evaluation of Comprehensive School Reform. Paper presented at the Annual meeting of the American Educational Research Association, San Francisco, CA. Vernez, G., Karam, R., Mariano, L. T., & DeMartini, C. (2006). Evaluating comprehensive school reform models at scale: Focus on implementation. Santa Monica, CA: RAND Corporation. Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006

Two Categories of Work Access to Effective Practices Ensuring that students (staff) have access to effective practices in an integrated behavior and reading Multi-Tiered System of Supports Practices are selected based on: Need, Fit, Resource Availability, Evidence, Readiness for Replication, Capacity to Implement Support for the Practices Ensuring that the interventions are implemented correctly with the “right people”, at the “right time”, in the “right amounts” (Implementation Fidelity)

Moving from Model Demonstration to Standard Practice Modifying System for Standard Practice Examine Feasibility/Efficacy of Practice Being Developed Embed w/in existing work: Provide access as current practice Model Demonstration: Does it Work in a specific setting with a specific population? Replications: Can it be reproduced in different settings with different populations? Scale-Up: Increase “reach” of practice to critical mass of implementation Invest heavily in resources to produce results Utilize typical/existing resources

Vision for Supporting Scaling MTSS Across Michigan Local capacity to support implementation of MTSS with fidelity will be developed and provided within … 47 of 57 Intermediate School Districts (approx. 80%) 319 of 798 Local Education Agencies (approx. 40%) 1595 or 4192 schools (approx. 38%) Local capacity support means… MTSS Coordination Implementation Team Cabinet level (executive leadership) support Effective/efficient data systems for collecting, storing and analyzing data at district level District trainers (training capacity) Coaching structures (coaching capacity and systems for coaching to occur in an on-going way)

Vision for Supporting Scaling MTSS Across Michigan Continued “Scaling up is defined as having enough of something so that it is useful. Scaling up is the process of moving from “exemplars” to the “typical.” The process of scaling involves the development of organizational capacity to move from exemplars sustained by extra-ordinary supports, to typical application with typical supports”. “Scaling up is defined as having enough of something so that it is useful. Scaling up is the process of moving from “exemplars” to the “typical.” The process of scaling involves the development of organizational capacity to move from exemplars sustained by extra-ordinary supports, to typical application with typical supports. While there is no firm agreement about the level at which “scaling” is achieved, we hypothesize that an organization (district) has reached the “tipping point” for functional scaling when approximately 40% of the units in the organization are implementing a practice with fidelity. At that point, the education system would have changed to provide typical supports for evidence-based practices across the (district).” Fixsen et al 2008 Fixsen et al 2008

MiBLSi add value by providing to districts assistance to address… Michigan School Improvement Framework ESEA Flexibility Waiver State Board of Education policy on Positive Behavioral Interventions and Supports Matt’s Safe School Law (bully prevention) Drop Out Prevention State Board of Education Policy on Seclusion and Restraint State Board of Education Resolution to Address School Discipline Issues Impacting Student Outcomes State Board of Education Professional Learning Policy, Standards and Guidance Disproportionality in race and discipline MiBLSi collaborates with ISDs to develop local capacity in supporting constituent districts as they work to successfully implement an integrated behavior and reading MTSS model. MiBLSi professional development materials, resources and technical assistance align with the following mandates, priorities and policies of the State of Michigan Department of Education:

What makes a program scale-able or scale-worthy? Your Turn Your Turn What makes a program scale-able or scale-worthy?

MiBLSi Value-Added Work System Political Regulatory/Legal Economy Geography Society/Culture Environmental Factors MiBLSi Value-Added Work System Receivers Stakeholders/Funders Determine worth Management/Coordination Capital Resources People Materials Information Work Systems: Providing the MTSS practices and the supports for the these practices to take place successfully within schools and districts Financial Evaluation Professional Learning Technical Assistance MiBLSi Value-Added Work System Investments: Funding Visibility Political support This is a graphic representation of the MiBLSi value added system. It is important to note that the concept of “value added” is strongly related to the feedback from the stakeholders and the consumers of MiBLSi. Given the current economic environment, it is more important than ever that we wisely invest in practices that demonstrate successful outcomes. Additionally, we need to be good stewards of the resources made available to us. We understand that schools/district can choose to implement MTSS on their own or choose another MTSS program. It is important that we focus on the “value added” through participation with MiBLSi. This value comes from experience in developing MTSS skills and practices that have been implemented across time, locations, and various populations. Added to this is an emphasis on ensuring that structures are in place to successfully support implementation of MTSS practices. If MiBLSi is to improve and scale-up, we need to be cognizant of what brings educators into the program and why they continue on. Returns: Addressing critical issues (Discipline/Ethnicity) Addressing program directives (State Performance Plan) Consumers schools, districts, ISDs/RESAs Valued MTSS Products/ Services Priority, Plan, Formative Assessment, Continuous Improvement, Risk Management

Technical Assistance Regions Provides technical assistance and information dissemination to Intermediate School Districts (ISDs) and Local Education Agencies (LEAs) within a regional structure This slide displays a map of Michigan with the state sectioned off into various regions for the purpose of providing technical assistance

Developing Organizational Infrastructure

Transformation Zone: Make it easy to change Transformational Process: Developing “Host Environments” for MTSS Implementation with Fidelity Existing System of Practice Transformation Zone: Make it easy to change Desired System of Practice: Make it easy to sustain This figure depicts the transformation zone as the place to transition from “what is” (the left hand side) to “what is envisioned” (the right hand side). We call the transformation zone the “zone of discomfort” in recognition of the difficulties inherent in changing the ways of work among teachers, building administrators, district staff, individuals and leaders within the state education system, and parents and stakeholders. Implementation Teams Analyze existing system of practice Develop Plan Coordinate and monitor Create/modify materials Identify barriers to implementation Modify plan for continuous improvement Adjust system to address new challenges/pressures to implementation fidelity Coordinate and monitor

Statewide Structure of Support Provides guidance, visibility, funding, political support for MiBLSi Students Building Staff Building Leadership Team LEA District Cabinet and Implementation Team Across State Multiple ISD/LEA Teams All staff All students Multiple schools w/in local district Provides guidance, visibility, funding, political support, and implementation supports Provides coaching and TA for LEA and/or ISD Teams Provides guidance and manages implementation Provides effective practices to support students Improved behavior and reading ISD Cabinet and Implementation Team Regional Technical Assistance Michigan Department of Education/MiBLSi Leadership Multiple LEAs w/in intermediate district Who is supported? Statewide Structure of Support This stair step of a cascading system illustrates that there are units within educational systems that build upon each other and are connected through feedback loops and mechanisms of support How is support provided? Statewide Structure of Support

Feedback Loops Successful Student Outcomes Feedback Loops Develop Organizational Structures Schools effectively implementing MTSS Practices Successful Student Outcomes Inputs Outcomes Outcomes In the transformation zone, reflective feed back and feed forward loops are embedded in implementation, organization change, and system transformation methods. A key guiding principle for scaling up involves, defining, strengthening, and aligning the interfaces between and among different operating/ system components by increasing the flow of relevant and reflective two-way-information. Develop Staff Competencies Feedback Loops Based on formative and summative measures Compared with external standards set by stakeholders/consumers and also internal standard set by project Feedback Loops

Leadership Function of District Teams Cabinet Level Team Implementation Planning Team Vision Political support Priority Visibility Allocation of resources Barrier busting 1 Collect, summarize, evaluate data Develop district plan Coordinate and monitor plan Create/modify materials and tools Identify barriers to implementation Coordinations between the Cainet and the Implementation Planning Team includes communication around four variable: Need to know Nice to know Don’t need to know Need to act on 3 Coordination 2

Implementation Supports District Model Cabinet Team Implementation Team Vision Political support Priority Visibility Allocation of resources Barrier busting Collect, summarize, & eval data Develop plan Coordinate/monitor plan Create/modify materials Identify barrier Liaison Coordination MTSS Coordinator Implementation Supports Direction Training Coaching Content Expertise Materials Evaluation Coordinations between the Cabinet and the Implementation Planning Team includes communication around four variable: Need to know Nice to know Don’t need to know Need to act on Schools Schools District Model

Your Turn Your Turn What are critical components necessary to ensure sustainable implementation?

Supporting Implement Fidelity

Logic of MiBLSi Emphasis is on developing local implementation capacity Partnership between Intermediate School District (ISD) and Local School District focuses on shared ownership and accountability around implementation of an integrated MTSS model Schools/Districts will be supported through stages of implementation rather than through a training sequence Goal: Schools/Districts implement MTSS practices with fidelity supported by infrastructure the promotes continuous improvement and sustainability

An analogy of implementation stages When you read the album cover, it appears that all track are equal in duration. When you actually look at the record you see that the tracks vary in duration. Additionally, even went moving through the tracks (stages) a scratch may cause you to be stuck in one location for awhile and a bump may send you to an earlier track.

Stages of Implementation Focus Stage Description Exploration/Adoption Decision regarding commitment to adopting the program/practices and supporting successful implementation. Installation Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan. Initial Implementation Try out the practices, work out details, learn and improve before expanding to other contexts. Elaboration Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation. Continuous Improvement/Regeneration Make it easier, more efficient. Embed within current practices. Should we do it? Work to do it right! Implementation of reading and behavior MTSS is a process that takes place over time (stages of Implementation). Within a school setting, this process may take 2 – 4 years; district-wide, this process will take many more years to achieve. It is important to remember that we should not expect to see change in student outcomes until effective practices have been implemented with fidelity. Work to do it better!

Implementation Drivers Successful Student Outcomes Implementation Drivers Program/Initiative (set of practices that are implemented with Fidelity) Staff competency to support students/families with the selected practices Organizational capacity to support staff in implementing practices with fidelity Competency Capacity There are three categories of Implementation Drivers: Leadership, Capacity and Competency. When these core components are in place they provide the support to a successful implementation that will be sustained. Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention Leadership Implementation Drivers Adapted from Fixsen & Blase, 2008 Ability to provide direction/vision of process

Illustration of Guided Notes for District Meetings

Exploration/Adoption Installation Initial Implementation Focus of Work: 2012- 2014 Sep Oct Nov Dec Jan Feb Mar Apr May ISD Cabinet ISD Implementation Team ISD Implementation Team (Phase I & Phase II Conversations) Focus of Work: 2012- 2014 LEA Cabinet LEA Implementation Team This slide illustrates a timeline of the focus of the work over the rest of the current school year and into the next. LEA Implementation Team Admin Teams Admin Teams School Teams Exploration/Adoption Installation Initial Implementation

Developing ISD Infrastructure Developing the structures to help LEAs develop support structures or to directly support school implementation This figure illustrates a map of Michigan with dots representing Intermediate School Districts. The intermediate School Districts are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts. District Cohort 1 District Cohort 2 Readiness work for District Cohort 3

Focused Planning Sessions Provide for an efficient dissemination of content and technical assistance for developing local implementation infrastructure. These meeting are conducted monthly in several regions throughout the state. The full day sessions involve content delivery and translation to contextual fit. Participants: ISD implementation team members MiBLSi TAP(s) Other members of from the transformation zone team TAP takes lead in coordination

Developing LEA Infrastructure Accessing implementation support from ISD This figure illustrates a map of Michigan with dots representing Intermediate School Districts with their constituent local school districts. The intermediate School District along with constituent local school districts are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts and constituent local districts. District Cohort 1 District Cohort 2 Readiness work for District Cohort 3 LEA

Local Education Agency: Scope and Sequence Topic 6: Evaluating the Effectiveness of MTSS Implementation Supports (District Implementation Team) Pre-Session: Leading District Implementation (District Liaison and MTSS Coordinator Topic 1: Leading District Implementation (District Implementation Team) Leading District Implementation: (Administrative Team) Topic 2: District Data Review Process (District Implementation Team & Administrative Team) Topic 3: Developing District Capacity for Sustaining MTSS (District Implementation Team) MTSS Common Threads and What Really Works (Administrative Team) Topic 4: Selection Procedures (District Implementation Team & Administrative Team) Topic 5: Supporting Buildings in Stages of MTSS Implementation (District Implementation Team) Sessions Timeline for 2013 -2014

Training Schools Teams Accessing implementation support from LEA or ISD This figure illustrates a map of Michigan with dots representing Intermediate School Districts with their constituent local school districts and schools. The intermediate School District along with constituent local school districts and school are clustered together for training purposes as demonstrated by the circles around each cluster of Intermediate School Districts, constituent local districts and schools. District Cohort 1 District Cohort 2 Readiness work for District Cohort 3 LEA School

Training Schools Teams Continued The number of schools to be selected for involvement in your training plan will depend on the constant tension of three factors regarding implementation Training Schools Teams Continued Resource Allocation Moving through the stages: How fast you move through these stages will depend on the constant tension of three factors regarding implementation illustrated in the figure. If you want quality implementation, then you may need either more time to engage implementation work or you may need more resources allocated to get the work done. Implementation Quality Implementation Timeframe

Your Turn Your Turn What are critical components necessary to ensure fidelity of implementation?

Lessons Learned for Scaling Up

Supporting Implementation Changes over time Support for your first 50 schools (model demonstrations) Support for your first 500 schools (learning how to scale and support implementation in complex systems) Support for scaling and beyond (becoming standard practice of how we do education)

Supporting Infrastructure Use existing structures to scale up the work No new funding available, need to be creative with existing funding stream Re-allocate staff Organize around efficiencies to make room for the work

Local Context Counts Local Context Counts Emphasize adoption of critical non- negotiable “Big Ideas” Allow for contextual variations in implementation of “Big Ideas”

Transformation Zones Transformation Zones Learn from “Transformation Zones” to help in mapping out scaling up plan Learn what works from those most willing to implement More dynamic- changes with need and participation