Implementation Science

Lessons Learned… “It's a wise man who profits by his own experience, but it's a good deal wiser one who lets the rattlesnake bite the other fellow” We did not have the benefit of implementation science when we built this system in TTSD. Now can be much more deliberate and efficient. Josh Billings

Stages of Implementation Exploration/Adoption Installation Development Commitment and acquiring more information about the innovation Initial Implementation Creating the systems and materials to prepare for implementation Provide Significant Support to Implementers; implementer is focused on short-term day to day use of the innovation with little time for reflection Full Implementation Elaboration (Innovation) Use of the innovation is stabilized. Few if any changes are being made in on-going use Continuous Regeneration (Sustainability) Embedding within Standard Practice and is able to find ways to make the innovation more efficient Improvements: Increase Efficiency and Effectiveness

Exploration/Adoption Development Commitment and acquiring more information about the innovation Assess capacity, resources, and buy-in Begin developing awareness and support for adoption (parents, staff, central office) Explain the risks involved in adopting (“What will be different for us as a result?”) Identify staff who will be comfortable managing the risks (change in practice, ambiguity) for leadership roles

Exploration/Adoption Development Commitment and acquiring more information about the innovation Assess whether you will have access to experts (trainers, coaches, colleagues) who know the innovation well Build consensus and commitment for adoption More about developing consensus is coming up next!

Creating the systems and materials to prepare for implementation Installation Installation Creating the systems and materials to prepare for implementation Create “space” for the work to happen Begin recruiting and hiring staff Conduct training

Creating the systems and materials to prepare for implementation Installation Installation Creating the systems and materials to prepare for implementation Develop the infrastructures to support implementation On-going training Coaching and supervision Staff and fidelity evaluations Outcome evaluations How will the work get done? Committees, regular meeting times, how often, how long

Initial Implementation Provide Significant Support to Implementers; implementer is focused on short-term day to day use of the innovation with little time for reflection “You cannot do something well without doing it poorly first!” Survive the awkward stage Learn from mistakes-implementation will feel messy! Continue “buy-in” efforts-people will ask, “What the heck did I agree to do?!” Manage expectations- keep people focused on what is expected for implementation

Initial Implementation Provide Significant Support to Implementers; implementer is focused on short-term day to day use of the innovation with little time for reflection Change practices Put the components you have been planning (infrastructures to implement) in place Change organizational and community structures and culture Manage the change process- “Nope, I change my mind!” Overcome fear and inertia- “I feel like a failure”

Full Implementation Full Implementation Use of the innovation is stabilized. Few if any changes are being made in on-going use Implementers are in a routine while implementing the innovation Comfortable with the materials and components of the innovation Implementation is not awkward Few if any changes are made in the on-going use of the innovation

Elaboration (Innovation) Embedding within Standard Practice and is able to find ways to make the innovation more efficient Replicate school implementations within the district based on initial implementation Select a model demonstration site(s) and then learn from implementation efforts Let early adopters make the mistakes first Continue to refine systems that support how data are analyzed and how decisions are made

Elaboration (Innovation) Embedding within Standard Practice and is able to find ways to make the innovation more efficient Ensure feedback loops are in place between the implementation support systems Building leadership team Implementation Planning Team All the components of the innovation are in place and fully functioning

Continuous Regeneration (Sustainability) Improvements: Increase Efficiency and Effectiveness It is important to first, do it right (fidelity), then do it better (innovate) Continuous Regeneration is doing it better More efficient with implementation It is important to have feedback loops in place to review data to determine if alterations to the innovation are still positively impacting student outcomes

Continuous Regeneration (Sustainability) Improvements: Increase Efficiency and Effectiveness Consultation with: Experts, trainers, coaches, colleagues Systems adoption within the district New staff orientation Should get easier over time

These stages are recursive: Setbacks at one stage moves us back to the previous stage Apply for PBIS New District Initiative Today is a book study? “We already do that.” Violate Norms Vote coach off Be on time Go to a PLC Ignore e-mails Go to Book Study Dominate conversation Snow Day! Late for meeting Attend District PD Ignore Data File Grievance Change Practice Have a “AHA!” Prep for Meeting New State Initiative

Stages of Implementation Establishing consensus is a recursive process. Managing the change process requires the leader to monitor the pulse of staff Exploration/Adoption Installation Development Commitment and acquiring more information about the innovation Initial Implementation Creating the systems and materials to prepare for implementation Provide Significant Support to Implementers; implementer is focused on short-term day to day use of the innovation with little time for reflection Full Implementation Elaboration Use of the innovation is stabilized. Few if any changes are being made in on-going use Continuous Regeneration Principals should anticipate staff who will perceive the change as first or second order before Exploration. When Exploration begins, it will become apparent Embedding within Standard Practice and is able to find ways to make the innovation more efficient Improvements: Increase Efficiency and Effectiveness

Understanding People’s Concerns about Implementation Leading Change is Hard Work!

Implementation Takes Time Exploration (Sustainability) Installation (Sustainability) Initial Implementation Full Implementation (Effectiveness & Sustainability) 2–4 Years Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

Change is Cyclical and Iterative-- 10/12/2010 CONSENSUS Exploration Sustainability INFRASTRUCTURE CONSENSUS Installation Sustainability INFRASTRUCTURE First we build consensus, then add the infrastructure, then implement. But we are always coming back to consensus even after we build the infrastructure and implement practices. Maybe change the names to exploration, Exploration is never done. During installation you haven’t implemented anything yet, hiring new people looking at resources, office space, then need those two pieces before you start day one of the implementation. Consensus happens in exploration, Stages aren’t linear and always overlap between the stages, What if we keep terms we have their now. Keep these terms but tag to the stages either outside or inside of the stages. Now four stages exploration, installation, Sustainability happens at every stage rather than something that is initially built in from day 1. For Innovation: do people ever really get to full innovation? If engaging in improvement cycle while doing this work becoming more innovative while doing this work. You’re always refining products. In full implementation some of the work is innovative. Some of the it’s don’t change, but the processes around implementing it may change. Now only four stages. Maybe label the arrows sustainability? CONSENSUS IMPLEMENTATION Implementation

Implementation Science 7. Special Ed Referral and Evaluation Report 6. Teaming/Data-Based Decision Making: Individual Problem Solving 5. Teaming/Data-Based Decision Making: Group Interventions 4. Progress Monitoring 3. Interventions with Fidelity 2. Core Instruction with Fidelity David: All of these components make up RTI. This is the standard process that TTSD uses to determine eligibility for LD. This is the process that is being “scaled-up” in Oregon. Currently Oregon RTI has worked with 50 districts, (1/4 of districts in the state) 1. Screening Competency Drivers Implementation Science Organization Drivers Leadership

Know Your Driver

Implementation Drivers Sustainable Student Outcomes Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Integrated & Compensatory Competency Drivers Organization Drivers Leadership Adaptive Technical David does this as last one in this section. Even the most effective interventions (the ones showing the most promise in small-scale studies) cannot improve student outcomes if they aren’t implemented as intended. This graphic lists the essential components necessary for effective implementation and sustained change. These components are called “drivers” because they drive the implementation. There are three types of drivers, Competency, Organization, and Leadership. (Integrated and Compensatory). Comensatory: Holding a group of balls under the water. You may not get them all at once, and when you focus on one, another one ops up. But you can manage most of them most of the time. © Fixsen & Blase, 2008

Selection Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Competency Drivers Organization Drivers Integrated & Compensatory Sally: In this session we will be discussing the Competency Drivers. These are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment Use of all of the drivers leads to effective … which leads to sustainable student outcomes Selection is the first of these drivers. Selection Leadership © Fixsen & Blase, 2008

Sample Interview Questions for Elementary Principal Position If you were to walk into a classroom with excellent instruction, what would you expect to see?   If you were to walk into a classroom with poor instruction, what would you do about it?  What are the common features of Positive Behavior Intervention Support and Response to Intervention systems? What is your experience coordinating, leading, or performing within these systems?   How would you teach reading to a 1st grader?   

Interview Data Activity Directions: Please review the data provided for the following six (6) students. At the bottom of each page, please complete the instructional recommendation section and include your rationale.

Selection gone wrong Hillsboro Ths principal

Training Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Training Competency Drivers Organization Drivers Integrated and Compensatory PD is not a panacea to address every problem PD must be housed in a systems to support this effective practices resulting in successful sustainable student outcomes. Selection Leadership © Fixsen & Blase, 2008

Implementation Driver: Training Purpose: Train staff in specific skills needed to implement the EBP Created a PD plan that is developed collaboratively across departments PD continually recycles key elements (i.e., core) to address drift and train new staff Data from fidelity measures (SET, EBIS Group Planning Form, etc) used to inform PD plan Data from outcome measures (DIBELS, Maze, OAKS) used to inform PD plan Training plan developed each year, in the spring of the previous year. All interventionists are trained. Core is Trained. District level coaches ensure trainings occur with support from district administrators. Train in-house trainers. Analyze fidelity data. Add links for SET data and Group Planning, Dibels, etc. Ongoing coordinated

Implementation Driver: Training New staff receive an extra five days of training and extra coaching throughout the year that focuses on EBIS components All PBIS teams get a full day each year for training, coaching, and time to plan All principals trained to conduct walkthroughs and instructional rounds for coaching and instructional leadership

Coaching Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Coaching Training Competency Drivers Organization Drivers Integrated and Compensatory PD is not a panacea to address every problem PD must be housed in a systems to support this effective practices resulting in successful sustainable student outcomes. Purpose: Ensure transfer from training to practice Selection Leadership © Fixsen & Blase, 2008

Implementation Driver: Coaching Purpose: Ensure transfer from training to practice A “Coaching Delivery Plan” is developed Lead Person is designated and supported Coaches are fluent with RTI and PBIS Feedback from multiple sources is collected, provided to practitioners and used to determine areas in need of more coaching and training

Implementation Driver: Coaching Joyce and Showers, 2002 OUTCOMES (% of Participants) TRAINING COMPONENTS Demonstrate Knowledge Skill Use in the Classroom Theory and Discussion 10% 5% 0% + Demonstration in Training 30% 20% + Practice & Feedback in Training 60% + Coaching in Classroom 95%

Implementation Driver: Coaching District level coaching Attend grade level EBIS meetings Provide feedback using Coaching Rubric Facilitate monthly meetings with school literacy coaches, counselors and principals Update EBIS Handbook annually Get Walkthrough form.

Implementation Driver: Coaching School level coaching: Principal lead walkthroughs/Instructional rounds Literacy specialists coach teachers and instructional assistants on instructional practices and literacy across content areas Counselors and PBIS team members coach teachers and instructional assistants on behavior Get Walkthrough form.

Performance Assessment (Fidelity) Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Performance Assessment (Fidelity) Competency Drivers Organization Drivers Leadership Coaching Coaching Training Training Integrated and Compensatory Selection Selection © Fixsen & Blase, 2008

Implementation Driver: Performance Assessment Purpose: Ensure EBP is Implemented with fidelity Performance of practices, teams and coaches are assessed at least annually. (Quarterly until fidelity is reached.) Assessment measures include direct observation, and review of fidelity and outcome measures. Feedback is provided and successes are celebrated Are we doing what we said we would do?

Implementation Driver: Performance Assessment Data Sources: Fidelity Assessment: SET, Benchmarks of Quality, EBIS Group Planning Form, Coaching Rubric, Walk Through Forms Outcome measures: DIBELS, Maze, OAKS Annual EBIS Systems Assessment: DSSP Continuous Improvement Plan goals and Elementary School Literacy Plans are directly linked to these measures. Are we doing what we said we would do?

Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Performance Assessment (Fidelity) Coaching Training Selection Competency Drivers Organization Drivers Leadership Integrated & Compensatory Decision support: Helps you make decisions Data System: System of data, tools, and data management systems that allow you to know what is working—or not– and why, so that you can intervene. Decision Support Data System © Fixsen & Blase, 2008

Implementation Driver: Decision Support Data System Purpose: Employ data systems that guide MTI and RTI practices in order to increase student achievement Implementation Fidelity Assessments Data from SET (School-wide Evaluation Tool) Coaching Rubric, walk through form, etc. Elementary EBIS Team Planning Survey Annual EBIS Systems Assessment: DSSP Put a slide in from IPAS.

Implementation Driver: Decision Support Data System Student Performance Measures Outcomes measures: OAKS, ODRs, DIBELS Universal Screening: DIBELS, OAKS, ODRs, Maze Progress Monitoring: Curriculum Based Measures (AIMSweb, DIBELS) Diagnostic Measures: Grade +, Phonics Screener Put a slide in from IPAS.

Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RTI) Performance Assessment (Fidelity) Coaching Training Selection Integrated and Compensatory Competency Drivers Organization Drivers Leadership Facilitative (Barrier Busting) Administration Clearing the path so teachers can do their best Decision Support Data System © Fixsen & Blase, 2008

Implementation Driver: Facilitative Administration Purpose: To ensure that administrators provide staff with support to implement EBP AND ensure that staff are implementing with fidelity. Support Schedule Resources Training Procedures Monitoring Fidelity checks Principal Walk Throughs Staff Evaluation process

Sustainable Student Outcomes Training Coaching Selection Systems Intervention Performance Assessment (Fidelity) Facilitative Administration Decision Support Data System Competency Drivers Organization Drivers Leadership Integrated and Compensatory Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Something about aligning things and the other things on the next page. Not sure any of your graphics work. © Fixsen & Blase, 2008

Implementation Driver: Systems Intervention Engage and grow “champions” District leadership actively participates with state education department agencies and legislature Multiple district and school level teams engage in reciprocal communication (PEP – PIP cycles) Positive feedback provided to schools on achieving CIP Goals

Sustainable Student Outcomes Effective Implementation of Evidence Based Practice (e.g. RtI) Performance Assessment (Fidelity) Coaching Systems Intervention Training Facilitative Administration Competency Drivers Organization Drivers Integrated and Compensatory Decision Support Data System Selection Leadership © Fixsen & Blase, 2008 Adaptive Technical

Adaptive vs Technical Leadership

Lessons Re-Learned… OR What SISEP Has Taught Us! Pick the right people Teach them what to do Coach them Make sure they’re doing it right Determine if the system is working Keep lines of communication open and support your people Get help from everyone you can Celebrate success! “About the most originality that any writer can hope to achieve honestly is to steal with good judgment”. Josh Billings