Karen A. Blase, PhD National Implementation Research Network FPG Child Development Institute University of North Carolina at Chapel Hill MiBLSi Coaches Conference Lansing, Michigan November 21, 2013 Implementation Drivers: The Infrastructure to Promote Success

Education continues to rely on the “excellent teacher” myth rather than on the “improved system” approach. Excellent teachers are a gift, and we should admire, learn from and support them. But the real issue is how to (a) train more “excellent teachers”, (b) give them curricula that really works, and (c) give them a system that makes it easier and more likely that excellent teaching happens by design rather than by heroic efforts to overcome the system. ~ Rob Horner, 2013 Formula for Success

Socially Significant Outcomes Effective & Usable Interventions Effective Implementation Methods Enabling Contexts Formula for Success © 2012 Dean Fixsen and Karen Blase, National Implementation Research Network

Socially Significant Outcomes Effective & Usable Interventions Effective Implementation Methods Enabling Contexts Formula for Success © 2012 Dean Fixsen and Karen Blase, National Implementation Research Network MiBLSi Coaching Functions Effective Implementation Methods Use Data About Effective Implementation Processes

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). “Evidenced-Informed” Implementation Strategies Download at:

Experimental Data Show These Methods, When Used Alone, Are Insufficient: Diffusion/ Dissemination of information Training Passing laws/ mandates/ regulations Providing funding/ incentives Organization change/ reorganization Data: Realize 5% to 15% Intended Outcomes Across Disciplines… Necessary But Not Sufficient

EFFECTIVE & USABLE INTERVENTIONS – What exactly are educators saying and doing that improve outcomes for students? STAGES – How do we match our activities to the stage? TEAMS – Who takes responsibility for and helps guide the change process? …..at each level? IMPROVEMENT CYCLES – How can we create more hospitable environments, efficiently solve problems, and get better? AND….How do we change the behavior of educators, administrators, and organizations? Making It Happen Effective & Usable Interventions Stages Improvement Cycles Teams ©Copyright Dean Fixsen and Karen Blase 5 Active Implementation Frameworks

MiBLSi Implementation Drivers “Implementation drivers (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005) are fundamental elements that promote implementation of practices with fidelity and durability and can be brought to scale. These drivers focus on three integrated components essential to effective organizations: Leadership, Staff Competency and Organizational Capacity.” ~MiBLSi Implementation Drivers Reference Sheet (Sept. 2013) Our Focus Today – Fifth Framework

Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Did We Do What We Said We Would Do? Fidelity data serve as a diagnostic tool for improvement

When my grandmother died I kept her cookbook. Inside was this poem that she must have clipped from a senior center or church newsletter. I didn't have potatoes, so I substituted rice. Didn't have paprika, so I used another spice. I didn't have tomato sauce, so I used tomato paste. A whole can not a half can - I don't believe in waste. My friend gave me the recipe - she said you couldn't beat it. There must be something wrong with her, I couldn't even eat it! - Susan Kay

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative SO THAT practices are implemented with fidelity

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching

Recruitment and Selection Purposes: Select for the “unteachables” and pre-requisites Allow for mutual selection through interview process Improve likelihood of retention after “investment” Improve likelihood that training, coaching and supervision will result in implementation Selection Through an Implementation Lens Implementation Best Practices: Job or role description clarity about accountability and expectations Sampling of skills and experience is related to “new practices” and expectations “Coachability” assessment Skilled interviewers Using Data for Integration and Compensatory Features Feed interview information forward to trainers, coaches, administrators

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching

Pre-Service and In-Service Training Through An Implementation Lens Purposes: Knowledge acquisition Basic Skill Development “Buy-in” Implementation Best Practices: Theory grounded (adult learning) Skill-based – Behavior Rehearsals vs. Role Plays – Knowledgeable Feedback Providers – Practice to Criteria Feedback to Selection and Feed Forward to Supervision Data-based (pre and post testing)

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching

OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5%0%..+Demonstration in Training 30% 20% 0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002 Training and Coaching Drivers

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity (assessments) Selection Training Coaching

N=153 Aarons et al., 2009 Coaching and Fidelity Assessments and Staff Turnover

Coaching Through an Implementation Lens Implementation Best Practices: Design a Coaching Service Delivery Plan Develop accountability structures for Coaching – Coach the Coach! – Regular satisfaction feedback from those who are coached – Regular review of adherence to Coaching Service Delivery Plan – Look at data – Fidelity, Staff Satisfaction with Support, Skill Acquisition

Fidelity/Performance Assessment Purposes and Practices: Ensure and motivate implementation Reinforce staff and build on strengths Develop skills and abilities – feedback for coaches Interpret Outcome Data Diagnostic Tool for Building, District, and ISD Implementation Teams on functioning of the Drivers – Recruitment and Selection Practices – Training Programs (pre and in-service) – Supervision and Coaching Systems – Organization Drivers – Leadership Drivers Transparent, timely, celebratory, for action planning Competency Drivers

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative SO THAT practices are implemented with fidelity (assessments) Selection Training Coaching

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching Decision Support Data System Information Materials/Tools/ Resources Feedback

Decision Support Data Systems Thru an Implementation Lens Purposes and Practices: Provide information to assess and improve fidelity and effectiveness of MiBLSi – programs, practices, frameworks Celebrate success Be accountable for a functional infrastructure (are Drivers ‘working’) and for positive student outcomes Useful data are: – Reliable (standardized protocols, trained data gatherers) – Reported frequently (e.g. weekly, quarterly) – Reported at relevant and “actionable” levels (e.g. student, classroom, grade level, subject, school-wide) – Widely shared – Practical to collect – Useful for and used for making decisions (PDSA)

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching Decision Support Data System Information Materials/Tools/ Resources Feedback Facilitative Administration Management/ Coordination Vision

Adapted from Fixsen & Blase, 2008 Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity Selection Training Coaching Decision Support Data System Information Materials/Tools/ Resources Feedback Facilitative Administration Management/ Coordination Vision

Facilitative Administrative Through An Implementation Lens Purposes: Facilitates moving through implementation stages Ensures effective use of Competency Drivers Takes the lead on Systems Interventions Utilizes data for improvement Looks for ways to make MiBLSi more efficient and effective!! Implementation Best Practices: Links “on purpose” to multi-level, functional implementation teams (e.g. school, district, ISD, SEA) Uses feedback to ensure quality of all Implementation Drivers Revises policies and procedures to support MiBLSi Solicits and uses feedback from each “level” Reduces administrative barriers

Adapted from Fixsen & Blase, 2008 Selection Decision Support Data System Facilitative Administration Feedback Training Coaching Information Vision Materials/Tools/ Resources Management/ Coordination Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity General Capacity

Adapted from Fixsen & Blase, 2008 Selection Decision Support Data System Facilitative Administration Feedback Training Coaching Information Vision Materials/Tools/ Resources Management/ Coordination Successful Student Outcomes Program / Initiative Set of practices that are implemented with fidelity

Leadership Based on meta-analysis of 30 years of leadership studies, transformation leaders make changes that “disturb every element of the system.” They – Break with the past – Operate outside of existing paradigms – Conflict with prevailing values and norms – Find solutions that are emergent, unbounded, and complex Waters, Marzano, McNulty (2003)

Types of Leadership Needed Different strategies for different challenges – Technical Leadership – Adaptive Leadership According to Ron Heifitz and his colleagues, one of the biggest mistakes is to incorrectly identify the type of challenge you are facing – Using technical approaches for adaptive issues

Technical Challenges Perspectives are aligned (views, values) Definition of the problem is clear Solution and implementation of the solution is relatively clear, although may be complicated There can be a “primary” locus of responsibility for organizing the work

Technical Leadership Use established norms/goals Define problems Provide solutions Clarify roles and responsibilities Assign tasks Manage conflict Maintain order

Adaptive Challenges Legitimate, yet competing, perspective emerge Definition of the problem is unclear There are different perspectives on the “issue” at hand Solution and implementation are unclear and require learning Primary locus of control is not single entity or person Ron Heifetz, Leadership without Easy Answers, 1996

Adaptive Leadership Get on the balcony Identify the adaptive challenge Regulate distress Maintain disciplined attention Give the work back to the people Protect all voices

Challenges Technical Building a new school Choosing a new curriculum Adopting a discipline policy Evaluating Teachers Adaptive Deciding to build a new school Choosing a new reading curriculum Fairly administering a discipline policy Engaging the union in teacher evaluation

Managing the Flow Technical Adaptive

Periodic Assessment of Implementation Drivers: Discussion Tool Who is Accountable? How much control do you have over the Driver? Shift needed? Rating of Importance to Quality and Sustainability (1=low to 5=high) Level of Assistance from Others (e.g. Coaches, TA) (1=low to 5=high) Quality Control process (e.g. effectiveness of the Driver) How integrated with other Drivers? (1 = siloed to 5 = Highly integrated) Competency Drivers for Position: _______ QualitySust. Selection Training Coaching Fidelity/Performance Monitoring System Organization Drivers Decision Support Data Systems (Outcome and Process) Facilitative Admin at School Level Facilitative Admin at District Level Facilitative Admin at ISD Level © 2013, Karen Blase, National Implementation Research Network

Exploration and Periodic Assessment of Leadership Driver: Discussion Tool What challenges do you expect? How can they be avoided or minimized? What challenges are on the table now? Technical, adaptive, combo? Who needs to be part of the solutions? What ‘next right steps” seem appropriate given the nature of the challenge? Competency Drivers for Position: _______ Selection Training Coaching Fidelity/Performance Monitoring System Organization Drivers Decision Support Data Systems (Outcome and Process) Facilitative Admin at School Level Facilitative Admin at District Level Facilitative Admin at ISD Level © 2013, Karen Blase, National Implementation Research Network Leadership Driver – Technical and Adaptive

Results from Child Wellbeing Project Case Example: Metz et al. ComponentT1T2T3 Selection * 1.89* Training * 1.10 Coaching * 1.83* Perf. Assessment * DSDS * Fac. Administration * Systems Intervention * 2.00* Average Composite Score * 1.83* Fidelity (% of cases) 18%83% Success Coach model involved intense program development of core intervention components and accompanying implementation drivers

Education continues to rely on the “excellent teacher” myth rather than on the “improved system” approach. Excellent teachers are a gift, and we should admire, learn from and support them. But the real issue is how to (a) train more “excellent teachers”, (b) give them curricula that really works, and (c) give them a system that makes it easier and more likely that excellent teaching happens by design rather than by heroic efforts to overcome the system. ~ Rob Horner, 2013 Formula for Success

Free 24/7 Implementation Training for You and Your Team NIRN’s newly launched “Active Implementation Hub” is a free, online learning environment for use by any stakeholder — practitioners, educators, coaches, trainers, purveyors — involved in active implementation and scaling up of programs and innovations. The site goal is to increase the knowledge and improve the performance of persons engaged in actively implementing any program or practice.” The AI Hub is an initiative of the State Implementation & Scaling- up of Evidence-based Practices Center (SISEP) and The National Implementation Research Network (NIRN) located at The University of North Carolina at Chapel Hill’s FPG Child Development Institute.

©Copyright Dean Fixsen and Karen Blase This content is licensed under Creative Commons license CC BY- NC-ND, Attribution-NonCommercial-NoDerivs. You are free to share, copy, distribute and transmit the work under the following conditions: Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this work for commercial purposes; No Derivative Works — You may not alter or transform this work. Any of the above conditions can be waived if you get permission from the copyright holder.

Fidelity in the Funnies