Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing Gaps Between Policy and Practice

Acknowledgements  Special thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission. 2

Sixth Year of Working with SEAs  State Implementation & Scaling-up of Evidence-based Practices Center  scalingup.org scalingup.org 3

Implementation Science`  A policy is one thing  Implementation of a policy is a very different thing  Students cannot benefit from a policy they do not experience

An Implementation Failure labeled as an Intervention Failure Example: $500 million for Homebuilders (Family Support Services)  Funding only for interventions  No fidelity criteria insisted upon by the developers  National evaluation = not effective Chapin Hall Center for Children, 2002

An Implementation Failure labeled as an Intervention Failure Example: $500 million for Homebuilders (Family Support Services)  Funding only for interventions  No fidelity criteria insisted upon by the developers  National evaluation = not effective  But, over 25% was spent on in-office interventions with parents or children (< 0 fidelity) Chapin Hall Center for Children, 2002

Longitudinal Studies of CSR Programs Source: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Evidence-Base For Effectiveness Every Teacher Trained Every Teacher Continually Supported Actual Supports Years 1-3 Fewer than 50% of the teachers received some training Fewer than 25% of those teachers received support Fewer than 10% of the schools used the CSR as intended Vast majority of students did not benefit Outcomes Years 4-5

Formula for Success 8 Socially Significant Outcomes Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts

Formula for Success 9 WHY: Socially Significant Outcomes Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts (Vision!)

Formula for Success 10 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts

Formula for Success 11 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions Effective Implementation Methods WHERE: Enabling Contexts

Formula for Success 12 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions HOW & WHO: Effective Implementation Methods WHERE: Enabling Contexts

Table Discussions  Share a situation you have experienced/ observed where one of the variables in the formula was zero

Implementation Science

Excellent evidence for what does not work  Implementation without changing supporting roles and functions  Implementation by edict  Diffusion/dissemination of information by itself  Implementation by “following the money”  Training alone, no matter how well done Paul Nutt (2002). Why Decisions Fail

What does work? An Implementation Framework  Implementation Teams  Implementation Drivers  Improvement Cycles  Implementation Stages

An Implementation Framework

IMPLEMENTATION Implementation Science EffectiveNOT Effective Effective NOT Effective INTERVENTION (Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999) Poor outcomes Inconsistent, not sustainable, poor outcomes Poor outcomes; sometimes harmful

IMPLEMENTATION Implementation Teams Effective INTERVENTION 80%, 3 Yrs 14%, 17 Yrs Impl. TeamNO Impl. Team NOT Effective Balas & Boren, 2000 Green, 2008 Fixsen, Blase, Timbers, & Wolf, 2001

Implementation Team  A group that knows:  the intervention/policy  implementation  improvement cycles

Simultaneous, Multi-Level Interventions School Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Teacher Region District Implementation Team State Partners

Iowa’s Collaborating for Kids (C4K) 22

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Cascading Logic Model

An Implementation Framework

Improvement Cycles  Practice-Policy Communication Loops  Plan-Do-Study-Act cycles  Usability Testing 25

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Feedback Loops Feedback Loops

North Carolina Example 30

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Authentic Work Informs the PEP-PIP Cycle Plan-Do- Study-Act Plan-Do- Study-Act

Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Authentic Work Informs the PEP-PIP Cycle Usability Studies

Table Discussions  How do teams at each level of your system interact?  What work are the teams doing with each other?  What feedback loops exist between teams?  How might you increase interaction among teams?  Have/might you incorporate improvement cycles into pilots and policy innovations? 33

Growing Implementation Capacity Using Transformation Zones

Implementation Team  Minimum of 3 people (4-5 preferred)  Tolerate turnover; sustainable

State Transformation Team First School Implementation Team N = 10 Schools Growing Implementation Capacity (Phase 1) External Support & 2 FTEs First Regional Implementation Team N = 9 Staff Invest up front to build capacity

State Transformation Team First School Implementation Team N = 10 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff)

State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff) STAFF

Table Discussions  What responsibility will your state take to ensure implementation of your policies?  How can you grow implementation capacity within your state?  How can you rethink staffing in order to invest in the early stages of implementation? 42

Poll  One key take-away OR  What questions do you have? 43

Possibilities to consider…. 44  SCEE webinars  SCEE Topical Meetings  SCEE Discussion Groups  GIC  Individual state follow-up

Thank you 45