1. Circe Stumbo, West Wind Education Policy Inc. and CCSSO CCSSO/SCEE National Summit on Educator Effectiveness April 10, 2013 Implementation Science: Closing Gaps Between Policy and Practice

2. Acknowledgements  Special thanks to Dean Fixsen, Karen Blase, and the National Implementation Research Network for their generosity in sharing their findings and their passion for all things implementation. This PowerPoint slide deck is built off their original work, with permission. 2

3. Sixth Year of Working with SEAs  State Implementation & Scaling-up of Evidence-based Practices Center  scalingup.org scalingup.org 3

4. Implementation Science`  A policy is one thing  Implementation of a policy is a very different thing  Students cannot benefit from a policy they do not experience

5. An Implementation Failure labeled as an Intervention Failure Example: $500 million for Homebuilders (Family Support Services) 1993-1998  Funding only for interventions  No fidelity criteria insisted upon by the developers  National evaluation = not effective Chapin Hall Center for Children, 2002

6. An Implementation Failure labeled as an Intervention Failure Example: $500 million for Homebuilders (Family Support Services) 1993-1998  Funding only for interventions  No fidelity criteria insisted upon by the developers  National evaluation = not effective  But, over 25% was spent on in-office interventions with parents or children (< 0 fidelity) Chapin Hall Center for Children, 2002

7. Longitudinal Studies of CSR Programs Source: Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Evidence-Base For Effectiveness Every Teacher Trained Every Teacher Continually Supported Actual Supports Years 1-3 Fewer than 50% of the teachers received some training Fewer than 25% of those teachers received support Fewer than 10% of the schools used the CSR as intended Vast majority of students did not benefit Outcomes Years 4-5

8. Formula for Success 8 Socially Significant Outcomes Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts

9. Formula for Success 9 WHY: Socially Significant Outcomes Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts (Vision!)

10. Formula for Success 10 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions Effective Implementation Methods Enabling Contexts

11. Formula for Success 11 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions Effective Implementation Methods WHERE: Enabling Contexts

12. Formula for Success 12 WHY: Socially Significant Outcomes WHAT: Effective Policies/ Interventions HOW & WHO: Effective Implementation Methods WHERE: Enabling Contexts

13. Table Discussions  Share a situation you have experienced/ observed where one of the variables in the formula was zero

14. Implementation Science

15. Excellent evidence for what does not work  Implementation without changing supporting roles and functions  Implementation by edict  Diffusion/dissemination of information by itself  Implementation by “following the money”  Training alone, no matter how well done Paul Nutt (2002). Why Decisions Fail

16. What does work? An Implementation Framework  Implementation Teams  Implementation Drivers  Improvement Cycles  Implementation Stages http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

17. An Implementation Framework http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

18. IMPLEMENTATION Implementation Science EffectiveNOT Effective Effective NOT Effective INTERVENTION (Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999) Poor outcomes Inconsistent, not sustainable, poor outcomes Poor outcomes; sometimes harmful

19. IMPLEMENTATION Implementation Teams Effective INTERVENTION 80%, 3 Yrs 14%, 17 Yrs Impl. TeamNO Impl. Team NOT Effective Balas & Boren, 2000 Green, 2008 Fixsen, Blase, Timbers, & Wolf, 2001

20. Implementation Team  A group that knows:  the intervention/policy  implementation  improvement cycles

21. Simultaneous, Multi-Level Interventions School Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Teacher Region District Implementation Team State Partners

22. Iowa’s Collaborating for Kids (C4K) 22

23. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Cascading Logic Model

24. An Implementation Framework http://sisep.fpg.unc.edu/learning-zone/science-of-implementation/implementation-frameworks

25. Improvement Cycles  Practice-Policy Communication Loops  Plan-Do-Study-Act cycles  Usability Testing 25

26. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

27. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

28. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Policy Enabled Practice (PEP) Practice Informed Policy (PIP) Practice/Policy Communication Loops

29. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Feedback Loops Feedback Loops

30. North Carolina Example 30

31. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Authentic Work Informs the PEP-PIP Cycle Plan-Do- Study-Act Plan-Do- Study-Act

32. Students & Families Building Teachers and Staff Building Leadership and Implementation Team District Leadership and Implementation Teams Regional Implementation Teams State Transformation Team State Department of Education Leadership State Authentic Work Informs the PEP-PIP Cycle Usability Studies

33. Table Discussions  How do teams at each level of your system interact?  What work are the teams doing with each other?  What feedback loops exist between teams?  How might you increase interaction among teams?  Have/might you incorporate improvement cycles into pilots and policy innovations? 33

34. Growing Implementation Capacity Using Transformation Zones

35. Implementation Team  Minimum of 3 people (4-5 preferred)  Tolerate turnover; sustainable

36. State Transformation Team First School Implementation Team N = 10 Schools Growing Implementation Capacity (Phase 1) External Support & 2 FTEs First Regional Implementation Team N = 9 Staff Invest up front to build capacity

37. State Transformation Team First School Implementation Team N = 10 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff)

38. State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

39. State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

40. State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff)

41. State Transformation Team First School Implementation Team, Expanded N = 20 Schools Growing Implementation Capacity (Phase 2) External Support & 2 FTEs First Regional Implementation Team N = 3 Staff Subsequent Regional Implementation Team (3 Staff) Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent School Implementation Teams N=20 Schools Subsequent Regional Implementation Team (3 Staff) STAFF

42. Table Discussions  What responsibility will your state take to ensure implementation of your policies?  How can you grow implementation capacity within your state?  How can you rethink staffing in order to invest in the early stages of implementation? 42

43. Poll  One key take-away OR  What questions do you have? 43

44. Possibilities to consider…. 44  SCEE webinars  SCEE Topical Meetings  SCEE Discussion Groups  GIC  Individual state follow-up

45. Thank you 45