1. Scaling Up Innovations Dean L. Fixsen, Ph.D. & Karen A. Blase, Ph.D. University of North Carolina - Chapel Hill Rob Horner, Ph.D. University of Oregon George Sugai, Ph.D. University of Connecticut OR Superintendents’ Forum April 2010 Scaling Up Effectiveness

2. OR Public Schools  Students: 565,000  Schools: 1,800  School Districts: 196  Counties: 36  Budget: $8.5 Billion “Bottom 20%”  113,000  360  39  36 Challenges

3. Follow Through Programs Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.

4.  Hattie (2009) recently reported a meta- analysis of 816 meta-analyses  52,649 research studies in education involving over 83 million students, teachers, staff, parents, and others.  "It is what teachers get the students to do in the class that emerged as the strongest component of the accomplished teachers' repertoire." Hattie (2009) Achieving Student Benefits

5.  Increasing opportunities to respond and the amount/ accuracy of feedback is an important correlate of student achievement  The feedback to the teachers about what students can and cannot do is more powerful than feedback to the student  This requires a change in the conception of what it means to be a teacher – not a solo performer Hattie (2009) Achieving Student Benefits

6.  We now know a lot about WHAT to do to educate students  We can improve education for students – on purpose! Achieving Student Benefits

7. Science “to” Service SCIENCE SERVICE GAP IMPLEMENTATION

8. Challenges  Science to Service Gap  What is known is not what is adopted to help students  Implementation Gap  What is adopted is not used fully and effectively in practice

9. Achieving Student Benefits Good Intentions Actual Supports Years 1-3 Outcomes Every Teacher Trained Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of the teachers received support Vast majority of students did not benefit Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Longitudinal Studies of a Variety of Comprehensive School Reforms

10.  You are not alone!  Superintendents across the nation are facing the same problems:  Lack of consistency across teachers, schools, and years  Lack of capacity to make meaningful changes and sustain them Challenges

11. In 2007  State of Oregon participated in a process to select States to create an infrastructure for implementation of innovations statewide.  1 of 36 interested States  1 of 16 applicant States  1 of 6 chosen States that met the selection criteria and site visit criteria (IL,MI, MN, MO, OR, VA) SISEP Center

12. State Implementation and Scaling up of Evidence-based Practices (SISEP) www.scalingup.org Dean Fixsen and Karen Blase National Implementation Research Network, University of North Carolina at Chapel Hill Rob Horner University of Oregon George Sugai University of Connecticut

13. The SISEP Center – Intensive and focused activity to build state capacity and align system structures, roles, and functions Use implementation science and best practices across programs and innovations Large scale, real time change Capacity Building

14.  Students cannot benefit from interventions they do not experience  Teachers and staff have to change if students are to benefit Dobson & Cook (1980) Challenges

15.  Know-WHAT  Knowledge of the intervention  Know-HOW  Knowledge of implementation Tucker, Edmondson, & Nembhard (2005) Implementation Science

16. Know WHAT Choose Interventions Wisely  Meaningful Improvement  Must be “worth the effort” to scale up (e.g. EBISS)  Eventually want to see educationally and socially significant changes in student outcomes across the State

17. Know HOW EffectiveNOT Effective Effective NOT Effective IMPLEMENTATION INTERVENTION Student Benefits Highly variable, often ineffective, sometimes harmful to students, families, and adults (Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999) Poor Outcomes

18. Implementation Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 Implementation Research: A Synthesis of the Literature

19. Data Show These Methods, When Used Alone, Do Not Result In Uses of Innovations As Intended:  Diffusion/ Dissemination of information  Training  Passing laws/ mandates/ regulations  Providing funding/ incentives  Organization change/ reorganization Implementation Science

20. Data Show These Methods, When Used Alone, Do Not Result In Uses of Innovations As Intended  We know a lot about ineffective methods because they are the ones we use!  Implementation science will improve as implementation practices improve (create a better “laboratory”) Implementation Science

21. Student Benefits Technical Integrated & Compensatory Performance Assessment (Fidelity) Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System InnovationOrganization Leadership Adaptive Exploration (Sustainability) Installation (Sustainability) Initial Implementation Full Implementation (Effectiveness & Sustainability) Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 2 – 4 Years Implementation Takes Time

22.  Implementation Drivers  Common features of successful supports to help make full and effective uses of a wide variety of innovations Implementation Science

23. © Fixsen & Blase, 2007 Integrated & Compensatory Performance Assessment (Fidelity) Coaching Training Selection Staff Competence Systems Intervention Facilitative Administration Decision Support Data System Organization Supports Technical Leadership Adaptive Reliable Benefits for Students Consistent uses of Innovations

24. OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5%0%..+Demonstration in Training 30% 20% 0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002 Staff Coaching

25. Support Implementation  Students cannot benefit from education practices they do not experience  Support implementation practices within schools and districts

26. Capacity Building  Scaling up = at least 60% of the students who could benefit from an innovation have access to that innovation  Achieve significant benefits to students and society

27.  Letting it happen  Recipients are accountable  Helping it happen  Recipients are accountable  Making it happen  Purposeful use of implementation practices and science  Implementation teams are accountable Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004 Capacity Building

28.  Letting it happen  Recipients are accountable  Helping it happen  Recipients are accountable  Making it happen  Implementation Teams are accountable: THEY DO THE WORK (Heart of Scaling) Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004 Capacity Building

29. Implementation Team Minimum of three people (four or five preferred) to promote effective, efficient, and sustainable implementation, organization change, and system transformation work Tolerate turnover; teams are sustainable even when the players come and go

30. Implementation Team A group that knows the innovations very well (formal and craft knowledge) A group that knows implementation very well (formal and craft knowledge) A group that knows improvement cycles to make intervention and implementation methods more effective and efficient over time

31. Implementation Team School & District Supports Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Teacher & Staff Competence State and Community Supports Regional Authority Supports Implementation Team Simultaneous, Multi-Level Interventions

32. Implementation Team Prepare Communities Prepare schools and staff Work with Researchers Assure Implementation Prepare Regions Assure Student Benefits Create Readiness Parents and Stakeholders © Fixsen & Blase, 2009

33. Impl. TeamNO Impl. Team Effective Effective use of Implementation Science & Practice IMPLEMENTATION INTERVENTION 80%, 3 Yrs 14%, 17 Yrs Balas & Boren, 2000 Fixsen, Blase, Timbers, & Wolf, 2001 Implementation Science Letting it Happen Helping it Happen

34. School Wide PBS

35. 12% in 17 Years (1992-2009)

36. Costs and Savings Short-Term Investment in Imple. Capacity Realize Long- Term Benefits

37.  This year’s success pays for next years increase in capacity Barber & Fullan (2005) Costs and Savings

38. Change Systems To scale up, we need to:  Turn policy into effective practice  Create an infrastructure for implementation of innovations  Turn effective practice into policy

39. OR Public Schools  Students: 565,000  Schools: 1,800  School Districts: 196  Counties: 36  Budget: $8.5 Billion “Bottom 20%”  113,000  360  39  36 Challenges PROBLEM: The “bottom 20%” is distributed throughout the state and shifts each year SOLUTION: Plan capacity to reach ALL schools

40. © Fixsen & Blase, 2008 N = 565,000 All Students & Families School Teachers and Staff State Department Leadership State District Leaders and Staff

41. © Fixsen & Blase, 2008 N = 565,000 All Students & Families School Teachers and Staff State Department Leadership State 1 for each School (N = 1,800 School Teams) School Implementation Team (N=4) N = 7,200 N = 6 N = 288 Implementation- Skilled Workforce N = 76 < 0.1% $$ N = 70 Re-Purpose “District” Implementation Teams (N=4) 1 for every group of 25 Schools (N = 72 “District” Teams) Regional Implementation Teams (N=5) 1 for every group of 5 “Districts” (N = 14 Regional Teams) 1 for every 10 RITs (N = 1 State Teams) State Transformation Team (N=6)

42. Intensive Development Saturation Intensive Development

43. Oregon Districts and Student Enrollment ODE Report Card 2008-2009

44. Scale Up  To scale up interventions we must first scale up implementation capacity  Building implementation capacity is essential to maximizing the statewide use of EBPs and other innovations

45. Oregon needs  About 14 Regional Implementation Teams (and support staff)  One State Transformation Team  Annual cost about $8 million  About $40,000 per year per district (or $14 per year per student) Scale Up

46. System Change  Innovative practices do not fare well in existing organizational structures and systems  An infrastructure for implementation does not exist  Organizational and system changes are essential to successful use of innovations

47. Legacy Systems  A legacy system is a system or application that continues to be used despite its poor competitiveness and compatibility with modern equivalents  Difficult to integrate new systems into legacy systems because it is a difficult and time intensive process to understand current system functionalities  Legacy methods create a huge conversion challenge for implementation teams Ashok R. B. Samuel (2009)

48. EXISTING SYSTEM EFFECTIVE INNOVATIONS ARE CHANGED TO FIT THE SYSTEM EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION EFFECTIVE INNOVATION System Change

49. Compliance and Crises, Urgent, Time Sensitive!! Services not meeting Standards Deal with urgent and high profile issues Best Practices Implemented Fully With Good Outcomes System Supports & Stability Regulatory roles Basic Data Systems Financing and Fiscal Accountability Accreditation/ Licensing Standards HR rules and regulations Safety Standards Work with Legislature Inclusion of Stakeholders System Supports & Stability Mandates, System Supports, Foundational Polices & Regulations Leadership Responsibilities and Leverage Points Thanks to Tom Bellamy

50. Implementation Team Management Team Teachers Innovations Students Policy Enabled Practice (PEP) Practice Informed Policy (PIP) System Change “External” System Change Support Adaptive Challenges RFP methods IHE curricula Salaries Funding Credentialing Licensing Time/ scheduling Union contracts Duplication Fragmentation Hiring criteria Federal/ State laws

51. SYSTEM ALIGNMENT State Department Districts Schools Teachers/ Staff Effective Practices ALIGNMENT Federal Departments Implementation Teams FORM SUPPORTS FUNCTION

52. The End in Mind  With the purposeful use of implementation science, we can:  Make statewide use of good instruction, evidence-based practices, and other innovations…  To produce increasingly effective outcomes for all students…  For the next 50 years.

53. Implementation Science  Global Implementation Conference 2011  www.implementationconference.org www.implementationconference.org  Integrate the science, practice, policy of implementation, organization change, and system transformation

54. Call for Applications  Do you know of an organization/coalition currently implementing an evidence based practice in their local community?  Is the implementation of this program/innovation producing beneficial outcomes to the community?  Are they a role model for moving the evidence-based practice from science to service? If you answered “YES” to all of the above, check out: http://www.samhsa.gov/scienceandservice Nominate your own organization or someone you know!! Science and Service Award Program

55. For More Information Dean L. Fixsen, Ph.D.  919-966-3892  dean.fixsen@unc.edu dean.fixsen@unc.edu Karen A. Blase, Ph.D.  919-966-9050  karen.blase@unc.edu karen.blase@unc.edu Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC http://nirn.fpg.unc.edu/

56. For More Information State Implementation and Scaling up of Evidence- based Practices (SISEP) Dean Fixsen, Karen Blase, Rob Horner, George Sugai www.scalingup.org “Resources” Tab  Concept paper  Annotated bibliography  Data on scaling up  Scaling up Briefs

57. Evidence-based Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 Implementation Research: A Synthesis of the Literature

58. Thank You for your Support  Annie E. Casey Foundation (EBPs and cultural competence)  William T. Grant Foundation (implementation literature review)  Substance Abuse and Mental Health Services Administration (implementation strategies grants; national implementation awards)  Centers for Disease Control & Prevention (implementation research)  National Institute of Mental Health (research and training grants)  Juvenile Justice and Delinquency Prevention (program development and evaluation grants  Office of Special Education Programs (Scaling up Capacity Development Center)  Administration for Children and Families (Child Welfare Leadership Development)  Duke Endowment (Child Welfare Reform)