Karen A. Blase, PhD Dean L. Fixsen, PhD, Melissa Van Dyke, LCSW, Allison Metz, PhD, Sandra Naoom, MSPH, Michelle Duda, Ph D From Outcomes to Implementation July 31, 2010

2 ECO Logic Model (simplified) Early Childhood Outcomes Center ECO Center Activities Strategic use of data High quality state outcomes measure- ment systems Improved services, supports, and practices Improved outcome s for children and families Celebrate!

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Early Childhood Outcomes Center 3 State Early Childhood Accountability and Improvement System A system of standards-based assessments of (a) children’s development and learning and (b) program quality, designed to inform state policy decisions, investments, and improvement efforts for early education programs for three- and four- year-old children, linked to a continuum of kindergarten through third grade standards, curriculum, assessments, and program improvement efforts. National Early Childhood Accountability Task Force

From Outcomes to Implementation 4Early Childhood Outcomes Center ECO Center Activities Strategic use of data High quality state outcomes measure- ment systems Improved services, supports, and practices Improved outcomes for children and families Implementation Science

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Goals Today Understand how implementation science and frameworks can positively impact program improvement efforts what functions data serves in the implementation process Think about implementation-informed efforts to improve program quality and how to align policy to support effective programs and practices.

6Early Childhood Outcomes Center ECO Center Activities Strategic use of data High quality state outcomes measure- ment systems Improved services, supports, and practices Improved outcomes for children and families Implementation Science What is implementation? How do we use the science?

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Early Childhood Programs, Services, and Data Systems ….Through an Implementation Lens

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 But FIRST… What Do We Mean by Implementation? A specified set of activities designed to put into practice a policy, activity, or program of known dimensions. Implementation processes are purposeful and defined in sufficient detail such that independent observers can detect the presence and strength of these “specified activities”

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 What Works EffectiveNOT Effective Effective NOT Effective IMPLEMENTATION – The HOW INTERVENTION The WHAT High Fidelity = Good Outcomes Low Fidelity = Poor Outcomes Getting Here and Staying Here is the Challenge!

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Science Review and synthesis of the implementation evaluation literature (1970 – 2004) Multi-disciplinary Multi-sector Multi-national

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: Implementation Research: A Synthesis of the Literature

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Science Implementation science is universal (like physics, chemistry)

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 There are Shared Challenges!! Implementation is implementation in: Agriculture and Forestry Manufacturing and Business Substance abuse Child welfare, Education, Engineering, Health and Medicine Juvenile Justice, Mental health, Nursing, …and more They have similar implementation problems They have similar implementation solutions We can learn more …more quickly

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Data Show These Methods, When Used Alone, Do Not Result In Implementation As Intended: Diffusion/ Dissemination of information Training Passing laws/ mandates/ regulations Providing funding/ incentives Organization change/ reorganization Business as Usual ≠ Impact Necessary But Not Sufficient

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 What Works EffectiveNOT Effective Effective NOT Effective IMPLEMENTATION – The HOW INTERVENTION The WHAT Performance Implementation (High Fidelity) Paper Implementation Procedure Implementation (Low Fidelity) Getting Here and Staying Here is the Challenge!

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Pre-Requisite: Defining “IT” From what current state to what future state? The “it” must be operationalized whether it is: An evidence-based practice or program A best practice Initiative A broad early childhood systems change initiative Operationalize Part of Speech: verb Definition: to define a concept or variable so that it can be measured or expressed quantitatively Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7) Copyright © Lexico Publishing Group, LLC

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation = Putting “It” Into Action Who will do what differently? Who will help change occur? AND How will change happen? At all levels…. Practice (individual child data) Program (summative “unit” child data) Organization(s) (agency or coalition data) System (state data)

Higher Fidelity is correlated with better outcomes across a wide range of programs and practices I didn't have potatoes, so I substituted rice. Didn't have paprika, so I used another spice. I didn't have tomato sauce, so I used tomato paste. A whole can not a half can - I don't believe in waste. My friend gave me the recipe - she said you couldn't beat it. There must be something wrong with her, I couldn't even eat it! ~ Senior Center Newsletter Policy Implications 1.IF THERE IS NOT THE “WILL” OR “MEANS” TO DO IT RIGHT YOU WON’T GET RESULTS 2.YOU CAN’T DO A COST-BENEFIT ANALYSIS IF THERE IS NO BENEFIT! WHAT WORKS ~ IMPLEMENTING PROPERLY MATTERS

Higher Fidelity is correlated with better outcomes across a wide range of programs and practices I didn't have potatoes, so I substituted rice. Didn't have paprika, so I used another spice. I didn't have tomato sauce, so I used tomato paste. A whole can not a half can - I don't believe in waste. My friend gave me the recipe - she said you couldn't beat it. There must be something wrong with her, I couldn't even eat it! ~ Senior Center Newsletter Measure Fidelity AND Measure Outcomes BECAUSE you need to know: 1.Are we having an implementation problem? Low fidelity & Poor outcome = implementation problem 2.Or are we having an effectiveness problem? High fidelity & Poor outcome = effectiveness problem Program Improvement Implications

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Frameworks Multi-dimensional, Fully integrated Implementation Drivers Improvement Cycles Implementation Teams Implementation Stages Need attention to all but today focus on just the first two

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Frameworks Multi-dimensional, Fully integrated Implementation Drivers Improvement Cycles Implementation Teams Implementation Stages

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Drivers… HOW to: Build Competency and Confidence Develop, improve, and sustain competent & confident use of innovations. Change Organizations and Systems Create and sustain hospitable organizational and systems environments for effective instructional and educational services Provide Leadership that Matches the Challenge

Core Implementation Components © Fixsen & Blase, 2008 Improved Early Childhood Outcomes Competency Drivers Organization Drivers Leadership Evidence-Based and Evidence- Informed Programs and Practices How: What: Why: Capacity to provide direction, vision, & the “right” leadership approach Staff capacity to support children/families with the selected practices Institutional capacity to support staff in implementing practices with fidelity

© Fixsen & Blase, 2008 Performance Assessment Coaching Training Selection Competency Drivers Improved Early Childhood Outcomes Implementation Drivers = Infrastructure Implementation Lens Evidence-Based and Evidence-Informed Programs and Practices

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Coaching Driver OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10%5%0%..+Demonstration in Training 30%20%0% …+ Practice & Feedback in Training 60% 5% …+ Coaching in Classroom 95% Joyce and Showers, 2002

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 Sobering Observations "All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get." R. Spencer Darling Business Expert

© Fixsen & Blase, 2008 Performance Assessment Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Competency Drivers Organization Drivers Graphics by Steve Goodman,2009 Implementation Drivers = Infrastructure Improved Early Childhood Outcomes Evidence-Based and Evidence-Informed Programs and Practices

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The Ghost System) System Stability

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (Ghost System) Effective System Change EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION (Host System) EFFECTIVE INNOVATION

© Fixsen & Blase, 2008 Performance Assessment Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Adaptive Technical Competency Drivers Organization Drivers Leadership Implementation Drivers = Infrastructure Improved Early Childhood Outcomes Evidence-Based and Evidence-Informed Programs and Practices

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Types of Leadership Needed Different strategies for different challenges Technical Leadership Adaptive Leadership According to Ron Heifitz and his colleagues, one of the biggest mistakes “leaders” make is to incorrectly identify the type of challenge they are facing Using technical approaches for adaptive issues

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 The Adaptive Work of the Leader Get on the Balcony Identify the Adaptive Challenge Regulate Distress Maintain Disciplined Attention Give the Work Back to the People Protect All Voices »R. Heifetz and D. Laurie: The Work of Leadership. Harvard Business Review, 1998.

© Fixsen & Blase, 2008 Performance Assessment Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Adaptive Technical Integrated & Compensatory Competency Drivers Organization Drivers Leadership Implementation Drivers = Infrastructure Improved Child Outcomes EBP/Program/Initiative/Framework for Effective Practices & System Change

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Integrated and Compensatory Implementation Drivers Integrated Consistency in philosophy, goals, knowledge and skills across these processes (S/T/C/SE/DSDS/FA/SI) Compensatory At the practitioner level At the program level

© Fixsen & Blase, 2008 Performance Assessment Coaching Training Selection Systems Intervention Facilitative Administration Decision Support Data System Adaptive Technical Integrated & Compensatory Competency Drivers Organization Drivers Leadership Implementation Drivers = Infrastructure for Data Systems Improved Early Childhood Outcomes System Reliable, Valid, Timely, Actionable Child, Fidelity, and Program-Level Data

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Skill and “Will” Current Skills Will Do New Skills Needed Sufficient “Will” Exists Data as Feedback more likely to improve program quality Data necessary but not sufficient. Need to Build Competence and Confidence even if ‘incentives’ and policy are aligned Lack of “Will” New Policies, Funding, Incentives will be needed to ‘motivate’ and enable the use of current skill sets Need all the Drivers in play to: Increase confidence and competence Be guided by data Create a hospitable environment Provide leadership for adaptive challenges

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Skill and “Will” Current Skills Will Do New Skills Needed Sufficient “Will” Exists Data as Feedback is more likely to improve program quality Data necessary but not sufficient. alone won’t work. Need to build Competence and Confidence even if ‘incentives’ and policy are aligned Lack of “Will” New Policies, Funding, Incentives will be needed to ‘motivate’ use of current skill sets and create a hospitable environment Need all the Drivers in play to: Increase confidence and competence Be guided by data Create a hospitable environment Provide leadership for adaptive challenges

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Implementation Frameworks Multi-dimensional, Fully integrated Implementation Drivers Improvement Cycles Implementation Teams Implementation Stages

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Improvement Cycle Uses Practice-Policy Feedback Loops Transformation Zones Usability Testing Rapid Cycle Teams Problem-solving Practice Improvement What Do They Have In Common?

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 PDSA Cycles Shewhart (1924); Deming (1948); Six- Sigma (1990) Plan – Develop specific things to do (based on DATA) Do – Do them (make sure) (DATA) Study – See what happens (DATA) Act – Make adjustments (based on DATA) Cycle – Do over and over again until the goal is reached (DATA) or the problem is clearer (go again)

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Improvement Cycle Uses Practice-Policy Feedback Loops Transformation Zones Usability Testing Rapid Cycle Teams Problem-solving Practice Improvement

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Policy Practice Policy (Plan) Practice (Do) Policy Practice Policy Enabled Practices (PEP) “External” System Change Support Structure Procedure

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Policy Practice Feedback Loops Policy (Plan) Practice (Do) Structure Procedure Policy Practice Form Supports Function at every level (National, State, District, Region, Agency, Practitioner) Feedback Study - Act Policy Enabled Practices (PEP) Practice Informed Policy (PIP) “External” System Change Support

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 Policy Practice Feedback Loops Policy (Plan) Practice (Do) Form Supports Function at every level (National, State, District, Region, Agency, Practitioner) Feedback Study - Act Policy Enabled Practices (PEP) Practice Informed Policy (PIP) “External” System Change Support Requires: Buy – in from all Facing Adaptive Challenges Linked Teams (right issues at the right level) Linking Communication Protocols Transparent Formal Regular Intervals Rewards Honesty Able to be Evaluated

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 System Alignment State Departments Communities, Alliances, Councils, Provider Agencies Practitioners Effective Practices ALIGNMENT Federal Departments Implementation Teams FORM SUPPORTS FUNCTION

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 For More Information Dean L. Fixsen, Ph.D Karen A. Blase, Ph.D At the Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: Monograph/ Monograph/ To order the monograph go to: