Scaling Up: From Research to National Implementation Policy Maker’s Summit Center for Evidence-Based Practice: Young Children with Challenging Behavior Washington, DC November 17, 2006 Karen A. Blase, Ph.D. Dean L. Fixsen, Ph.D. National Implementation Research Network Florida Mental Health Institute (c) Dean Fixsen and Karen Blase, 2004

The Overall Puzzle Each year, more research is done More careful methods are crafted for reviewing research and identifying evidence-based practices and programs More attention is directed to evidence- based practices and programs in journals, conferences, and meetings Yet, it is a challenge to realize sustainable benefits for children, families, and caregivers on a broad scale (c) Dean Fixsen and Karen Blase, 2004

The Center’s Pieces of the Puzzle Focus of research has been on making a difference in the world…the child’s natural environment Wisdom and commitment to: Attend to values and “fit” for caregivers and children – What matters and for whom! Recognize multi-level influences (e.g. workforce development issues, policy, organizational culture) Build caregivers capacity and practitioner ability (e.g. attending to the independent variables) Inspire and create hope….tell the story and tout the science (c) Dean Fixsen and Karen Blase, 2004

Naming The Challenge Science to Service Gap Implementation Gap What is known is not what is adopted to help children, families, and caregivers Implementation Gap “….While we have good evidence that the trajectory of a child’s social and emotional development can be changed, the field lacks the necessary information to ensure adoption and sustainability of these program practices…” Recommended Practices: Program Practices for Promoting the Social Development of Young Children and Addressing Challenging Behavior - Fox We talk about ebp’s as though they are taking over the service sectors from mental health intervention to primary prevention…from older adults to infants. While the fact of the matter is that what is known is not what is generally adopted to help children, families and adults. Along with the Science to Service Gap there is the Implementation Gap: There are not clear pathways from a series of rigorous scientific studies to widespread use of a practice or program. When adoption does occur what is is adopted is often not done with fidelity and benefits to consumers are not realized. And often with staff turnover, the practice or program walks out the door with the exiting staff. (c) Dean Fixsen and Karen Blase, 2004

Going to Scale: What Would It Take? How many implementations of a program or practice will be needed to address needs nationally? 800 child care and referral agencies 14,310 school districts 51,000 incorporated child care centers 906,993 children enrolled in Head Start 11.6 million children under age 5 (with estimates of 10 to 15% with mild to moderate problems) We need to produce THOUSANDS of replications of effective programs and practices across the country. We need to learn to work in hospitable and difficult implementation sites. We need to develop and replicate effective and robust implementation systems. We need both a science and practice of implementation. (c) Dean Fixsen and Karen Blase, 2004

Research to Service GAP RESEARCH IMPLEMENTATION SERVICE (c) Dean Fixsen and Karen Blase, 2004

The Implementation Gap It is one thing to say with the prophet Amos, “Let justice roll down like mighty waters,” and quite another to work out the irrigation system. William Sloane Coffin Social activist and clergyman (c) Dean Fixsen and Karen Blase, 2004

The National Implementation Research Network (NIRN) Craft knowledge EBP purveyors (program developers) EBP implementation site managers Implementation researchers Survey of EBP program developers Scientific information Program development and replication data Synthesis of the implementation evaluation and research literature (c) Dean Fixsen and Karen Blase, 2004

From the Synthesis of the Literature We Know That… Implementation issues are common across widely diverse domains Implementation solutions are common across widely diverse domains HOPE – Implementation research findings likely have broad practice implications across domains! (c) Dean Fixsen and Karen Blase, 2004

Puzzle Pieces Effective intervention practices and programs + Effective implementation practices = Good outcomes for children, families and consumers The usability of a program has little to do with the weight of the evidence regarding program outcomes Evidence on effectiveness helps us select what we want implement Evidence on outcomes does not help us implement the program No other combination of factors reliably produces desired outcomes for children, families, and caregivers (c) Dean Fixsen and Karen Blase, 2004

Puzzle Pieces IMPLEMENTATION Effective NOT Effective Paper & Process Implementation (Low or No Fidelity) Performance Implementation (High Fidelity) Effective INTERVENTION NOT Effective So if we are to close the science to service gap we must also close the implementation gap…effective interventions and effective implementation are both necessary. Implementation can be effective when it is focused on performance implementation, that is practitioners and program personnel are doing the program “as intended” ….. It is also possible to have a highly effective intervention that is NOT effectively implemented. Too often paper implementation occurs, the mandates, the policies and the rhetoric are all in place and available for a paper or verbal “audit”. Or process implementation occurs. Didactic training and orientation to the new practice occurs, but supervision systems don’t change, data are collected but are not available in a timely understandable format and not used for decision-making. But these changes do significantly impact front line practice or generate consumer benefits. Only performance implementation of an effective intervention that changes behavior at the practitioner, program and system level makes a difference. (c) Dean Fixsen and Karen Blase, 2004

Puzzle Pieces Excellent experimental evidence for what does not work Dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines) Training alone, no matter how well done, does not lead to successful implementation While the science is not strong the science that does exist is clear about a couple of things. Spray and pray strategies don’t work. This is likely not the type of faith-based initiative that the Federal Gov’t is interested in. Information dissemination alone does not lead to successful implementation. Training alone, no matter how well done does not lead to successful implementation (c) Dean Fixsen and Karen Blase, 2004

Puzzle Pieces Excellent experimental evidence for what does not work Implementation by edict does not work Implementation by “following the money” does not work Implementation without supporting role and function changes does not work Paul Nutt (2002). Why Decisions Fail (c) Dean Fixsen and Karen Blase, 2004

What Seems to be Required Integration of Implementation Factors Core Implementation Components: Training, Coaching, Performance Measurement Organizational Components: Selection, Program Evaluation, Administration, Systems Intervention Influence Factors: Social, Economic, Political Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 (c) Dean Fixsen and Karen Blase, 2004

What Seems to be Required Successful implementation on a useful scale requires a purveyor An individual or group of individuals representing a program or practice who actively work with organizations and communities to help them implement that practice or program with fidelity and good effect Purveyors accumulate data & experiential knowledge to become more effective and efficient over time (c) Dean Fixsen and Karen Blase, 2004

Implementation Framework Organizational Structures/Culture Practitioner Purveyor Fidelity & Outcome Measures Evidence-based Practices Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 (c) Dean Fixsen and Karen Blase, 2004

Implementation Framework Organizational Structures/Culture Infrastructure (Train, Coach, Evaluate) Purveyor Fidelity & Outcome Measures Evidence-based Practices Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 (c) Dean Fixsen and Karen Blase, 2004

Simultaneous, Multi-Level Interventions Active Purveyor Role Simultaneous, Multi-Level Interventions Organization Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Practitioner State, County and Local Context Service Systems Purveyor Purveyors also quickly learn that the Influence Factors are critical to the success of the implementation effort and over time take on a very active, simultaneous and multi-level intervention role to help increase the likelihood that such meta-contingencies as funding, licensing, referral mechanisms, regulations, and reporting requirements are aligned to support the new way of work Federal Context (c) Dean Fixsen and Karen Blase, 2004

Purveyor Sources and Strategies Purveyor group organized and promoted by the originators (e.g. Success for All, Positive Behavior Support) Implementation team with the knowledge, skill, freedom, and authority to act (e.g. within a larger organization or a collaboration of agencies) An Intermediary Purveyor Organization that becomes expert in implementation and a “bridge” or expert with multiple EBPs (a new way of doing T & TA). (c) Dean Fixsen and Karen Blase, 2004

What Seems to be Required Purveyors Help Ensure the Integration of Implementation Factors Core Implementation Components: Training, Coaching, Performance Measurement Organizational Components: Selection, Program Evaluation, Administration, Systems Intervention Influence Factors: Social, Economic, Political Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 (c) Dean Fixsen and Karen Blase, 2004

Implementation Drivers Implementation Drivers Coordinate and Operationalize the Core Implementation Components and the Organizational Components Implementation Drivers are mechanisms that Help to develop, improve, and sustain practitioners’ ability to implement an intervention to benefit children Help ensure sustainability and improvement at the organizational level (c) Dean Fixsen and Karen Blase, 2004

Implementation Drivers STAFF PERFORMANCE EVALUATION SUPERVISION & COACHING DECISION SUPPORT DATA SYSTEMS INTEGRATED & COMPENSATORY FACILITATIVE ADMINISTRATIVE SUPPORTS PRESERVICE & INSERVICE TRAINING RECRUITMENT AND SELECTION SYSTEMS INTERVENTIONS (c) Dean Fixsen and Karen Blase, 2004

Integrated and Compensatory Consistency in philosophy, goals, knowledge and skills across these processes (S/T/C/E/A/SI) Compensatory At the Practitioner Level At the Program Level (c) Dean Fixsen and Karen Blase, 2004

Implementation Drivers STAFF PERFORMANCE EVALUATION SUPERVISION & COACHING DECISION SUPPORT DATA SYSTEMS INTEGRATED & COMPENSATORY FACILITATIVE ADMINISTRATIVE SUPPORTS PRESERVICE & INSERVICE TRAINING RECRUITMENT AND SELECTION SYSTEMS INTERVENTIONS (c) Dean Fixsen and Karen Blase, 2004

..+Demonstration in Training …+ Practice & Feedback in Training Coaching Impact  OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% …+ Practice & Feedback in Training 60% …+ Coaching in Classroom 95% Joyce and Showers, 2002 (c) Dean Fixsen and Karen Blase, 2004

Stages of Implementation Implementation occurs in stages: Exploration Installation Initial Implementation Full Implementation Innovation Sustainability 2 – 4 Years Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 (c) Dean Fixsen and Karen Blase, 2004

Stages of Implementation Implementation is not an event A mission-oriented process involving multiple decisions, actions, and corrections Implementing an evidence-based program takes 2 to 4 years (c) Dean Fixsen and Karen Blase, 2004

Organizational Change: Top to Bottom Facilitate or Hinder Settings Practitioners State policies. coordination, funding, support Federal policies, coordination, funding, supports System of care Active alignment of policies and coordination of efforts in support of practitioner’s use of effective practices to benefit children, families, and caregivers (c) Dean Fixsen and Karen Blase, 2004

Policy - Practice - Feedback Policy (Plan) Structure Procedure Policy Practice Feedback Study - Act Practice (Do) Form follows Function (c) Dean Fixsen and Karen Blase, 2004

Organizational Change Successful and sustainable implementation of evidence-based practices and programs always requires organizational change at multiple levels. Changing the behavior of adult human service professionals Changing organizational structures, cultures, and climates Changing the thinking of system directors and policy makers (c) Dean Fixsen and Karen Blase, 2004

Getting From Here to There Who is going to do the work of implementation?....at each level? Who will be the “purveyors”? What/who will be the source of implementation drivers? Who will facilitate efficient and effective movement through implementation stages? How can we build the practice and science of implementation? (c) Dean Fixsen and Karen Blase, 2004

Summary We need to pay attention to implementation as a separate set of issues, strategies and procedures. We need to know more about the science of implementation. Implementation likely requires: Knowledgeable “purveyors” Implementation Drivers to support practitioner and organizational change (Select, Train, Coach, Evaluate, etc) Matching implementation activities to Stages Multi-level organizational change that creates alignment to support and sustain new ways of work (Federal, State, Regional, Local, Organizational) (c) Dean Fixsen and Karen Blase, 2004

Don’t mistake a clear view for a short distance. Grand Canyon Hiking Advice (c) Dean Fixsen and Karen Blase, 2004

Thank You We thank the following for their support W. T. Grant Foundation (implementation literature review grant) Substance Abuse and Mental Health Administration (ORC Macro sub-contract for SOC implementation analyses; Implementation Strategies Grant, Mental Health Services Gap Grant) National Institute of Mental Health (research and training grants) Juvenile Justice and Delinquency Prevention (program development and evaluation grants) (c) Dean Fixsen and Karen Blase, 2004

For More Information Dean L. Fixsen, Ph.D. Karen A. Blase, Ph.D. 813-974-4446 dfixsen@fmhi.usf.edu Karen A. Blase, Ph.D. 813-974-4463 kblase@fmhi.usf.edu National Implementation Research Network At the Louis de la Parte Florida Mental Health Institute University of South Florida http://nirn.fmhi.usf.edu (c) Dean Fixsen and Karen Blase, 2004