Installing Alternative Response in Ohio Using Implementation Drivers for Systems Change Allison Metz, PhD Karen Blase, PhD Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina at Chapel Hill Ohio Alternative Response Summit Workshop on Implementation Drivers May 14, 2010 (c) Dean Fixsen and Karen Blase, 2004

Common Goal The Ohio Department of Job and Family Services [in collaboration with several entities] will develop a statewide child protection system with two pathways: an investigation pathway and an alternative response pathway – that assesses and responds to unique safety concerns, risks, and protective capacities of each family who is subject of an accepted report of child maltreatment.

Linking the “What” AND “How” To successfully implement and sustain a new intervention, you need to understand: The WHAT - What is the intervention (e.g. Evidence-based practice, “best practice” initiative, broad system change initiative) AND The HOW - Effective implementation and sustainability frameworks (e.g. strategies to change and maintain behavior of adults)

Defining your Intervention From what current state to what future state? The “intervention” must be operationalized whether it is: An evidence-based practice or program A best practice Initiative A broad systems change initiative Operationalize Part of Speech:  verb Definition:  to define a concept or variable so that it can be measured or expressed quantitatively Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7) Copyright © 2003-2008 Lexico Publishing Group, LLC

OH Alternative Response Model Principles All child protective services (CPS) interventions Alternative response specific Core Elements E.g., Two or more discrete pathways; screening decisions; assessments are child-centered and family focused (c) Dean Fixsen and Karen Blase, 2004

OH Alternative Response Model Foundational Tenets of the Practice Model Alternative response, families, services Workers’ Skill Sets Characteristics, competencies, values Selected Indicators of Quality Practices with Families Behaviors indicative of fidelity to the Alternative Response model (c) Dean Fixsen and Karen Blase, 2004

Reflection Work – Operationalizing. . . Take a moment to consider. . . . When the Alternative Response Model has been developed and implemented as intended – What will child welfare staff be saying and doing more often? . . . Less often? What will child welfare supervisors, managers, and leaders be saying and doing more often? . . . Less often? What will partner agency personnel be saying and doing more often? . . . Less often? Which child welfare activities and processes will be affected? (c) Dean Fixsen and Karen Blase, 2004

Putting “It” Into Action Who will do what differently? Who will help change occur? AND How will change happen? At all levels…. Practice Program Organization System (c) Dean Fixsen and Karen Blase, 2004

Work of Implementation “Systems don’t change - people do.” Jim Wotring, Georgetown University Changing the thinking and behavior of: Adult human service professionals (frontline practitioners) so that positive outcomes are more frequently created for children and families. Administrators in organizations (Supervisor, Managers, Directors) so that structures, cultures, and climates change to support new ways of work System directors, policy makers, and funders (State Agency, Family Courts) so that implementation is supported and outcomes can be achieved (c) Dean Fixsen and Karen Blase, 2008

The Work of Implementation Changing the behavior of adult human service professionals is hard work It requires a systematic approach to support behavior change of practitioners, volunteers, administrators, and policy makers It requires the application of Implementation Drivers [ © Karen A. Blase & Dean L. Fixsen, 2004 (c) Dean Fixsen and Karen Blase, 2008

What are the Implementation Drivers? Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention with fidelity and benefits to consumers. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention Leadership Drivers are methods to manage Technical problems where there is high levels of agreement about problems and high levels of certainty about solutions and to constructively deal with Adaptive challenges where problems are not clear and solutions are elusive

Improved outcomes for children and families Implementation Drivers Performance Assessment Coaching Systems Intervention Facilitative Administration Training Adaptive Competency Drivers Organization Drivers Integrated & Compensatory Decision Support Data System Selection Technical Leadership Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 12

Why These Drivers Focus First on Practitioner’s Behavior? In child welfare, the Practitioner IS THE INTERVENTION Build supports in relationship to what the Practitioner needs to be competent Create the conditions under which practitioners can do the right thing for the right reason at the right time to maximize positive results Wide ranging inputs (individuals with all their past history and current realities) (c) Dean Fixsen and Karen Blase, 2004

Improved outcomes for children and families Implementation Drivers Performance Assessment Coaching Training Competency Drivers Implementation Lens Selection Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 14

Recruitment and Selection Purposes: Select for the “unteachables” Screen for pre-requisites Make expectations explicit Allow for mutual selection through interview process Improve likelihood of retention after “investment” Improve likelihood that training, coaching and supervision will result in implementation (c) Dean Fixsen and Karen Blase, 2004

Recruitment and Selection Implementation Best Practices: Job or role description clarity about accountability and expectations Sampling of skills and experience is related to “new practices” and expectations Interactive Interview Process Skilled interviewers Using Data for Integration and Compensatory Features Feed interview information forward to trainers, coaches, administrators (c) Dean Fixsen and Karen Blase, 2004

Recruitment and Selection Driver in Early Stages of Implementation Identify selection criteria for different levels of staff and roles Select and recruit participants for Implementation Work Groups Assess capacity to recruit and select appropriate staff to implement planned systems reforms Assess staff readiness and commitment Identify possible data sources and processes that will tell you how well the Selection Driver is currently functioning in your State or County agency. (c) Dean Fixsen and Karen Blase, 2004

Pre-Service and In-Service Training Purposes: Knowledge acquisition Basic Skill Development “Buy-in” Implementation Best Practices: Theory grounded (adult learning) Skill-based Behavior Rehearsals vs. Role Plays Knowledgeable Feedback Providers Practice to Criteria Feedback to Selection and Feed Forward to Supervision Data-based (pre and post testing) (c) Dean Fixsen and Karen Blase, 2004

Training Driver in the Early Stages of Implementation Identify your populations of concern Whose capacity are you building?...to do what? Whose knowledge skills and abilities are you attempting to impact? Identify and assess training requirements and needed resources Assess capacity to train staff effectively at multiple levels of the system to implement and sustain change initiative (c) Dean Fixsen and Karen Blase, 2004

Training Driver in the Early Stages of Implementation Identify training requirements and available training resources Assess how current child welfare training practices in your state or county currently reflect implementation best practices? Identify possible data sources and processes that will tell you how well the Training Driver is currently functioning in your State or County agency. (c) Dean Fixsen and Karen Blase, 2004

Supervision and Coaching Purposes: Ensure implementation Develop good judgment Ensure fidelity Provide feedback to selection and training processes (c) Dean Fixsen and Karen Blase, 2004

Coaching Driver OUTCOMES 10% 5% 0% 30% 20% 60% 95% Knowledge % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% …+ Practice & Feedback in Training 60% …+ Coaching in Classroom 95% (c) Dean Fixsen and Karen Blase, 2004

Coaching Implementation Best Practices: Design a Coaching Service Delivery Plan Develop accountability structures for Coaching – Coach the Coach! Regular satisfaction feedback from employees and volunteers Regular review of adherence to Coaching Service Delivery Plan Look at data – Fidelity, Staff Satisfaction with Support, Skill Acquisition (c) Dean Fixsen and Karen Blase, 2004

Coaching Driver in Early Stages of Implementation Ensure that Alternative Response has been fully operationalized and practice profiles completed Assess levels of buy-in for executing coaching plan in agency Assess coaching requirements and identify available resources Assess capacity to coach staff effectively at multiple levels of the system to implement change initiative Identify possible data sources and processes that will tell you how well the Coaching Driver is currently functioning in your State or County agency. (c) Dean Fixsen and Karen Blase, 2004

Improved outcomes for children and families Implementation Drivers Performance Assessment Coaching Training Competency Drivers Implementation Lens Selection Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 25

Performance Assessment Purposes: Ensure implementation Reinforce staff and build on strengths Develop skills and abilities Measure fidelity Interpret Outcome Data Feedback to Agency, County, State Implementation Teams on functioning of Recruitment and Selection Practices Training Programs (pre and in-service) Supervision and Coaching Systems (c) Dean Fixsen and Karen Blase, 2004

Performance Assessment Implementation Best Practices: Transparent Processes – Orientation What, When, How, Why Use of Multiple Data Sources Content Competency Context Tied to positive recognition – not used ‘punitively’ (c) Dean Fixsen and Karen Blase, 2004

Definition of Fidelity The degree to which the program or practice is implemented ‘as intended’ as defined by the program developers/researchers or experts Adherence Integrity Delivered in a “comparable” manner SO THAT it is more likely that comparable outcomes will be more consistently achieved

Competency Implementation Drivers Analysis and Discussion Template Does Driver exist? (yes/no) Responsibility for Driver? (who, what dept/entity) How does the Driver currently support implementation? How well integrated is the Driver with the other Drivers? Is there a measure of Driver effectiveness? How might you know the Driver is effective? How can the Driver be used more purposefully to support fidelity, outcomes, and/or sustainability? Staff Selection Staff Training Staff Coaching Staff Performance Evaluation (Fidelity) (c) Dean Fixsen and Karen Blase, 2004

Sobering Observations "All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get." R. Spencer Darling Business Expert “The tyranny of the status quo.” Fritz Oser Educator © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 Horner, George Sugai, 2008 (c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008 30 30

Improved outcomes for children and families Implementation Drivers Performance Assessment Coaching Systems Intervention Facilitative Administration Training Competency Drivers Organization Drivers Decision Support Data System Selection Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 31

Decision Support Data Systems Purposes: Provide information to assess effectiveness of Alternative Response Model To guide further program and practice development Celebrate success Engage in continuous quality improvement Be accountable for quality infrastructure (are Drivers ‘working’) and for outcomes (c) Dean Fixsen and Karen Blase, 2004

Decision Support Data Systems Implementation Best Practices: Includes intermediate and longer term outcome measures Includes process measures (fidelity) Measures are “socially important” Useful data are: Reliable (standardized protocols, trained data gatherers) Reported frequently (e.g. weekly, quarterly) Reported at relevant and “actionable” levels (e.g. child, family, agency) Widely shared Practical to collect Useful for and used for making decisions (PDSA) (c) Dean Fixsen and Karen Blase, 2004

Decision Support Data Systems in Early Stages of Implementation Assess data collection requirements including technology and training needs Assess technology capacity and integration issues Identify resources for assessing fidelity Identify possible data sources and processes that will tell you how well the Decision Support Data Systems Driver is currently functioning in your State or County agency. Make plans for incorporating data into program improvement cycles (c) Dean Fixsen and Karen Blase, 2004

Facilitative Administrative No such thing as a purely administrative decision!!....They are all decisions about Quality Practice with Children and Families!! Purposes: Facilitates moving through implementation stages Ensures effective use of Competency Drivers Takes the lead on Systems Interventions Utilizes data for improvement Looks for ways to make work of practitioners easier and more effective!! (c) Dean Fixsen and Karen Blase, 2004

Facilitative Administrative Implementation Best Practices: Forms and supports multi-level, functional implementation teams (e.g. service area, cross-agency, senior management) Uses feedback to make changes in Implementation Drivers Revises policies and procedures to support the new way of work Solicits and uses feedback from child welfare staff Reduces administrative barriers (c) Dean Fixsen and Karen Blase, 2004

Facilitative Administration in Early Stages of Implementation Assess policies and procedures needed to support new way of work Provide leadership and support Identify champions Create communication plans Support teaming approach; share decision-making Create buy-in for business NOT as usual - Take risks! Identify possible data sources and processes that will tell you how well the Facilitative Administration Driver is currently functioning in your State or County agency. (c) Dean Fixsen and Karen Blase, 2004

Systems Intervention Purposes: Identify barriers and facilitators for the new way of work Create a “hospitable” environment for the new way of work Contribute to cumulative learning in multi-site projects. (c) Dean Fixsen and Karen Blase, 2004

Systems Intervention Implementation Best Practices Match leadership level needed to intervene Engage and grow “champions” and “opinion leaders” Objectively document barriers Establishes formal PEP – PIP cycles Uses Transformation Zones to Identify Systems Issues Create time-limited, barrier busting processes Make constructive recommendations and assist in implementing and evaluating them (PDSA) (c) Dean Fixsen and Karen Blase, 2004

Systems Interventions Driver in Early Stages of Implementation Identify systems barriers and facilitators Create and strengthen necessary external partnerships Set up policy to practice to policy to practice (4P) feedback loops Identify transformation zones Engage and grow systems champions Identify possible data sources and processes that will tell you how well the Systems Intervention Driver is currently functioning in your State or County agency. (c) Dean Fixsen and Karen Blase, 2004

Organizational Implementation Drivers Analysis and Discussion Template Does Driver exist? (yes/no) Responsibility for Driver? (who, what dept/entity) How does the Driver currently support implementation? How well integrated is the Driver with the other Drivers? Is there a measure of Driver effectiveness? How might you know the Driver is effective? How can the Driver be used more purposefully to support fidelity, outcomes, and/or sustainability? Systems Interventions Facilitative Administration Decision Support Data Systems (c) Dean Fixsen and Karen Blase, 2004

Prevention and Intervention Benefits for Individuals and Community Implementation Drivers Performance Assessment Coaching Systems Intervention Facilitative Administration Training Adaptive Competency Drivers Organization Drivers Decision Support Data System Selection Technical Leadership Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 42

Types of Leadership Needed Different strategies for different challenges Technical Leadership Adaptive Leadership According to Ron Heifitz and his colleagues, one of the biggest mistakes “leaders” make is to incorrectly identify the type of challenge they are facing Using technical approaches for adaptive issues (c) Dean Fixsen and Karen Blase, 2004

Technical Work Perspectives are aligned (views, values) Definition of the problem is clear Solution and implementation of the solution is clear Primary locus of responsibility for organizing the work is the leader (c) Dean Fixsen and Karen Blase, 2004

Adaptive Work Legitimate, yet competing, perspectives emerge Definition of the problem is unclear Solution and implementation is unclear and requires learning Primary locus of responsibility is not the leader (c) Dean Fixsen and Karen Blase, 2004

The Adaptive Work of the Leader Get on the balcony Identify the adaptive challenge Regulate distress Maintain disciplined attention Give the work back to the people Protect all voices R. Heifetz and D. Laurie: The Work of Leadership. Harvard Business Review, 1998.

Improved outcomes for children and families Implementation Drivers Performance Assessment Coaching Systems Intervention Facilitative Administration Training Adaptive Competency Drivers Organization Drivers Integrated & Compensatory Decision Support Data System Selection Technical Leadership Graphics by Steve Goodman,2009 © Fixsen & Blase, 2008 47

Integrated and Compensatory Consistency in philosophy, goals, knowledge and skills across these processes Compensatory Assessment of weaknesses and strengths in Driver functioning Installation of Drivers at multiple levels of the system (practitioner, agency, county, state) (c) Dean Fixsen and Karen Blase, 2008

Are you ready to get a bit more complicated Are you ready to get a bit more complicated? Applying the Drivers to a Driver Example: Creating a “system” for competent trainers for Alternative Reponse Model in OH How to Select them? How to train them? How to coach them? How and what to measure to see if fidelity to the training model, content and processes over time? What training outcomes should we measure? What internal and external systems changes will be needed for trainers to be supported and effective? (c) Dean Fixsen and Karen Blase, 2004

Innovation Fluency X Drivers How much do the experts know about what it takes to implement Alternative Response in typical settings? What will they contribute related to Drivers and their “innovation”? Knowledge? Theory? Data? Recommendations based on implementation in typical settings Tools, training, monitoring systems? (c) Dean Fixsen and Karen Blase, 2004

Applications Implementation Drivers INTRODUCING DRIVERS TO SUPPORT YOUR CHANGE INITIATIVE: Discuss how you will introduce Drivers to your county workgroups or implementation teams. What would you need to do to prepare for such a conversation? DRIVERS AND DATA: Consider Alternative Response model. List the data sources for assessing each driver and discuss a yearly process for evaluating the quality and integration of the Drivers. BEST PRACTICES FOR DRIVERS: Consider an innovation that your state is currently implementing. Using the Drivers Checklist, assess the current implementation practice against the best practices listed on the handout for one “key position” (e.g. family group conferencing facilitator). Given your assessment, what are your next steps. If you don’t have information, what are your sources of information?   (c) Dean Fixsen and Karen Blase, 2004

For More Information Allison Metz, Ph.D. 919-218- 7540 Allison.metz@unc.edu At the Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC www.scalingup.org http://nirn.fpg.unc.edu/ http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/  (c) Dean Fixsen and Karen Blase, 2004

For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/  To order the monograph go to: https://fmhi.pro-copy.com/  (c) Dean Fixsen and Karen Blase, 2004