Getting Ready for Evidence-Based Practice in Schools: Exploration Community Partnerships Getting Ready for Evidence-Based Practice in Schools: Exploration with Scott Caldwell, MA, MS, CASC 3rd Annual Growing School Mental Health Summit, June 22, 2018 Motivational Interviewing, August 2011 1

Presentation Overview Opening activity Definitions Typical approach to implementation Exploration: A new approach for success Bottom line

Opening Activity Why address student behavioral health in the school setting? Why consider evidence-based practice to accomplish this?

The National Implementation Research Network http://nirn.fpg.unc.edu/ Fixsen, Naoom, Blase, Friedman, & Wallace (2005).

What is Evidence-Based Practice (EBP)? Specified components, procedures, skills Key concepts/theory Fidelity standards (delivery as intended) Presence can be detected by trained observers EBP is the technology of the field. Fixsen et al. (2005).

What is Implementation? A process through which staff… learn to skillfully deliver the EBP as intended with integration into routine practice. Fidelity Integration Fixsen et al. (2005).

Integration into Routine Practice Low High Low Fidelity Good outcomes High Fixsen et al. (2005).

EBP delivered with fidelity Integration into routine practice Formula for success… EBP delivered with fidelity Integration into routine practice Good outcomes symptoms frequency remission completion satisfaction Staff learn new ways of working; make changes to practice. Fixsen et al. (2005).

Success is difficult to achieve… Institute of Medicine (2001): “crossing the quality chasm” Neal et al. (2015): “brokering the research practice gap” Stirman et al. (2016): “bridging the gap” Burns et al. (2016): 1-3% of behavioral health provider organizations successfully implement EBPs

A typical approach to implementation: Staff champion learns about an EBP, upcoming training Goes to leader, gets approval for training Colleagues are informed about the upcoming training Staff attend training Leader to staff: just do it. Staff to leader: we got it!

The “train and hope” approach…

Meta-analysis of training Joyce & Showers (2002).

Skillful Practice (Fidelity) Integration into Practice Meta-analysis of training Training Component Learning Outcomes Knowledge of Content Skillful Practice (Fidelity) Integration into Practice Presentation & Lecture 10% 5% 0% + Demonstration 30% 20% + Practice 60% + Coaching (after training) 95% Joyce & Showers (2002).

EBP delivered with fidelity Integration into routine practice Back to the formula for success… EBP delivered with fidelity Integration into routine practice Poor outcomes Students cannot benefit from an intervention they do not receive. Fixsen et al. (2005).

A new approach to implementation: Process with stage-specific tasks Exploration Installation Initial Implementation Full Implementation Fixsen et al. (2005).

A new approach to implementation: Process with stage-specific tasks Exploration Adaptive Leadership The Implementation Team Staff Readiness Fixsen, Ward, & Sims (2014).

Adaptive Leadership Tasks Get on the balcony Identify the initial adaptive challenges Regulate staff distress about change Maintain disciplined focus Give the work back to staff Protect voices from below Heifetz & Laurie (1997); Northouse (2016).

Adaptive Leadership Tasks Get on the balcony Identify the initial adaptive challenges Regulate staff distress about change Maintain disciplined focus Give the work back to staff Protect voices from below Heifetz & Laurie (1997); Northouse (2016).

Adaptive Leadership Tasks Get on the balcony Why address student behavioral health? Examine student need (data) Develop vision for services: universal, selected, targeted approach? prevention/intervention?

Adaptive Leadership Tasks Get on the balcony Identify the initial adaptive challenges Initiative fatigue Staff perceive EBP as “one more thing to do” OR… Staff say they’re already doing it! (reliance on self-report versus direct observation) The work of implementation isn’t the workshop

Adaptive Leadership Tasks Get on the balcony Identify the initial adaptive challenges -- Give the work back to staff Create and empower a team to engage the work of implementation

The Implementation Team Purpose Do the work of implementation at the organizational level; help the organization get ready for change. Fixsen et al. (2005); Higgins, Weiner, & Young (2012).

The Implementation Team Purpose Membership At least 3-5 core members Dedicated time Basic understanding of implementation science Each individual has some degree of influence across multiple levels

The Implementation Team Purpose Membership Tasks Identify EBP options Gather information about each Systematic exploration with Hexagon Tool

The Hexagon Tool Six domains to explore NEED CAPACITY FIT READINESS RESOURCES EVIDENCE Six domains to explore

The Implementation Team Purpose Membership Tasks Identify EBP options Gather information about each Systematic exploration with Hexagon Tool Share results with leadership; facilitate discussion with staff

Implementation Teams Matter Exploration Approach (N = 376 orgs) Two year outcome % implementation success # months to achieve Leader Edict (just do it) 35% 15 Leader Persuasion (why you should do it) 47% 21 Implementation Team 73% 16 Nutt (2001) cited in Fixsen et al. (2013).

Implementation Teams Matter Exploration Approach Outcome % implementation success # years to achieve Without Implementation Team in place (let EBP happen) 14% 17 With Implementation Team in place (assure EBP happens) 80% 3 Fixsen et al. (2005).

Fixsen, Blase, Horner, Sims, & Sugai (2013). Staff Readiness Purpose Getting staff ready for new ways of working and making changes to practice is critical for later success in an EBP initiative. Fixsen, Blase, Horner, Sims, & Sugai (2013).

Staff Readiness Purpose Strategies identified by session participants & presenter: Leader communicates the why of EBP implementation based on student need (the data) and the evidence of EBP effectiveness to address need Include staff in the EBP selection process Connect EBP components with staff values and how staff are already working Make clear the requirements for delivery (fidelity standards)—“truth in advertising” Normalize ambivalence Staff self-select as first implementers Ensure that staff will have protected time for initial service delivery and ongoing learning

Getting Ready for EBP: Exploration Staff champion learns about an EBP, upcoming training; asks leader to approve Leader gets on the balcony; why is addressing student behavioral health important? Forms an Implementation Team to do exploration work With Team, leader shares and discusses exploration results with staff Assesses staff readiness; makes decision to move forward (or not!)

Romney, Israel, & Zlatevski (2014). Exploration Matters Exploration Approach (N = 6 orgs) One year outcome % implementation success cost ($) per staff Did Not Engage Exploration 13% $7,811 Did Engage Exploration 65% $1,052 Romney, Israel, & Zlatevski (2014).

Romney, Israel, & Zlatevski (2014). Exploration Matters “…these findings indicate a striking cost savings for sites completing the readiness process and support the thoughtful application of readiness procedures in the early stages of an implementation initiative.” Romney, Israel, & Zlatevski (2014).

Adaptive Leadership Implementation Team Staff Readiness Bottom line: Exploration Adaptive Leadership Implementation Team Staff Readiness Bottom line: Why is Exploration important? How are you now thinking a little differently about EBP implementation? If you’re considering an EBP initiative, what might be a next step to take?

References & Resources What is evidence-based practice, implementation, formula for success… The National Implementation Research Network: https://nirn.fpg.unc.edu/ Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida. Access from http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf Implementation success is difficult to achieve… Burns, E. J., Kerns, S. E. U., Pullmann, M. D., Hensley, S. W., Letterman, T., & Hoagwood, K. (2016). Research, data, and evidence-based treatment use in state behavioral health systems, 2001-2012. Psychiatric Services, 67(5), 496-503. Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academy Press. Neal, J. W., Neal, Z. P., Kornbluh, M., Mills, K. J., & Lawlor, J. A. (2015). Brokering the research-practice gap: A typology. American Journal of Community Psychology, 56(3-4), 422-435. Stirman, S. W., Gutner, C. A., Langdon, K., & Graham, J. R. (2016). Bridging the gap between research and practice in mental health service settings: An overview of developments in implementation theory and research. Behavior Therapy, 47(6), 920-936. Train and hope? Training meta-analysis… Joyce, B. R., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision & Curriculum Development.  Exploration: Adaptive Leadership Heifetz, R. A., & Laurie, D. L. (1997). The work of leadership. Harvard Business Review (January-February), 124-134. Northouse, P. G. (2016). Adaptive leadership [Chapter 11, pp. 257-294]. Leadership: Theory and practice (7th ed.). Los Angeles, CA: Sage Publications. Exploration: Implementation Teams Hexagon Tool: https://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/NIRN-Education-TheHexagonTool.pdf Higgins, M. C., Weiner, J., & Young, L. (2012). Implementation teams: A new lever for organizational change. Journal of Organizational Behavior, 33, 366-388. Fixsen, D. L., Ward, C., & Sims, B. (2014). Exploration stage [Scaling-up Brief, No. 4]. Chapel Hill, NC: National Implementation Research Network. Access from https://nirn.fpg.unc.edu/resources/exploration-stage Romney, S., Israel, N., & Zlatevski, D. (2014). Effect of exploration‐stage implementation variation on the cost effectiveness of an evidence‐based parenting program. Zeitschrift für Psychologie, 222(1), 37-48. Exploration: Staff Readiness Fixsen, D. L., Blase, K., Horner, R., Ward, C., & Sugai, G. (2013). Readiness for change [Scaling-up Brief, No. 3]. Chapel Hill, NC: National Implementation Research Network. Access from https://files.eric.ed.gov/fulltext/ED507442.pdf Exploration Matters The National Implementation Research Network: https://implementation.fpg.unc.edu/module-3/topic-2 Nutt, P. C. (2001). Contingency approaches applied to the implementation of strategic decisions. International Journal of Business, 6(1), 53‐84.

Thank you. scott.caldwell@wisconsin.gov