1. Getting Ready for Evidence-Based Practice in Schools: Exploration
Community Partnerships
Getting Ready for Evidence-Based Practice in Schools: Exploration
with Scott Caldwell, MA, MS, CASC
3rd Annual Growing School Mental Health Summit, June 22, 2018
Motivational Interviewing, August 2011 1

2. Presentation Overview
Opening activity
Definitions
Typical approach to implementation
Exploration: A new approach for success
Bottom line

3. Opening Activity
Why address student behavioral health in the school setting?
Why consider evidence-based practice to accomplish this?

4. The National Implementation Research Network
Fixsen, Naoom, Blase, Friedman, & Wallace (2005).

5. What is Evidence-Based Practice (EBP)?
Specified components, procedures, skills
Key concepts/theory
Fidelity standards (delivery as intended)
Presence can be detected by trained observers
EBP is the technology of the field.
Fixsen et al. (2005).

6. What is Implementation?
A process through which staff…
learn to skillfully deliver the EBP as intended
with integration into routine practice.
Fidelity
Integration
Fixsen et al. (2005).

7. Integration into Routine Practice
Low
High
Low
Fidelity
Good outcomes
High
Fixsen et al. (2005).

8. EBP delivered with fidelity Integration into routine practice
Formula for success…
EBP delivered with fidelity
Integration into routine practice
Good outcomes
symptoms
frequency
remission
completion
satisfaction
Staff learn new ways of working; make changes to practice.
Fixsen et al. (2005).

9. Success is difficult to achieve…
Institute of Medicine (2001): “crossing the quality chasm”
Neal et al. (2015): “brokering the research practice gap”
Stirman et al. (2016): “bridging the gap”
Burns et al. (2016): 1-3% of behavioral health provider organizations successfully implement EBPs

10. A typical approach to implementation:
Staff champion learns about an EBP, upcoming training
Goes to leader, gets approval for training
Colleagues are informed about the upcoming training
Staff attend
training
Leader to staff: just do it.
Staff to leader: we got it!

11. The “train and hope” approach…

12. Meta-analysis of training
Joyce & Showers (2002).

13. Skillful Practice (Fidelity) Integration into Practice
Meta-analysis of training
Training
Component
Learning Outcomes
Knowledge of Content
Skillful Practice (Fidelity)
Integration into Practice
Presentation & Lecture
10% 5% 0%
+ Demonstration
30%
20%
+ Practice
60%
+ Coaching (after training)
95%
Joyce & Showers (2002).

14. EBP delivered with fidelity Integration into routine practice
Back to the formula for success…
EBP delivered with fidelity
Integration into routine practice
Poor outcomes
Students cannot benefit from an intervention they do not receive.
Fixsen et al. (2005).

15. A new approach to implementation: Process with stage-specific tasks
Exploration
Installation
Initial Implementation
Full Implementation
Fixsen et al. (2005).

16. A new approach to implementation: Process with stage-specific tasks
Exploration
Adaptive Leadership
The Implementation Team
Staff Readiness
Fixsen, Ward, & Sims (2014).

17. Adaptive Leadership Tasks
Get on the balcony
Identify the initial adaptive challenges
Regulate staff distress about change
Maintain disciplined focus
Give the work back to staff
Protect voices from below
Heifetz & Laurie (1997); Northouse (2016).

18. Adaptive Leadership Tasks
Get on the balcony
Identify the initial adaptive challenges
Regulate staff distress about change
Maintain disciplined focus
Give the work back to staff
Protect voices from below
Heifetz & Laurie (1997); Northouse (2016).

19. Adaptive Leadership Tasks
Get on the balcony
Why address student behavioral health?
Examine student need (data)
Develop vision for services: universal, selected, targeted approach? prevention/intervention?

20. Adaptive Leadership Tasks
Get on the balcony
Identify the initial adaptive challenges
Initiative fatigue
Staff perceive EBP as “one more thing to do”
OR… Staff say they’re already doing it! (reliance on self-report versus direct observation)
The work of implementation isn’t the workshop

21. Adaptive Leadership Tasks
Get on the balcony
Identify the initial adaptive challenges --
Give the work back to staff
Create and empower a team to engage the work of implementation

22. The Implementation Team
Purpose
Do the work of implementation at the organizational level; help the organization get ready for change.
Fixsen et al. (2005); Higgins, Weiner, & Young (2012).

23. The Implementation Team
Purpose
Membership
At least 3-5 core members
Dedicated time
Basic understanding of implementation science
Each individual has some degree of influence across multiple levels

24. The Implementation Team
Purpose
Membership
Tasks
Identify EBP options
Gather information about each
Systematic exploration with Hexagon Tool

25. The Hexagon Tool Six domains to explore NEED CAPACITY FIT READINESS
RESOURCES
EVIDENCE
Six domains to explore

27. The Implementation Team
Purpose
Membership
Tasks
Identify EBP options
Gather information about each
Systematic exploration with Hexagon Tool
Share results with leadership; facilitate discussion with staff

28. Implementation Teams Matter
Exploration Approach
(N = 376 orgs)
Two year outcome
% implementation success
# months to achieve
Leader Edict
(just do it)
35% 15
Leader Persuasion
(why you should do it)
47% 21
Implementation Team
73% 16
Nutt (2001) cited in Fixsen et al. (2013).

29. Implementation Teams Matter
Exploration Approach
Outcome
% implementation success
# years to achieve
Without Implementation Team in place (let EBP happen)
14% 17
With Implementation
Team in place (assure EBP happens)
80% 3
Fixsen et al. (2005).

30. Fixsen, Blase, Horner, Sims, & Sugai (2013).
Staff Readiness
Purpose
Getting staff ready for new ways of working and making changes to practice is critical for later success in an EBP initiative.
Fixsen, Blase, Horner, Sims, & Sugai (2013).

31. Staff Readiness Purpose
Strategies identified by session participants & presenter:
Leader communicates the why of EBP implementation based on student need (the data) and the evidence of EBP effectiveness to address need
Include staff in the EBP selection process
Connect EBP components with staff values and how staff are already working
Make clear the requirements for delivery (fidelity standards)—“truth in advertising”
Normalize ambivalence
Staff self-select as first implementers
Ensure that staff will have protected time for initial service delivery and ongoing learning

32. Getting Ready for EBP: Exploration
Staff champion learns about an EBP, upcoming training; asks leader to approve
Leader gets on the balcony; why is addressing student behavioral health important?
Forms an Implementation Team to do exploration work
With Team, leader shares and discusses exploration results with staff
Assesses staff readiness; makes decision to move forward (or not!)

33. Romney, Israel, & Zlatevski (2014).
Exploration Matters
Exploration Approach
(N = 6 orgs)
One year outcome
% implementation
success
cost ($) per staff
Did Not Engage Exploration
13%
$7,811
Did Engage
Exploration
65%
$1,052
Romney, Israel, & Zlatevski (2014).

34. Romney, Israel, & Zlatevski (2014).
Exploration Matters
“…these findings indicate a striking cost savings for sites completing the readiness process and support the thoughtful application of readiness procedures in the early stages of an implementation initiative.”
Romney, Israel, & Zlatevski (2014).

35. Adaptive Leadership Implementation Team Staff Readiness Bottom line:
Exploration
Adaptive Leadership
Implementation Team
Staff Readiness
Bottom line:
Why is Exploration important?
How are you now thinking a little differently about EBP implementation?
If you’re considering an EBP initiative, what might be a next step to take?

36. References & Resources
What is evidence-based practice, implementation, formula for success…
The National Implementation Research Network:
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida. Access from
Implementation success is difficult to achieve…
Burns, E. J., Kerns, S. E. U., Pullmann, M. D., Hensley, S. W., Letterman, T., & Hoagwood, K. (2016). Research, data, and evidence-based treatment use in state behavioral health systems, Psychiatric Services, 67(5),
Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st century. Washington DC: National Academy Press.
Neal, J. W., Neal, Z. P., Kornbluh, M., Mills, K. J., & Lawlor, J. A. (2015). Brokering the research-practice gap: A typology. American Journal of Community Psychology, 56(3-4),
Stirman, S. W., Gutner, C. A., Langdon, K., & Graham, J. R. (2016). Bridging the gap between research and practice in mental health service settings: An overview of developments in implementation theory and research. Behavior Therapy, 47(6),
Train and hope? Training meta-analysis…
Joyce, B. R., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision & Curriculum Development. 
Exploration: Adaptive Leadership
Heifetz, R. A., & Laurie, D. L. (1997). The work of leadership. Harvard Business Review (January-February),
Northouse, P. G. (2016). Adaptive leadership [Chapter 11, pp ]. Leadership: Theory and practice (7th ed.). Los Angeles, CA: Sage Publications.
Exploration: Implementation Teams
Hexagon Tool:
Higgins, M. C., Weiner, J., & Young, L. (2012). Implementation teams: A new lever for organizational change. Journal of Organizational Behavior, 33,
Fixsen, D. L., Ward, C., & Sims, B. (2014). Exploration stage [Scaling-up Brief, No. 4]. Chapel Hill, NC: National Implementation Research Network. Access from
Romney, S., Israel, N., & Zlatevski, D. (2014). Effect of exploration‐stage implementation variation on the cost effectiveness of an evidence‐based parenting program. Zeitschrift für Psychologie, 222(1),
Exploration: Staff Readiness
Fixsen, D. L., Blase, K., Horner, R., Ward, C., & Sugai, G. (2013). Readiness for change [Scaling-up Brief, No. 3]. Chapel Hill, NC: National Implementation Research Network. Access from
Exploration Matters
The National Implementation Research Network:
Nutt, P. C. (2001). Contingency approaches applied to the implementation of strategic decisions. International Journal of Business, 6(1), 53‐84.

37. Thank you. scott.caldwell@wisconsin.gov