1. An Introduction to Implementation Tools to Help Build Implementation Capacity
SPDG Evaluators May 2012
Michelle A. Duda, Dean L. Fixsen, Karen A. Blase and Barbara Sims
University of North Carolina at Chapel Hill
© 2012 Karen A. Blase and Dean L. Fixsen

2. Introductions Name States you support/work with
Level of the Education “system” you support (State, Region, District, Building)
What you hope to learn
SISEP 2012

3. SISEP Intentions Develop implementation capacity in 6 States
Establish a protocol for selecting States that are ready
Select evidence-based programs ready for scaling
Evaluate the state-level application of the approach
Assess State capacity; improve outcomes for students
Extend findings to additional States
Work with RRC and TA Centers
Develop Communities of Practice
Disseminate findings
Make presentations and write papers
Develop web site and share tools

4. Objectives A deeper review of a selected tool
Introduce tools to help support implementation capacity development
State Capacity Assessment
Stages of Implementation Analysis
Stage-Based Implementation Driver Analysis
A deeper review of a selected tool
a) Explore how it leads to developing Action Plans
b) Discuss Administration and Implications
c) Next Steps
© 2012 Karen A. Blase and Dean L. Fixsen

5. Active Implementation Frameworks
Implementation Teams
Implementation Stages
Implementation Drivers
Improvement Cycles
Briefly Summarize the Four Implementation Frameworks
Set the Context for
Tools and Resources
State and RRC Examples
Table Discussion
© 2012 Karen A. Blase and Dean L. Fixsen

6. Active Implementation Frameworks
Successful implementation on a useful scale requires. . .
Organized, expert assistance – “IMPLEMENTATION TEAMS”
Purposeful matching of critical implementation activities to the stage of the process – “STAGES OF IMPLEMENTATION”
Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS”
A focus on continuous, purposeful improvement – “IMPROVEMENT PROCESSES”
SISEP 2012

7. “Implementation Drivers”
Common features of successful supports to help make full and effective uses of a wide variety of innovations

8. Reliable Student Benefits
Consistent Use of
Educational Innovations
Interventions Meet
Implementation
Implementation Drivers
Performance Assessment (fidelity)
Systems
Intervention
Coaching
Organization Drivers
Facilitative
Administration
Training
Competency Drivers
Integrated & Compensatory
Decision Support
Data System
Selection
Leadership Drivers
Technical
Adaptive
SISEP 2012
© Fixsen & Blase, 2008 8

9. Stage–Based Assessments/ Stage-Based TA
The Exploration Stage
Installation Stage
Initial Implementation
Full Implementation
© 2012 Karen A. Blase and Dean L. Fixsen

10. Stages of Implementation Analysis
Purpose
Help Team plan for and/or assess the use of stage-based activities to improve the success of implementation efforts for EBPs or evidence-informed innovations (action planning/anticipatory guidance)
The tool can be used to Self-Assess current stage related activities (e.g. “We are in the midst of Exploration”) or past efforts related to a stage (e.g. “We just completed most of Installation? How did we do? What did we miss?) (manage expectations)

11. Administration of the Stages of Implementation Analysis
Define the desired function of the tool in advance (assess current status? Action planning?)
For self-assessment Implementation lead or Implementation Team completes entire assessment to achieve “strength of stage” score for each stage of Implementation
For Items “Initiated or Partially in Place” and “Not Yet Initiated” Action Plans can be developed to determine next steps or what needs to be revisited
Usually used during exploration stage, but can be used thorough out all stages to check back or when implementation dips occur (change in leadership, staff turnover, etc.)

12. Using the Implementation Drivers
Interviews and Self-Assessments to analyze the implementation drivers for action planning:
What infrastructure exists?
How implementation-informed is it?
What additional infrastructure is needed?
What are the “next right steps” to improve the infrastructure supports?
Integrated & Compensatory
Competency Drivers
Organization Drivers
Leadership Drivers
© 2012 Karen A. Blase and Dean L. Fixsen

13. Implementation Drivers Tools
High level tool to get State, Districts, or Buildings thinking about
The functions
Who be accountable for them?
How can they be improved to better support implementation?

14. Tools to Help Build Capacity
Stage of Implementation
Assessments
Exploration
Assessment of Implementation Stages
ImpleMap
Installation
Installation Stage Assessment
Installation Stage Action Planning Guide
Initial Implementation
Initial Implementation Component Assessment
Initial Implementation Action Planning Guide
Full Implementation
Full Implementation Component Assessment
Implementation Tracker

15. Exploration: ImpleMap
Purpose:
To assess and understand “the implementation landscape”
A data collection tool to assess Implementation drivers and implementation best practices
Pre-Requisites
Interviewer needs to have expertise in Implementation Science
Site has a history of successful (or not) use of evidence-based or evidence informed practices

16. Administration of the ImpleMap
Semi-structured interview lead by individuals fluent in Implementation Science or has deep knowledge of the Implementation Drivers
Participants include individuals who are immersed in helping building staff use innovations and leadership involved in selecting and sustaining these innovations
Select more than one EBP to discuss so that themes can be identified
SISEP 2012

17. ImpleMap Administration
INTERVENTION
WHAT
HOW
WHO
Enter the name of each intervention provided by the respondent.
You may know the intervention by another name, but record the name used in this provider organization.
Ask questions to get information about the vetting process.
Ask about the “core intervention components” as they are described by the respondent.
Core intervention components are the critical functions that define an intervention.
Ask about the Implementation Drivers.
Implementation Drivers are components related to: ►developing staff competency (selection, training, coaching, performance assessments); ►organization supports (decision support data systems, facilitative administration, systems interventions); and ►leadership supports (technical and adaptive).
Ask about the person accountable for providing each Implementation Driver.
Record the name, position, and physical location of each person.
SISEP 2012

18. Installation & Initial Implementation Stage Driver Component Assessments
Purpose
Serve as a Self-Assessment for Implementation Infrastructure for Evidence-based Practices Identified
Help Teams Create an Implementation Action Plan for installing Driver Best Practice Components
Pre-Requisites
Implementation Team or “Accountability/Leadership” Team that can achieve action items that will be identified
Clearly Defined Evidence-based Practice Components

19. Administration of the Installation Stage Assessment
Facilitated by Implementation Specialist or Member of Implementation Team
May choose to review 1 – 2 Drivers/per meeting
For components identified as “Fully In Place” provide a clear description or “evidence” that this component is transparent
Score components to help prioritize Action Items
Outcome: Implementation Action Plan

20. Scoring key
All Implementation best-practice components scored using the following format.
Fully In Place
2 points
- All dimensions of element adhered to and evidence available to support this
Partially In Place
1 point
- Some dimensions of element adhered to and/or some dimensions attended to
- Action Planning occurs with these elements
Not in Place
0 points
Element not adhered to
Action Planning occurs with these elements or teams may not be developmentally ready to build in this component

21. Installation Stage: Self-Assessment (3 districts)
Information for Action Planning
To what extent are best practices being used?
Academics
Behavior
In Place
Partially in Place
Not in Place
Recruitment and Selection Items 1%
28%
70% 7%
40%
50%
Training Items 3%
39%
57%
21%
40%
38%
Supervision/ Coaching Items 3%
26%
64%
12%
32%
52%
Performance Assessment/Fidelity 0%
33%
61%
15%
43%
40%
Leadership Items 8%
44%
41%
20%
46%
28%
© 2012 Karen A. Blase and Dean L. Fixsen

22. School District #1 Assessment of District Capacity for Implementation Subscale Scores : 2012, March

23. School District #2 Assessment of District Capacity for Implementation Subscale Scores : 2012, January

24. For More Information
Bertram, R.M., Suter, J.C., Bruns, E.J., & O'Rourke, K.C (2010). Building a research agenda: Implementation research and wraparound literature. Journal of Child and Family Studies.20:6.
Ogden, T., Bjørnebekk, G., Kjøbli, J., Patras, J., Christiansen, T., Taraldsen, K., & Tollefsen, N. (2012). Measurement of Implementation components ten years after a nationwide introduction of empiraclly suported programs-a pilot study. Implementation Science. 7:49.
SISEP 2012

25. Benefits of Driver-Based Planning
Infrastructure needed becomes visible to all
Strengths and progress get celebrated
Next right steps are planned and results measured
Resources can be aligned and re-purposed to improve implementation
© 2012 Karen A. Blase and Dean L. Fixsen

26. For more on Implementation Science
Stay Connected!
@SISEPcenter
SISEP
For more on Implementation Science 26

27. Implementation Science
Implementation Research:
A Synthesis of the Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). 27