1. Maximizing the Use of Implementation Science within Your System
Erin Loften, Improvement Planning Unit and
Kim Watchorn, Office of Learning Supports (OLS)
November 3, 2016

2. CDE OLS
8/13/2015
Outcomes
Learners… …will build awareness regarding Implementation Science. … will reflect on their own systems. …will build knowledge and/or fluency with resources. ….will consider mechanisms for improving local systems-level implementation. …will identify at least one actionable step.
Read and review.
“more of a brief transition slide”
(read quickly on your own)

3. Background
Please raise your hand and give us a fist to five to gain a little background information on
Unified Improvement Planning:
Fist- I think I know what UIP stands for
Five- I’ve been leading this process and writing plans for years
Multi-Tiered System of Supports:
Fist- Oh, they have combined the names from RTI and PBIS
Five- I could teach a class on this topic-or maybe I have…

4. Consider this barge…How “easy” is it to move?
CDE Webinar
Consider this barge…How “easy” is it to move?
This is an image of a barge…
Implementation is not “a quick shift” or a “quick change” or a “quick turn”
(transition type slide)
Substantive systems change is a process.

5. Considerations

6. CDE OLS
8/13/2015
Some have used the language of “first & second” order change to think of the substantive transitions that need to be made within educational reform.

7. CDE OLS MTSS Implementation Workshop
Look at this visual.
Can you imagine how these “gaps” could lead to these outcomes”?
Are all of these elements essential?
How can we assure change?
[We often discuss any/all of these within MTSS implementation…think strategically about your Action Plan…otherwise that “false start” or treadmill effect.]

8. Design matters…
"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get."
-R. Spencer Darling
Business Expert
Prompt: Consider the design of your organization/system…how is its “design” correlated to/connected to/impacting its outcomes?
Are you effectively ‘managing complex change’ (per slide 6)…or are you ‘on a treadmill’?
(c) Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008
Horner, George Sugai, 2008 8

9. Systems Thinking CDE MTSS Webinar
Our goal here is to provide the “discipline literacy” of “Systems Thinking”.
(Note: FYI…connection – in Colorado, Standards provide opportunities for rigor, relevance, and discipline literacy. “thinking like a writer, historian, etc.” With MTSS implementation, systems thinking is an emphasis.)

10. CDE MTSS Webinar
Focus on the System
To get to “all”, we must pay attention to “every”. We must pay attention to the “system” first, and then, we move to small groups and individuals. -Dave Tilly
This quotation from Dave Tilly summarizes what we have been highlighting.
It is most efficient and effective to be thinking about how we support the system and then can provide additional supports. Focus first on the universal. We say “every” (each and every student) to ensure we don’t mean “most.” Sometimes, when people say “all,” they imply “most” – not attending fully to the needs of ‘every’.
If we are intentional and develop an organized way of addressing every students’ needs, we are more likely to impact more students positively and improve student outcomes.

11. CDE MTSS Webinar
Systems Thinking
Systems thinking is an approach to organizational improvement based on awareness of the whole, the part, and the interactions between the two.
Image: MS ClipArt
We start with Systems thinking.
It’s about the whole & parts.
Adaptation of ideas expressed in Asayesh, G (1993, Fall). Using systems thinking to change systems. Journal of Staff Development, 14(4) 8-12.

12. Systems Thinking CDE Webinar
This quotation leads to some of the “how”. We always like to say that this work is not “person-specific” (or individual dependent); MTSS implementation demands certain elements, but it should be more than a solo project or a siloed activity. It is not “about the individual” –it’s not even about the individual STUDENT…it’s about every student. It IS about the processes and the drivers that ensure your system functions effectively to reach desired outcomes.
Systems change is about implementation and sustainability of: processes, protocols, procedures, and practices. “It’s not about the people” (as those can/should/do change). The system ‘sustains’.
It is not “a person” who implements systems change.
(as in it’s not = ex: “Kim, the interventionist, does MTSS at our school.”)
Adaption and expansion of ideas expressed in Asayesh, G (1993, Fall). Using systems thinking to change systems. Journal of Staff Development, 14(4) 8-12.

13. OLS Definition
Systems: Interdependent supports that enable educators to implement a practice with the highest degree of implementation fidelity. Within an MTSS framework, these supports include administrative leadership (policies, staffing patterns, funding), teaming structures (decision making, roles and responsibilities, documentation), use of a problem-solving process, coaching, operating routines, embedded and continuous professional development, and action planning. (CDE OLS, 2014)
Funding
Visibility
Policy

14. CDE OLS
8/13/2015
Systems Thinking
“A system is an entity that maintains its existence through the mutual interaction of its parts…. Systems can be described in terms of their goals, inputs, outputs, processes, and component parts or sub-systems.”
Reflect on the collect capacity of your system.
Possible prompt:
What goals/inputs/outputs/processes/component parts/sub-systems contribute to your systems approach?
From Quality Enhancement Research Initiative
(QUERI) Implementation Guide:

15. CDE OLS
8/13/2015
Common Language

16. CDE OLS
8/13/2015
Implementation Science (National Implementation Research Network – NIRN and State Implementation Scaling up Evidence-Based Practices Center (SISEP)
Primary Sources for content

17. Implementation Defined
CDE OLS
8/13/2015
Implementation Defined
"A specified set of activities designed to put into practice an activity or program of known dimensions" -NIRN
The purpose of implementation is to establish an effective practice with the fidelity and organizational support systems needed for large-scale, efficient, sustained, impact on outcomes that benefit students/families.
-Signetwork Presentation on Stages of Implementation
Getting the best return on our investment of time, resources, staff, money, etc.
Implement EBP and create a sustainable system to do so with fidelity- tell what we are doing, do what we say and measure if we did what we promised.
Do this at various levels: classroom, building, district and BOCES
IMPLEMENTATION
SCIENCE
CLASSROOM
GAP

18. The Golden Circle (Simon Sinek) https://youtu.be/IPYeCltXpxw
CDE OLS
8/13/2015
The Golden Circle (Simon Sinek)
Referencing S.S. “Golden Circle”
Often lots of focus on “what”, need to have clarity/consistency on “why”….
BUT…the “how” can be the implementation science” piece that’s often missing

19. Formula for Success CDE OLS 8/13/2015
Our goal in education is to improve student outcomes with efficiency and effectiveness. This graphic highlights explicitly the “how” to do this. If we utilize effective interventions (the what) and implement them using effective implementation methods (the how), in enabling contexts (the where), we will accomplish socially significant outcomes. (the why)
Education isn’t about exploration; We need to utilize an organized and intentional approach to plan and provide the learning for students.
An important note is Creating and sustaining an hospitable environment [the “where”] is necessary to “grow” whatever we hope will flourish; in this case, we’re talking about MTSS implementation. So, our culture, climate, and collaboration need to facilitate the process of implementation, not take away from it…and not be a barrier to progress.

20. CDE OLS
8/13/2015
Definition of Terms
Implementation practices include methods and procedures used by implementation agents (trainers, coaches, instructors, supervisors, etc.) to promote interventionists’ use of evidence-based intervention practices.
Intervention practices include methods and strategies used by intervention agents (teachers, therapists, clinicians, parents, etc.) to effect changes or produce desired outcomes in a targeted population or group of recipients (e.g., infants and toddlers with disabilities).
Clarity and distinctions
“which practices do we focus on”? (think of the adults)
Adapted from:
Trivette, 2012 (

21. Active Implementation Frameworks
Full set of icons with headers
Teams, stages, drivers, cycles, measures-for EBP
Usable Interventions= clearly described, essential functions defined, functions operationalized, assessment
-Blase & Fixsen

22. Stages of Implementation (Goodman—Adapted from Fixsen)
Focus
Stage
Description
Exploration/Adoption
Decision regarding commitment to adopting the program/practices and supporting successful implementation.
Installation
Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan.
Initial Implementation
Try out the practices, work out details, learn and improve before expanding to other contexts.
Elaboration
Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation.
Continuous Improvement/ Regeneration
Make it easier, more efficient. Embed within current practices.
Should we do it!
Work to do it right!
Implementation of an innovation or in our case an MTSS system, is not an event, it’s a process (!) involving multiple levels of decisions, actions, and corrections. And it takes time, thoughtful planning and action and continuous evaluation. This graphic illustrates the stages of implementation that any organization will go through in developing and implementing MTSS.
Look at these stages and think about where your system might be now in relation to MTSS implementation.
Work to do it better!

23. Assessing Stages

24. Stages of Implementation (Goodman—Adapted from Fixsen)
Focus
Stage
Description
Exploration/Adoption
Decision regarding commitment to adopting the program/practices and supporting successful implementation.
Installation
Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan.
Initial Implementation
Try out the practices, work out details, learn and improve before expanding to other contexts.
Elaboration
Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation.
Continuous Improvement/ Regeneration
Make it easier, more efficient. Embed within current practices.
Should we do it!
NIRN
Work to do it right!
Reminder: implementation takes time
(including implementing protocols and systemic change practices, including implementing MTSS, as a whole, or DBPSDM)
Work to do it better!

25. Lessons Learned You don’t get to skip any stages
CDE OLS
8/13/2015
Lessons Learned
You don’t get to skip any stages
Do the “right work” for the stage
Exploration and installation stages are (often) neglected
Always in “exploration” stage with somebody, somewhere
Stages are not necessarily linear; they can be iterative and overlapping
To provide perspective, in consideration of developing an MTSS system, note some of the lessons learned :
You don’t get to skip stages and should do the right work for that stage;
Be aware we sometimes move “too quickly” past exploration and installation stages that may lead to a false-start or poor implementation
We are always in exploration with somebody/somewhere: as new stakeholders come on – we must keep them up-to-date.
Stages aren’t necessarily linear, they are iterative and overlapping.
Reflection: Consider how these stages can apply to any innovation Be intentional. Think through data, think about what to choose; think about if it can be implemented and what steps to take to intentionally plan and implement an MTSS system in your settng.

26. CYCLES Rapid Cycle Problem Solving (immediate & pressing issues)
CDE OLS
8/13/2015
“PDSA”
CYCLES
Rapid Cycle Problem Solving (immediate & pressing issues)
Usability Testing (variation of PDSA; complex)
Practice-Policy Communication Cycles (feedback loops)
This HAS to be cross-walked with MTSS Problem Solving Process (per next slide) and UIP.

27. Integrated and Compensatory
Positive Outcomes for Students
Why:
What:
Effective Educational Practices
Integrated and Compensatory
Staff capacity to support children/families
with the selected practices
Institutional capacity to support teachers & staff in implementing practices with fidelity
Competency Drivers
Organization Drivers
How:
When these core components are in place they provide the support to a successful implementation that will be sustained.
(resource information: NIRN -
Leadership
Capacity to provide direction and vision
© Fixsen & Blase, 2008

28. Improved Educational Outcomes
Consistent Use of
Educational Innovations
Performance Assessment
(Fidelity)
Coaching
Systems
Intervention
Facilitative
Administration
Competency Drivers
Training
Organization Drivers
There are three categories of Implementation Drivers: Competency, Organization, and Leadership
When these core components are in place they provide the support to establish and maintain a successful implementation of an evidence-based practice or other innovation.
Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention to benefit students. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment
Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective educational services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention
Integrated & Compensatory
Decision Support
Data System
Selection
Leadership
Adaptive Technical
© Fixsen & Blase, 2008 28

29. From NIRN (http://nirn.fpg.unc.edu/)
CDE OLS
8/13/2015
From NIRN (
“more of a brief transition slide”
(read quickly on your own)

30. Leadership Drivers Technical leadership (“good management”)
CDE OLS
8/13/2015
Leadership Drivers
Technical leadership (“good management”)
Engaged, quick to recognize & respond to issues that arise, organizes groups to solve problems, & regularly produces desired results
Work in the zone where there is substantial agreement about what needs to be done &reasonable certainty about how to do it
Adaptive leadership: Required in the zone of complexity where there is little agreement and less certainty (large-scale systems reform)
Adapted from NIRN:

31. Leadership Team Coaching in Practice
CDE OLS
8/13/2015
Leadership Team Coaching in Practice
Six Criteria for Assessing “Real Teams” (Richardson, 2010)
Interdependence (collective tasks require working together)
Shared objectives (agreed collectively)
Autonomy (defined areas where they collectively decide)
Reflexivity (meets and reflects on performance and improvement)
Boundedness (clarity about boundaries)
Specified Roles (different roles contributing to collective performance)
Possible prompts?
How would you describe effective teaming? What does it look like/sound like? What are the operations? What procedures reflect authentic collaboration?
Consider the collaboration within your system = are teams functioning as “real teams”? What teaming structures are in place?

32. Organizational Drivers
CDE OLS
8/13/2015
Organizational Drivers
Systems Intervention
Multi-Level Alignment
“Creating & staying connected to champions”
Windows of opportunity and threats to fidelity
Facilitative Administration
Proactive, vigorous, and enthusiastic attention
Eliminate barriers
Decision Support Data System
Assess key aspects of the overall performance of the organization
Provide data to support decision making
Assure continuing implementation of the evidence-based intervention and benefits to consumers over time
Adapted from NIRN:

33. Competency Drivers Selection Training Coaching
CDE OLS
8/13/2015
Competency Drivers
Selection
Of: Practitioners; organization staff (trainers, coaches, evaluators, administrators); & staff for Implementation Teams.
Variables: Workforce development, economy, organizational financing, demands of programming, etc.
Training
Content varies, but methods are less variable.
History, theory, philosophy, and rationales for program components and practices; and Practice (“behavior rehearsals”)
Coaching
Can impact: Knowledge, skill, and craft
Newly-learned behavior is “crude, fragile, and incomplete”
Adapted from NIRN:

34. Coaching
Bruce Joyce and Beverly Showers (2002)

35. Most Effective Adult Learning Method Practices
Source: Trivette, 2012 (

36. Connections
At tables, identify connections or applications you see to:
Unified Improvement Planning
Multi-Tiered System of Supports
Be prepared to share out whole group.

37. Reflection on your Practices…
CDE OLS
8/13/2015
Reflection on your Practices…
What principles/standards of adult learning do you use to frame your learning?
How do you plan and differentiate your adult learning offerings?
What methods of measurement do you use to assure your adult learning opportunities are effective?

38. CDE OLS
8/13/2015
“Availability and integrity of the Implementation Drivers is the functional work of an organization”
…“organizational process as a largely unaddressed barrier and as a potential bridge between research and practice”… (Rosenheck, 2001)
Possible to include as ‘bringing together some ideas’

39. Lessons Learned (from Florida)
CDE OLS
8/13/2015
Lessons Learned (from Florida)
What are some likely reasons that implementation succeeds or fails at either district or school levels?
Failure to achieve consensus
School culture is ignored (practices, values, beliefs must be integrated in implementation plan)
Lack of training and support
Lack of feedback to implementers to support continued implementation
Unrealistic expectation of initial success
Failure to measure and analyze progress
Participants are not valued in planning
From the work done in Florida, here are some lessons to think about in relation to implementation.

40. Active Implementation Hub
CDE OLS
8/13/2015
Active Implementation Hub
Resources available to learn “all about” by using modules (NIRN).

41. CDE OLS
8/13/2015
Implementation Science CDE OLS video
Webpage from CDE OLS re: Imp Sci (more to come!!!)
Now let’s look at the How of MTSS?” It’s nice to know that there’s a way to do this. There’s a ‘science’ to it.

42. CDE MTSS Overview Slides
“new graphic” (as of 2016)
CO-MTSS is a prevention-based framework of team-driven, data-based problem solving for improving the outcomes of every student through family, school, and community partnering and a layered continuum of evidence-based practices applied at the classroom, school, district, region, and state level.

43. Infrastructure Culture
CDE MTSS Webinar
11/28/2018
Seeking Balance
Infrastructure
Culture
MTSS is about:
Collaboration
Systems Alignment
An Organizational Framework
What adults are doing
(infrastructure + culture = balanced)

44. CDE OLS
8/13/2015
Outcomes = what we see
“what’s underneath? behind the scenes? Etc.”
That’s systems work…
(transition type slide)

45. Personal Action Planning
CDE OLS
8/13/2015
Personal Action Planning
Consider the following…
What are the important takeaways from today?
How will this change your practice?
How will this help shape your system?
What data, systems, or practices do you most want to address?
How will your plan of action support student learning outcomes?

46. Action Steps
How might this learning affect your thinking in relation to your local implementation efforts?
What might you consider for changing your practice or your system?
What resources/supports might you need for your next steps?

47. This CDE guidance document is meant for clarification, is not legally binding, and is not to be confused with legal advice. This guidance reflects CDE’s recommendations, but Administrative Units (AUs) may have developed their own policies or procedures that differ from those described herein. Be sure to refer to your local AU’s policies and procedures through the Director of Special Education. If you are seeking legal advice, please contact your legal counsel.
The contents of this handout were developed under a grant from the U.S. Department of Education. However, the content does not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the federal government.
Key Points:
This slide notes the funding source for OLS & ESSU work.