1. Effective Implementation: Full Implementation

2. NC Implementation Structures
Leadership and Policy
State Implementation Team:
C&V, PD, TAC, RE
Implementation Design Team
Implementation Design Team
Implementation Design Team
Implementation Design Team

3. NC Implementation Structures
Leadership and Policy
State Implementation Team
Implementation Design Team
Implementation Design Team
Implementation Design Team
Implementation Design Team
K-3 Formative Assessment
MTSS
LEASA

4. “Students cannot benefit from interventions they do not experience.”
Rationale
RESEARCH
IMPLEMENTATION
PRACTICE
GAP
Why do we have to pay attention to implementation practice?
“Students cannot benefit from interventions they do not experience.”

5. Formula for Success Implementation Math WHAT: Effective Interventions
HOW:
Effective Implementation
WHERE: Supportive Contexts
WHY:
Positive Outcomes for Students
We know that for implementation to be successful, we must have all three (effective interventions, effective implementation, and supportive contexts) in place to get positive outcomes for students. Just like with multiplication, if any of these three factors are “0” then the outcome will also be “0”

6. Active Implementation Frameworks
Stages
Drivers
Usable Interventions
Cycles
There are five frameworks in the Implementation Science model – let’s review them quickly
Teams

7. Linked Team Structures
“We tend to focus on snapshots of isolated parts of the system and wonder why our deepest problems never seem to get solved.”
—Senge, 1990
School-based
Implementation Team
District-based
Regionally-based
State-based
Decision-makers: Data, Curriculum, Funding, Personnel
To ensure appropriate support, we need to not work just in teams, but in linked teams. A good communication flow between levels of teams leads to effective and efficient implementation by providing a structure for barriers or challenges identified along the way to be pushed up to the team that can problem-solve to remove the barrier

8. Performance Assessment
Usable Interventions
Performance Assessment
Operational
Definitions
Essential Functions
To be truly usable, interventions must be well-defined in ways that implementers can understand what is expected of them. Being able to describe what the intervention should look like in action is critical to high fidelity implementation.
Clear
Description

9. Implementation Drivers
Performance Assessment
(Fidelity)
Coaching
Systems
Intervention
Training
Organization Drivers
Facilitative
Administration
Competency Drivers
Each of the drivers plays a role in ensuring success of implementation. Competency drivers are about making sure implementers have the knowledge and skills to implement as intended, but without organizations that are good host environments, and leadership capable of solving both technical and adaptive challenges, implementation won’t be successful
Selection
Decision Support
Data System
Leadership
Adaptive
Technical
© Fixsen & Blase, 2008 10

10. Improvement Cycles: Usability Testing
Plan
Study
Act Do
Anytime we are trying new things, we can expect unanticipated challenges to arise. If we enter implementation with the mindset that we are engaging in a continuous improvement process, we can be prepared for making changes along to way as the data informs our process. 11

11. Implementation Stages Initial Implementation
Exploration
Installation
Initial Implementation
Full Implementation
Assess needs
Examine intervention components
Consider Implementation Drivers
Assess Fit
Acquire Resources
Prepare Organization
Prepare Implementation Drivers
Prepare Staff
Adjust Implementation Drivers
Manage Change
Deploy Data Systems
Initiate Improvement Cycles
Monitor & Manage Implementation Drivers
Achieve Fidelity & Outcome Benchmarks
Further Improve Fidelity & Outcomes
Finally, Implementation doesn’t occur all at once. By paying close attention to the stages of implementation, we can focus our work, anticipate challenges, and prepare for what’s coming next.
2-4 Years

12. Full Implementation “Work to do it better.”
How are we improving and sustaining?
Full implementation is reached when 50% or more of the staff are using the practice with fidelity and are achieving good outcomes
This is difficult to achieve and sustain without attending to the first three stages
In full implementation, the new practice becomes the way we do and carry out our work. However, it is important that implementation teams ensure that the gains in the use of effective practices are maintained and improved over time. Sustainability requires tenacity. First, you got started, then you got better. Now may be your biggest challenge – maintaining the quality over time and across staff through purposeful use of the Implementation Drivers and Improvement Cycles. The goal is have the use of these frameworks become second nature.
Implementation Teams are built into organization structures and are essential contributors to the ongoing success of using the evidence-based program.  The work of Implementation Teams is to ensure that the gains in the use of effective practices are maintained and improved over time and through transitions of leaders and staff.
A great deal of hard work has gotten you this far.  Celebration is motivation! When it becomes clear from your assessments that staff are implementing with fidelity and students benefitted from these efforts, be sure to spread the word!  Success will encourage everyone involved to maintain the innovation(s).
Questions to Consider:
It’s working fine, but how do others do it?
Is there a way to make it better?
How do we ensure this sustains over time and through staff changes?

13. Full Implementation Key Functions “Work to do it better.”
Skillful practices by all staff
Evaluation for expected outcomes
Full use of implementation drivers
Policy changes/ development for sustainability
Skillful practices by all staff
Fidelity scores signal full implementation. Fidelity measures are identified and/or developed during Exploration and Installation and are used during Initial Implementation to improve the competency of new Implementation Team members as they support teacher and staff learning and use of innovations.
Evaluation for expected outcomes
When 50% of the staff are using the innovation fully and effectively, it is legitimate to anticipate robust student outcomes. At this point, there is no doubt that the innovation is in place and is being used as intended across a district. Full Implementation creates the opportunity to see if the innovations/practices/systems are producing the anticipated outcomes. During this Stage, it is appropriate to analyze the results from the selected or created assessments for student outcomes coupled with implementation fidelity checks. Based on the results of this evaluation process, action plans are created or updated (e.g., reporting to stakeholders, celebrations, re-examination of drivers).
Full use of Implementation Drivers
The Implementation Team will continue to look at each of the Implementation Drivers during this Stage to monitor their effectiveness in “full throttle” mode. Continuous quality improvement of the Implementation Drivers is a hallmark of Full Implementation. Has there been turnover in staff on the Implementation Team? Are new staff members on board who need orientation and training? Are new staff members getting increased coaching compared to staff who are meeting fidelity? Is coaching being provided to all staff? Is coaching contributing to staff satisfaction and improvements in fidelity? Are the data systems providing timely, actionable, reliable, and valid data? Are data regularly used for decision-making? How are the stakeholders reacting to the widespread use of these practices?
Policy changes/development for sustainability
When Full Implementation has been achieved and is being sustained, people sometimes forget that changes in policy and procedures can adversely impact use of the innovation as intended. Throughout the life of the innovation the Implementation Team together with leadership pays attention to the degree to which policies and procedures help or hinder implementation and outcomes. Have you paid attention to the changes needed in policy and procedures with ongoing active use of the improvement cycles? It can take some time to change policies even after the Implementation Team recognizes the need to do so. There may be future innovations which will require the same type of changes in policy as the current one. Now is the time to get comfortable with ‘institutionalizing’ the use of improvement cycles to strengthen the infrastructure so you are ready to go next time!
Share your Success!
A great deal of hard work has gotten you this far. Celebration is motivation! When it becomes clear from your assessments that staff are implementing with fidelity and students benefitted from these efforts, be sure to spread the word! Success will encourage everyone involved to maintain the innovation(s).

14. Full Implementation Example: Cleveland County

15. Overview of Cleveland County Schools
23rd Largest School System
20th Largest in Number of EC Students
63.97% Free and Reduced Lunch
29 Schools (16 elementary, 2 intermediate, 4 middle, 4 high, 1 ECHS, 1 special purpose, 1 alternative and post-school programs)

16. Out-patient Therapy (Individual and Group)
School-Based Services
(PBIS, SOC, BIPs, Additional Adult Support)
Out-patient Therapy (Individual and Group)
(School-based)
Intensive
In-Home Services
Day Treatment
Mobile Crisis Unit
Community Services
Multi-Systemic Therapy
Therapeutic Foster Care

17. Behavior and Mental Health Support Framework

18. The Behavior/Mental health Liaison The Behavioral Interventionist
Establish and maintain rapport with targeted students
and their families
Implement behavioral strategies with targeted students
Assist in overseeing the implementation of BIPs
Provide training for school staff in behavioral
interventions
Model effective behavioral intervention strategies
Assist targeted students in behavioral crisis situations
Assist in implementing services as prescribed by the IEP
Teach replacement behaviors and implement social skill
training programs
Assist in conducting FBA process(e.g. interviews,
observations, identification of triggers, etc.)
Assist with behavioral data collection and recordkeeping
Supervise/train Behavioral Intervention Assistants
Assist with Single Portal process
Work with and collect data on students in the PRC 29
grant
Work with school staff to design and write BIPs
Assist in implementing effective behavioral
interventions
Attend Child and Family and/or IEP team meetings
Provide staff training in areas related to providing
services for at-risk students (e.g. NVCI, PBIS, crisis
resolution, preventive strategies, Behavioral
Interventions, etc.)
Act as liaison between Cleveland County Schools and
outside agencies (e.g. DJJ, Pathways, private providers,
DSS, etc.)

19. EC State and Federal Funds
Local Funds
EC State and Federal Funds
Provider Agency
Medicaid Billing
Disadvantaged Student Supplemental Funds
LME – MCO Contract Funds
IPRS Funds

20. EXPLORATION to choose School-Based Outpatient Therapy Providers

21. REQUEST FOR PROPOSALS (RFP)
1. Drafted RFP and got approval from Superintendent
2. Advertised RFP on our Local Management Entity (LME)/ Partners’ website
3. Invited Partners’ staff to assist in interviewing companies
4. Created scoring rubric to determine the most qualified agencies

22. Memorandum of Agreement (MOA) Request for Proposals (RFP)
Background Checks
Licensure and Liability Insurance
Physical Space and Equipment
On Campus Expectations (Sign-In, Notice of Cancellations, ID Badge, Follow all school and board policies and procedures)
Access to Records and Communication Flow
Agreement to assist in transition of services as need
Request for Proposals (RFP)
School Assignments

23. Connections School Counselors Social Worker
Outpatient Therapy Providers
Alexander Youth Network
Laura Rudisill
Piedmont Family Services
Preferred Health Choice
Connections
School Counselors
Social Worker
Behavior and Mental Health Liaisons
Behavior Intervention Assistants

24. INSTALLATION AND INITIAL IMPLEMENTATION

25. Burns Zone Crest Zone Kings Mountain Zone Shelby Zone
Burns High
Burns Middle
Union, Fallston and Casar
(AYN)
Washington
(Preferred Health Choice)
Crest Zone
Crest High
Crest Middle
Preferred Health Choice
Boiling Springs and Springmore
Township #3
(AlYN)
Kings Mountain Zone
Kings Mountain High
Kings Mountain Middle
Kings Mountain Intermediate
Bethware
(Laura Rudisill)
East, West, North and Grover
Shelby Zone
Shelby High
Shelby Middle
(AYYN)
Shelby Intermediate
Graham, JLE, Jefferson and Marion
County-Wide Programs
Turning Point Academy
Early College High School
North Shelby School
Students at Project SEARCH and ConneXions are served by homeschool therapist.

26. Initiating the Process
Recommendation for counseling can come from multiple sources to the guidance counselor (e.g. the school based management team (e.g. PST, SST, SSMT), parent, principal, assistant principal, school counselor, school social worker, behavior liaison, Department of Juvenile Justice, and/or Department of Social Services).
Referral
School Counselor contacts parent
Forms: Release of Information, CCS Request for Outpatient Therapy Form, Provider Request for Information Provider options discussed
Schedule Comprehensive Clinical Assessment (CCA) with provider (will be completed within 10 days)
Implementation
CCA indicates need for therapy
Provider obtains student schedule
Provider schedules therapy sessions based on student needs
Parent participation is encouraged
Intervention strategies are shared
Therapist attends EC, 504, SSMT and PST meetings as needed
After school hours family counseling permitted on-campus

29. FULL IMPLEM ENTATION

30. STEPS to Ensure Effective Implementation
1. Annual Meeting with all School Counselors, Social Workers, and OP Mental Health Providers for networking and to ensure all parties understand the MOA
2. Quarterly meetings with Behavior Liaisons and OP Mental Health Providers to discuss services
3. Quarterly stats from OP Mental Health Providers listing all students served and the number of sessions

31. STEPS to Ensure Effective Implementation
4. Survey School Counselors and Social Workers regarding OP Mental Health Providers and services provided
5. Require documentation of renewed mental health licensure (LPC, LCSW, etc.) for each individual OP Mental Health Therapist
5. Behavior Liaisons attend monthly Community Collaborative Meetings at Partners

32. Behavior and Mental Health Liaisons
TEAMWORK
Behavior and Mental Health Liaisons
Social Workers
School Counselors

33. School Behavior Liaison & PBIS Coordinator
Presenters
Dr. Nellie P. Aspel
EC Director
Teri Putnam MA, LPC, LCAS
School Behavior Liaison & PBIS Coordinator

34. Tools & Resources

35. Tools & Resources
The Active Implementation Hub
Quick Start Video
Stages of Implementation Analysis for Education by NIRN/SISEP

36. DCA

37. Activity: Full Implementation Checklist

38. General Supervision SPP/APR Data Collection Dispute Resolution System
Policies, Practices and Procedures
Monitoring Activities
Improvement, Correction, Incentives, & Sanctions
Targeted Technical Assistance
Fiscal Management
NCDPI has the responsibility to ensure that all children with disabilities receive FAPE. The comprehensive general supervision system:
1. Supports practices that improve educational results and functional outcomes for children and youth with disabilities;
Uses multiple methods to identify and correct noncompliance; and
Utilizes mechanisms to encourage and support improvement and enforce compliance.
(make connections to this PD)

39. Continuum of Transitions
SSIP:
Focus for Improvement
Academics
Continuum of Transitions
Behavior
Engagement
So while General Supervision defines the responsibilities of the EC Division and of LEA EC services, the State Systemic Improvement Plan, or SSIP, focuses specifically on improvement in graduation rates for students with disabilities. To accomplish that goal, the EC Division has identified broad root causes to inform our focus statewide on Academics, Behavior, and the Continuum of Transitions. For each of these, it will be necessary to consider how we are addressing engagement of students, staff, families, and communities. This framework allows you to inform, explain and make connections between Academics, Behavior, Continuum of Transitions and Engagement to increase the success of every student. Engagement efforts should encourage students and families to be involved in the process from PreK through adulthood.

40. Continuum of Transitions
This Professional Learning Supports Improved Graduation Rate for Students with Disabilities by:
Academics
Active implementation of evidence-based instruction, preK through secondary
Local capacity building
Technical and adaptive leadership
Comprehensive and efficient assessment system
Student, Family, Staff, and Community Engagement
Behavior
Active implementation of evidence-based behavioral programming, preK through secondary
Local capacity building
Culturally responsive instruction
Positive school/class climate
Comprehensive and efficient assessment system
Student, Family, Staff, and Community Engagement
Engagement
This slide lists some specific considerations for improving academics, behavior, and the transition continuum. This professional learning supports: (highlight areas) by (direct connections)
Continuum of Transitions
Active implementation of evidence-based transition practices, preK to post-secondary
Local capacity building
Systemic and coordinated supports (including wraparound), preK to post-secondary
Consistent attendance
Extracurricular and/or employment opportunities
Student, Family, Staff, and Community Engagement

41. Contact:
Teri Putnam, Cleveland County Schools Rhonda Harrell, Behavior Support and SIT Heather Reynolds, State Implementation Specialist & Data Analyst or