1. A9 – Action Planning with the Interconnected Systems Framework Implementation Inventory Leader Presenter: Katie Pohlman and Eryn Van Acker Exemplar: Dawn Moss, Central Intermediate Unit #10 (PA) Key Words: Evaluation, Assessment, Mental Health

2. Session Objectives
Become familiar with the ISF Implementation Inventory as a tool to measure fidelity of ISF
Identify strategies to strengthen and enhance PBIS implementation aligned with mental health efforts
Understand ways school teams use the ISF Implementation Inventory to install integrated approach

3. Maximizing Your Session Participation
When Working In Your Team
Consider 4 questions:
Where are we in our implementation?
What do I hope to learn?
What did I learn?
What will I do with what I learned?

4. Where are you in the implementation process
Where are you in the implementation process? Adapted from Fixsen & Blase, 2005
We think we know what we need so we are planning to move forward (evidence-based)
Exploration & Adoption
Let’s make sure we’re ready to implement (capacity infrastructure)
Installation
Let’s give it a try & evaluate (demonstration)
Initial Implementation
That worked, let’s do it for real and implement all tiers across all schools (investment)
Let’s make it our way of doing business & sustain implementation (institutionalized use)
Full Implementation

5. Leadership Team Action Planning Worksheets: Steps
Self-Assessment: Accomplishments & Priorities
Leadership Team Action Planning Worksheet
Session Assignments & Notes: High Priorities
Team Member Note-Taking Worksheet
Action Planning: Enhancements & Improvements

6. Who is in the room?
Rate your experience with Interconnected Systems Framework
Fist to Five
Role
State or regional
District
School

7. WHAT IS ISF?

8. An Interconnected Systems Framework (ISF) Defined
A Structure and process for education and mental health systems to interact in most effective and efficient way.
guided by key stakeholders in education and mental health/community systems, youth/family
who have the authority to reallocate resources, change role and function of staff, and change policy.

9. ISF Enhances MTSS Core Features
Effective teams that include community mental health providers
Data-based decision making that include school data beyond ODRs and community data
Formal processes for the selection & implementation of evidence-based practices (EBP) across tiers with team decision making
Early access through use of comprehensive screening, which includes internalizing and externalizing needs
Rigorous progress-monitoring for both fidelity & effectiveness of all interventions regardless of who delivers
Ongoing coaching at both the systems & practices level for both school and community employed professionals

10. 2. Utilize a Single System of Delivery
Key Messages
1. Install with MTSS
2. Utilize a Single System of Delivery
3. Promote Mental Health for All
4. Beyond Access: MH Interventions with Specific Outcomes

11. Where We’ve Been:
: Site Development with PBIS Expansion (informal and independent)
2005: CoP focus on integration of PBIS and SMH
2008: ISF White Paper: formal partnership between PBIS and SMH
Monthly calls with implementation sites, national presentations (from sessions to strands)
Grant Submissions
June September 2013 ISF Monograph
Monograph Advisory group
2015: ISF Learning Community, SOC Webinar Series
2016: RCT Grant awarded
2016: Targeted Work Group Webinars (8)
2017: Targeted Work Group Webinars continuing, knowledge development sites across country

12. School-Wide Positive Behavior Support
Advancing Education Effectiveness: Interconnecting School Mental Health and
School-Wide Positive Behavior Support
Editors: Susan Barrett,
Lucille Eber and Mark Weist
pbis.org
csmh.umaryland
IDEA Partnership NASDSE

13. Where We Are Headed:
TWG webinars and knowledge development sites continue
Training/TA curriculum and workbook available online
Other resources, tools, examples also available
In process of developing “Monograph Volume 2: An Implementation Guide”

14. ISF IMPLEMENTATION INVENTORY

15. Purpose of ISF Implementation Inventory
To assist school and community partners in their installation and implementation of ISF
To assess baseline and/or ongoing implementation progress of critical ISF features
To inform action planning that advances and enhances ISF implementation
To measure ISF implementation fidelity

16. Iterative Development of ISF-II
STAGE 1: Initial Pilot
Developed from tool piloted by Bob Putnam (May Institute)
5 pilot schools in 2 school districts
Usability survey
Phone interviews with school leaders and coaches re: process and content validity
STAGE 2: Revisions
Reword/shorten confusing items
Eliminate jargon and develop glossary of terms
Reworded ratings – “not yet in place”
“Not applicable”
Content revisions (add, reorganize items)
STAGE 3: Validation Study
Ongoing!
54 items across 6 domains assessed in all 3 tiers
Seeking national sample of schools implementing ISF
Conducted online via Qualtrics survey system
Evaluating…
Interrater reliability
Concurrent, Consequential, and Social validity

17. ISF Implementation Inventory: Tiered Domains
19 items
16 items
Implementation of SWPBIS: Are core features of SWPBIS implemented with fidelity?
Teaming: Do team members collaborate? Do team members include education and mental health system representatives, families, and students as indicated with active opportunities for participation and collaboration
Collaborative Planning and Training: Do all team members have PD and training across systems and core features of ISF, as well as intervention practices as appropriate?
Family and Youth Engagement: Are students and families included in teaming, decision making, intervention selection and implementation, intervention monitoring, and system processes?
Intervention Selection, Implementation and Progress: Are evidence-based interventions selected based on need, implemented with fidelity, progress monitored, and concluded after attainment of positive outcomes?
Data-Based Decision Making: Are data representative of school, home and community behavior collected, analyzed and used for decision making, including outcome/impact, process, and fidelity data?
ISF Implementation Inventory: Tiered Domains

18. ISF Implementation Inventory
Qualtrics View: ISF Implementation Inventory

19. ISF Implementation Inventory Brief Procedures
Completed individually by all members of school/community leadership, PBIS, ISF team
Approximately 25 minutes to complete
Results aggregated by research team into School Report Card
Consensus and action planning meeting facilitated by coach

20. ISF Implementation Inventory Report Card
Percent of implementation fidelity is graphed
Graphed by tier and assessment time point

21. Tier implementation score
ISF Implementation Inventory Report Card
Item to consider for action planning to create immediate change
Items will already be highlighted based on scoring by the ISF research team. Schools will be encouraged to review highlights for action planning
Item to consider for action planning but may need additional time/more intensive system changes for score improvement
Tier implementation score

22. ISF Implementation Inventory Report Card
Items will be pre-sorted by implementation level and tier
Supports action planning
Use this row for quick comparison of implementation across tiers
Use this row for quick comparison of item responses and action planning. Refer to previous report pages for actual items.

23. Number of goals/action items will depend on need and resources of school

24. Illinois Cohort

25. Demographics of Illinois Cohort
Focus at building level due to specific project
8 schools within 4 districts
3 Elementary
3 Middle Schools
1 Junior High/High School
1 Alternative building
For most TFI indicating fidelity of implementation
2 Districts with community mental health partnerships and 2 exploring

26. Timing of Completion Teams were new to ISF concepts
Tool helped define in detail an integrated system
Found it to be eye-opening to what they had in place
Especially teams with
community mental health
providers in buildings
Helped develop a roadmap!

27. ISF Implementation Inventory

28. Commonalities Strengths Area to develop
Tier 2 and 3 Implementation was generally more of a strength
Intervention selection, implementation and progress monitoring
Youth and family engagement at tier 2 and 3
Integrating community partners on building teams
Cross-training
Expanded data sources
Identifying mental health supports across the universal system
Resource mapping and selection process for interventions

29. Action Planning Consensus meeting Led to broad goals
Opportunity to review results
Discussion on different ratings for individual items
Led to broad goals
Achievable action steps
ISF II narrowed the focus

30. Achievable Action Steps
Included community mental health partner on Tier 1 and 2 teams
Sharing data with community mental partner
Conducting parent/guardian surveys to increase involvement
Identified requirements for entering and graduating or placing out of a tier/intervention
Adding pro-social and coping skills to their behavioral matrix and teaching schedule
Cross-training of community partners

31. Example

32. ISF Implementation Inventory Highlights
Successes
Lessons Learned
Gave teams something tangible
Allowed teams to reevaluate PBIS implementation as a whole
Allowed school teams to collaborate within and outside of the district
ISF implementation and teaming should occur across tiers
ISF is a slow process
Develop systems and protocols to support implementation

33. Regional PA PBIS Facilitator
Dawn Moss
Regional PA PBIS Facilitator
ISF Facilitator
CIU 10

34. Demographics of PA Cohort
Keystone Central School District DLT est 2015
1 High School
1 Jr/Sr High School
1 Middle School
6 Elementary Schools
*1 Elementary with a mental health partnership
Bellefonte Area School District DLT est 2012
4 Elementary Schools
PA Banner Status/Fidelity of Implementation using the SET

35. Example of Work Flow Checklist
Select District and Schools
Form or Expand District Team (Workgroup of existing team?)
Membership
Establish Operating Procedures
Conduct Resource Mapping of current programs/initiatives/teams
Identify gaps/needs
Assess staff utilization
Examine organizational barriers
Establish priority- measureable outcomes
Develop Evaluation Plan
District and School Level
Tools Identified
Economic Benefits
Develop Integrated Action plan
Identification of Formal Process for Selecting EBP’s
System for Screening
Communication and Dissemination Plan
Write MOU- Determine who will implement the plan

36. District Leadership Team Year One 2015-2016
Variety of stakeholders present
Child serving systems
Other community organizations
Family members
Overview of PBIS systems
Resource Mapping for MTSS
Reviewed Community and School Data
Developed Integrated School Action Plan
Universal Screening in one district

37. DCLT/ISF Teams Year Two 2016-2017
Focused on two schools per district (1 Elem 1 MS)
Narrowed the focus of our community members
Resource mapping/ reviewed data
Established cross system teaming structure
Conducted universal screening
The four schools participated in the Interconnected Systems Framework Inventory.
The ISF Inventory became the guiding document to action plan and evaluate.

38. DCLT/ISF Teams
Established protocol for selecting, implementing, and progress monitoring EBPs
Established protocol for data-based decision making

39. ISF Implementation Inventory

40. Dickey Elementary

43. #KEYSTONECARES
Clinton County Offices of Children and Youth, Courts, District Attorney, and Probation
Mission to offer multi-agency effort to focus on practices that will make life better for children and families
Monthly themes throughout school year – face to face activities and social media

44. ISF Critical Domains
Implement SWPBIS – goals address areas that need “booster” for fidelity
Teaming – ISF Core team includes school, mental health, and working on family rep.
Collaborative Planning and Training – action plan in place to address ongoing PD
Family and Youth Engagement – aligned with “Keystone Cares”

45. ISF Critical Domains
Expanding data to review, using DBDM process, have incorporated universal screening
Building enhanced continuum of EBPs with a protocol for selection
Building protocol for progress monitoring for fidelity and impact (student outcomes)

46. Challenges/Potential Barriers
Stakeholder group was “too big” – needed to pair down get work done
Those in authority are not easy to keep at the table – send delegates to do work, but need decision makers

47. Challenges/Potential Barriers
You don’t know what you don’t know
Frustration from team members because of “slow” process (wishing they were farther along or thinking they were farther along)
Keeping everyone motivated

48. Celebrations of Success
Improved communication and collaboration among stakeholders
On a path towards a single system of delivery
Created a platform for aligning and integrating initiatives

49. Celebrations of Success
Universal screening
Progress on implementation inventory – systems features
Development of integrated action plan and having mental health clinician participate actively in PD, teaming, etc.

50. ISF- II Validation Study Procedures
For more information about the ISF-II Validation Study contact Kelly Perales
Engagement and Enrollment
How will ISF-II help my ISF implementation?
Administrator consent for participation and scheduling (Contact us!)
Team members individually complete ISF-II on Qualtrics
Open 2 weeks
School Report Card
2 weeks after closing survey link
Coach-led review of results and action planning
Final report of scores
Data analyses and results reporting

52. Several Pages are Available

53. Sharing Examples from Sites

54. New Workbook with Hyperlinks

55. Join the Targeted Workgroup Webinars

56. Please Provide Feedback
Your feedback is important to us! Please take a few moments at the end of the session to complete an evaluation form for this session. Forms are available:
In our mobile application by clicking the link in the session description.
Online underneath the posted presentations at
Via paper form from your session facilitator

57. San Diego, California Pre-Conference Workshops: MARCH 28
Save the date, March 28 – 31, 2018 for the…
Pre-Conference Workshops:
MARCH 28
Skill-Building Workshops:
MARCH 31
Breakout Sessions
Networking
Posters
Exhibits
San Diego, California
Manchester Grand Hyatt San Diego, One Market Place
For more information,
go to:
conference.apbs.org

58. Discussion and Questions

59. Contact Us Katie Pohlman katie.pohlman@midwestpbis.org Dawn Moss