Team Self-Assessment to Action: Effective Use of the TFI Yuliana Manríquez Kent Smith manriquezy@wisconsinpbisnetwork.org @Yuli32974159 smithk@wisconsinpbisnetwork.org @kent1915

Assessment Connection The content of this session connects to TFI, items 1.2, 1.14, 1.15, 2.2, 2.12, 2.13, 3.2, 3.17

Why do we assess? Continuous Improvement! -A major feature of SWPBIS is the commitment to ongoing assessment of implementation. -PBIS Assessment provides surveys for teams to take as they examine their level of SWPBIS adoption and guides them through the process for how to improve implementation to benefit students, their families, and the overall school culture.  More speaker Notes: When planning and making data-based changes, we strategically use data in an on-going continuous improvement process in which schools and districts deliberately and strategically collaborate to understand, replicate and scale up successes as well as plan for and address areas of concern. When implemented effectively, the continuous improvement process leads to long-term, embedded, positive change and progress in the school or district, thereby improving student outcomes.  Moreover, this process leverages the adults in the system to make changes that are measurable to affect student outcomes.

Use of Data Leads to Sustainability Teams who regularly use data to plan and make changes are more likely to sustain implementation! recognize improved outcomes collect observable, measurable implementation and outcome data share data with all staff monitor data continuously -Staff are more likely to continue implementation when they recognize improved outcomes and perceive them to be directly related to the practice -Teams regularly collect observable, measurable implementation and outcome data -Teams regularly share data with all staff, who then use it to plan and make changes -Staff monitor data continuously and have feedback systems in place More speaker Notes: Research shows that a main factor that facilitates sustained implementation of an EMLSS is that leadership teams use data to plan and make changes. https://www.wisconsinrticenter.org/wp-content/uploads/2019/01/1493996457_2017-Evaluation-Brief-Sustainability-Five-Factors.pdf

What do we mean by “Data”? Guides decision-making processes Types Practice data (fidelity) System data (implementation) Outcome data (student-level) aggregate disaggregated Practice Data: Measures the extent to which staff are able to use practices as intended Assesses use of skills that have been trained and supported Helps to assure sustained implementation of practices Not a reflection on teachers, staff, or students “The most effective intervention will not produce positive effects if it is not implemented.” Systems Data PBIS Assessment Assists in supporting high fidelity, sustained implementation of PBIS Online Surveys Tiered Fidelity Inventory (TFI) Self Assessment Survey (SAS) Reports Supports action planning to improve implementation Outcome Data Informal data sources Minors (ongoing) Majors Attendance Grades Angel List Teacher request for assistance Health room visits Pupil Service request for assistance Dropout Early Warning System / Early Warning System Disciplinary actions (time out of class, detentions, missed recesses, ISS, OOSS, etc.)

What is the TFI? The TFI gives teams a single, efficient, valid, reliable survey to guide implementation and sustained use of SWPBIS. Using the TFI, teams measure the extent to which school personnel apply the core features of SWPBIS at all three tiers – either individually or collectively.

What’s the difference? The TFI is based on the features and items of existing SWPBIS fidelity surveys (e.g., SET, BoQ, TIC, SAS, BAT*, MATT*) and can be used to replace any or all of them. Tier 1 - Walkthrough Tool How is TFI different from other assessments?

What’s it Good For? An initial assessment A guide for implementation An index of sustained SWPBIS implementation A metric for state recognition -An initial assessment to determine if they are using, or need, SWPBIS -A guide for implementation of Tier I, Tier II, and/or Tier III practices -An index of sustained SWPBIS implementation -A metric for identifying schools for recognition within their state implementation efforts

What’s it Tell Us?

How do we use it? Assess the overall functioning of your systems at all three tiers for alignment to research Identify strengths and needs around systems and practices Action plan for growth and sustainability

Walkthrough at Tier 1 External coach/evaluator Use TFI Walkthrough Tool Interview 10% or at least 5 staff Interview at least 10 students 15 minutes

Benefits of an External Audit Provides more objective overview of what your team believes, practices from the past year Discovers errors Identifies internal weaknesses Lend themselves to being impartial/ less biased

Use of Data: Interpretation What areas are solid, what need maintenance and what are areas for growth? Why do we think that? What do we do?

Use of Data – Application (Action Plan!) Why do we think there is a need for growth Not just score, what is it about our climate and culture? What will we do about it Teach staff? Clarify with staff? More practices? Error correct? Who will do it? When will be done by? How do we know it worked?

Robust Action Planning Review and triangulate existing data related to Practice data (fidelity) System data (implementation) Outcome data (student-level) aggregate disaggregated Strategically and explicitly bridge and incorporate to school and/or district improvement planning/goal setting Review existing data related to (a) student behavior (e.g., suspension/expulsions, screening outcomes, behavior incidents, discipline referrals, attendance, achievement scores, dropout rates, school climate, student/family surveys) and (b) implementation fidelity

Barriers to Assessment Why? How? What? Access? Turnover Not sure of why to take it Not sure of how to use it Not sure what to take Access to surveys to complete them Turnover in coordinators/facilitators

Barriers: Why? Helps you build and maintain fidelity of your systems Fidelity means that your system has enough of the research based changes in place to expect to see the outcomes you desire Allows teams to identify areas of strength, maintenance and need within systems to be able to action plan based on the results Allows teams to examine systems in relation to perception and outcome Add? This is on-going support (along hopefully with coaching) after training is completed

Barriers: How? Identify areas that are critical to vision and mission and action plan for development. Staff development generally falls into: 1) Teaching and building fluency around new skills or ideas 2) Engaging in fluency and fidelity checks (are we doing it the way we should?) 3) Error correction (something isn’t being implemented as it should) 4) Maintenance (keep those skills strong!) Add? Team needs to pre-plan why they will carves out time to 1) take, and then 2) action plan instead of this being an after thought, an addition to, etc.

Barriers: What?

Barrier: Access

OR… Team members and participants will get a link to survey (SAS) when a survey window is open. Click link to take survey! Coordinators -

Barriers: Turnover Turnover in coordinators/facilitators If the school/team knows there will be shifts, then they should proactively revise this role

Audience Experts What questions have come up for you? How have your teams overcome barriers presented? Other?

csession.questionpro.com