Data Calendar-at-a-Glance Rebecca Nies| Alternative and Prevention Services Specialist, MDE PBIS Summer Institute 2017

Goals Participants will be able to navigate the state PBIS website to get information regarding data Participants will be able to state the three categories of data Participants will be able to name and sort common Minnesota PBIS data tools and sort them into the correct data category Participants will draft a data calendar

Setting the Stage- pg. 2 of handout Data Setting the Stage- pg. 2 of handout

Data Word Web Data What When How Why

Integrated Elements

Research identifies four factors that predict sustained implementation of PBIS. District Priority School Priority Capacity Building Team Use of Data school team/staff skill, regular team meetings, data collection, use of data for decision making, presenting data to staff and community (McIntosh, et al., 2014)

Are we implementing the plan as intended? Classes of Data Effort Data Fidelity Data Outcome Data What is the plan? Are we implementing the plan as intended? Is the plan working?

Classes of Data Effort data (What we think we’re doing; Self-perception data) Team Implementation Checklist (TIC) Self-Assessment Survey (SAS) Outcome data (Is it having an effect?) “Big 5” Graphs Triangle % Fidelity data (Is the plan being implemented as intended; Are we actually following the plan)? School Evaluation Tool (SET) Benchmarks of Quality (BoQ) Tiered Fidelity Inventory (TFI)

Tier 1 Evaluation Tools Team Implementation Checklist (TIC) Benchmarks of Quality (BoQ) for after full implementation School Evaluation Tool (SET) Self-Assessment Survey (SAS)

Tier 1, Tier 2 and/or Tier 3 Evaluation Tool Tiered Fidelity Inventory (TFI) Tool may be used to evaluation Tier 1, Tier 2 and/or Tier 3 or all three at once

Data Calendar Event Calendars

Data Calendar Data Calendars

Data Calendar

Data Calendar- Progress Monitoring Measure Function Schedule Team Implementation Checklist (TIC) A self-assessment tool serves as a multi- level guide for creating School-Wide PBIS Action Plans and evaluating the status of implementation activities. Completed annually Tiered Fidelity Inventory (TFI) The TFI gives teams a single, efficient, valid, reliable survey to guide implementation and sustained use of SWPBIS. Using the TFI, teams measure the extent to which school personnel apply the core features of SWPBIS at all three tiers Three times per year- External facilitator recommended Annual walkthrough recommended Self- Assessment Survey (SAS) Used by school staff for initial and annual assessment of effective behavior support systems in their school and to guide Action Planning.

Data Calendar-Fidelity Measure Function Schedule School-wide Evaluation Tool (SET) External evaluation to assess the critical features of school-wide effective behavior support across each academic school year.   Completed annually to provide an external evaluation perspective on fidelity of implementation (benchmark 80%) Benchmarks of Quality (BoQ) Benchmarks of Quality (BoQ) are used by teams to identify areas of success, areas for improvement for the sustained implementation of SW-PBIS, and by the MN PBIS Project to identify model PBIS schools. Completed annually after completion of training schedule (benchmark 70%) Tiered Fidelity Inventory (TFI) The TFI gives teams a single, efficient, valid, reliable survey to guide implementation and sustained use of SWPBIS. Using the TFI, teams measure the extent to which school personnel apply the core features of SWPBIS at all three tiers Once fidelity on a tier is met (70%), the team may choose to shift from three times per year to annually for the purpose of evaluating sustained implementation

Data Calendar-Outcome Measure Function Schedule Office Discipline Referrals (ODR) (SWIS™/Core Reports) Office discipline referrals (ODR) provide data for monthly team reviews and decision making by teachers, administrators, and other staff to guide prevention efforts and Action Planning. Monthly, based on school calendar

Data Planning Before- what preparation needs to happen to collect the data? During- when and how is the data collected? After- who and when is the data reviewed and shared?

Activity What is needed in the process of data collection? Before During After Enter examples

Creating a Data Calendar Aug. Sept. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May Jun Effort Fidelity Outcome

Creating a Data Calendar Aug. Sept. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May Jun Effort PBIS team completes TIC Open SAS window Have staff complete SAS Share progress with staff Fidelity Schedule Team meeting Make copies Open survey on PBISapps Take TFI- Tier 1 -Share TFI results at staff meeting Take TFI- Tier 2/3 - Share results with school board Outcome Share summary of previous year big 7 with staff PBIS meeting review big 7 Share big 7 with staff Include ODR summary in parent newsletters Share big 7 with school baord

Draft Data Calendar Select tools for effort, fidelity and outcome data Identify what is needed before, during and after

Recommendations Recommendations

Reconnecting with Data Recommendations Reconnecting with Data

Reconnecting with Data

PBIS Assessment within PBIS Apps Team Implementation Checklist Self Assessment Survey School Evaluation Tool Tiered Fidelity Inventory

A Brief Overview of PBIS Apps: link to www.pbisapps.org

Login

Dashboard

Accessing Reports

Accessing Reports con’t

PBIS Assessment Reports

Team Implementation Checklist (TIC) Self-assessment tool for monitoring the implementation of school-wide PBIS Team self-assessment in 6 areas Establish commitment Establish and maintain team Self-Assessment Establish School-wide Expectations Establish Information system Build capacity for function-based support

Team Implementation Checklist (TIC) con’t WHO: Universal PBIS Team WHAT: Gudies development, implementation, monitoring and improvement process for building a positive school culture and climate WHERE: at school during a Universal PBIS Team meeting WHEN: 3X/year HOW: One person enters the data online at www.pbisapps.org

Team Implementation Checklist (con’t)

TIC Total Score Report

TIC Subscale Report

TIC Individual Items Report

School-Wide Evaluation Tool (SET) Designed to determine the extent to which schools are implementing School- wide Positive Behavior Supports (PBIS) accurately (as intended) Some schools may already be using some core features accurately, but not all yet… High standard! research level tool to evaluate PBIS implementation

SET data can be used to: Assess features that are in place Determine annual goals Evaluate on-going efforts Design and revise procedures Compare year to year PBIS implementation over time

SET Evaluates 7 Areas Expectations defined Behavioral expectations taught Acknowledgement procedures Correction procedures Monitoring and evaluation Management District-level Support

SET Subscales

SET Items

Outcome Data SWIS (School wide Information System) New Format on the website Provides end of year Triangle percentages Note: Looking for the same types of data no matter what system your school/district uses

SWIS Report Options Average Referrals per Day per Month Referrals by Problem Behavior Referrals by Location Referrals by Student Referrals by Time Referrals by Day of the Week Referrals by Grade Referrals by Ethnicity

SWIS Graphs

SWIS Reports

Average Referrals per Day per Month

Referrals by Problem Behavior

Referrals by Location

Referrals by Time

Referrals by Student

Multiple Year Comparison Graphs

Self Assessment Survey (SAS) Evaluates staff perception of the extent to which PBIS systems are in place within a building School-wide Non-Classroom Classroom Individual Student SAS provides entire staff (certified and non-certified) perspective

SAS School-Wide Systems

SAS Basics WHO: All teachers, staff and administrators in building WHAT: status and need in 4 systems (school, classroom, non-classroom and individual students) WHERE: school WHEN: Annually HOW: All staff take online survey online at PBIS Assessment ww.pbisapps.org

SAS Total Score

SAS Subscale

SAS Item Report

TFI Tiered Fidelity Inventory Team can use the TFI to measure fidelity at Tier I, Tier 2 and/or Tier 3 individually, or all at once External Facilitator Guided Team self assessment based on data collected from a walk through tool and other data sources available at the building level

TFI Basics External Facilitator and Team complete Walk through process (30- 60 minutes to complete) May be used for progress monitoring (3x/year) May be used as annual fidelity evaluation Questions are scored on an operationalized rubric: 2= fully in place 1= in progress 0= not started

TFI on PBIS Assessment

Eventually evaluate all 3 Tiers

Final Thoughts

Experimental Research on SWPBIS Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group- randomized effectiveness trial. Prevention Science, 10(2), 100-115 Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26. Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14. Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128. Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 Bradshaw, Pas, Goldweber, Rosenberg, & Leaf, 2012 SWPBIS Experimentally Related to: Reduction in problem behavior Increased academic performance Improved attendance Improved perception of safety Reduction in bullying behaviors Improved organizational efficiency Reduction in staff turnover Increased perception of teacher efficacy Improved Social Emotional competence Garrett will move this slide to slide library to use for new applicants. Research clear about outcomes PBIS is experimentally correlated with achieving; however to achieve outcomes need to actively manage and measure implementation (need the “what” as well as the “how well” to obtain outcomes) Do not present to cohorts, save in slide library, use for informational meetings for application. bit.ly/PBISoutcomes2014-03

Review- Did We Meet Our Goals? Participants will be able to navigate the state PBIS website to get information regarding data Participants will be able to state the three categories of data Participants will be able to name and sort common Minnesota PBIS data tools and sort them into the correct data category Participants will draft a data calendar Exit Ticket: On the last page of your handouts write down one idea or action item you plan to bring back to your school or district

Thank you! Rebecca Nies Rebecca.Nies@state.mn.us 651-582-8648