2018 SEA Title IV Part A Coordinator’s Meeting Building Local Capacity to Implement Effective Practices: Lessons Learned about building and using Technical Assistance Rob Horner, University of Oregon OSEP and OSHS TA Center on Positive Behavioral Interventions and Supports (PBIS)

Goals: Sandy: Practices, Systems, Data, Capacity Acknowledge the growing expectations that SEAs will improve the capacity of local districts (LEAs) and regions to implement effective practices Propose 10 core elements of “capacity building” that affect both initial and sustained implementation. Summarize lessons learned about use of Technical Assistance from our experience with PBIS

Evolving role of State Education Agencies Initial expectations Leadership, Advocacy Compliance Flow-through funding Emerging Expectations Capacity Building SEA, Region, County, District

What is School-wide Positive Behavior Intervention and Support (PBIS)? School-wide PBIS is: A multi-tiered framework for establishing the social culture and behavioral supports needed for a school achieve behavioral and academic outcomes for all students. Evidence-based features of PBIS Prevention Define and teach positive social expectations Acknowledge positive behavior Arrange consistent consequences for problem behavior On-going collection and use of data for decision-making Continuum of intensive, individual intervention supports. Implementation of the systems that support effective practices

School-wide Positive Behavioral Interventions and Supports (PBIS) The social culture of a school matters. A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families. Effective practices with the systems needed for high fidelity and sustainability Multiple tiers of intensity

Purpose of OSEP TA-Center on PBIS The fundamental purpose of PBIS is to make schools more effective, efficient and equitable learning environments for all students. ------------------------------------------------------------------------------------ The purpose of the OSEP TA-Center on PBIS is to define, develop, implement, and evaluate a multi-tiered approach to Technical Assistance that improves the capacity of SEAs, LEAs, and schools to establish, scale-up, and sustain School-wide Positive Behavioral Interventions and Supports (PBIS).

Installing Positive Behavioral Interventions and Supports Step 1: Select effective practices that fit the needs and culture of the context? (ODR patterns, Attendance, Fidelity, School Climate Survey) The most significant behavioral challenge is ____________. It occurs most often ___(where/when)____. The students most likely engaging in this behavior are _________________, and they appear to do this behavior to get _____________. Student Outcomes Academic, Social, Attendance, Emotional Step 5: Use both fidelity and impact data to improve (Adjust practices to achieve effect, adjust systems to improve efficiency and sustainability. Data (Information and procedures for effective decision-making) Practices (Changing Student Behavior) Organizational Systems (Supporting Staff Behavior) Step 2: Combine Prevention and Clear Consequences: Prevent: School culture that is predictable, consistent, positive, safe Consequences: Accountable, function-based, fair, instructional Step 4: Implement organizational systems that will sustain and scale effective practices (policies, teaming, community of practice, data review, continuous improvement) Step 3: Build a Multi-tiered Continuum of Practices: Identification: Who Assessment: Tailored Need Comprehensive Supports:

Experimental Research on SWPBIS Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115 Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26. Bradshaw, C., Waasdorp, T., Leaf. P., (2012 )Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics, 130(5) 1136-1145. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14. Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128. Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 Bradshaw, C. P., Pas, E. T., Goldweber, A., Rosenberg, M., & Leaf, P. (2012). Integrating schoolwide Positive Behavioral Interventions and Supports with tier 2 coaching to student support teams: The PBISplus Model. Advances in School Mental Health Promotion, 5(3), 177-193. doi:10.1080/1754730x.2012.707429 Freeman, J., Simonsen, B., McCoach D.B., Sugai, G., Lombardi, A., & Horner, ( submitted) Implementation Effects of School-wide Positive Behavior Interventions and Supports on Academic, Attendance, and Behavior Outcomes in High Schools. Flannery, B., Fenning, P., Kato, M., & McIntosh, K. (2014). Effect of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly, 29, 111-124. SWPBIS Experimentally Related to: Reduction in problem behavior Increased academic performance Increased attendance Improved perception of safety Reduction in bullying behaviors Improved organizational efficiency Reduction in staff turnover Increased perception of teacher efficacy Improved Social Emotional competence

Schools Implementing PBIS August, 2017 ----------------------------------- 13,832,582 Students 3367 High Schools

Schools Implementing PBIS August, 2017 21 States with over 500 schools using PBIS

Proportion of Schools Implementing PBIS by State, August 2017 14 States with over 40% of schools using PBIS

Visibility and Dissemination Policy and Systems Alignment Implementing PBIS: Implementation Drivers and Capacity Development (www.pbis.org) Funding Visibility and Dissemination Policy and Systems Alignment Personnel Selection and Readiness Leadership Team (Active Coordination) Training & Orientation Coaching Evaluation and Performance Feedback Content Expertise Local Demonstrations

Visibility and Dissemination Policy and Systems Alignment Implementing PBIS: Implementation Drivers and Capacity Development (www.pbis.org) Funding Visibility and Dissemination Policy and Systems Alignment Personnel Selection and Readiness Leadership Team (Active Coordination) Training & Orientation Coaching Evaluation and Performance Feedback Content Expertise Local Demonstrations

Targeting Technical Assistance -------------- Building Capacity at Multiple Levels of System

Lessons Learned About Technical Assistance Technical Assistance involves multiple tiers of intensity. Tier I: Websites, webinars, conferences, materials Tier II: Planning, resource mapping, and coordination Tier III: On-site training, coaching, problem solving Use Technical Assistance to Invest in Systems as well as Practices Selection of new practices/ Alignment Funding/ Policy Staffing Data Collection and Use Investing in District Capacity

Use a Systems Approach to Technical Assistance State/Region are the units of facilitation District is the unit of implementation School is the unit of analysis Student is the unit of impact

Establish Local Training and Coaching Use the development of local “exemplars” to establish both Trainers and Coaches Training: The supports needed to develop new skills and knowledge Coaching: The supports needed to use new skills and knowledge in typical contexts Prompting , Fluency Building, Performance Feedback, Adaptation to Culture and Organization Local Trainers and Coaches are a key capacity for reducing the cost of scaling

Coach returns from leave Example of the Impact of Coaching on Student Outcomes: Average Major Discipline Referrals per Day per Month Coach returns from leave Coach goes on leave From Steve Goodman, Michigan

Establish the Systems needed for Sustainability as well as initial adoption Build systems to support implementation for at least 10 years. Policies, Funding, Communication, Staff Selection, Staff Orientation, Staff Evaluation Use implementation Science

Measure Fidelity of “Practice Implementation” as well as Outcome Impact Content Validity (Tier 1 .95; Tier II .93; Tier III .91) Usability (12 of 14 > 80%) (15 min per Tier) Inter-rater Agreement (.95; .96; .89) Test-retest reliability (.98; .99; .99) Factor Analysis Available from OSEP TA-Center www.pbis.org www.pbisapps.org No Cost Assessors Training PowerPoint and Assessors Training Video at www.pbisapps.org

Schools Reporting PBIS Tier I Fidelity August, 2017 14,324 Schools Reporting Tier I Fidelity 9,564 Schools Meeting PBIS Tier I Fidelity Criteria

Measure the effect of Technical Assistance on Local implementation Capacity Use the District Capacity Assessment Completed by District Team with an External Facilitator Facilitator orientation Focus on a specific “practice” Sub-scales Team Implementation Drivers Trainers, Coaches, Selection Data Systems https://youtu.be/YKFLzeybH0c

An example From One District in Oregon

Implementation of PBIS: One District Mean TFI (Total)

Build Technical Expertise: Implications for Selection & Training Tier I can be implemented with minimal development of behavioral competence Tiers II and III require knowledge of behavioral theory at the school level

Summary SEAs are being asked to provide the technical assistance needed to build local district and region capacity. Build the ability to provide “active” as well as “passive” TA Focus on Districts as the unit of implementation Measure and use Implementation Fidelity data as well as outcome data Incorporate the elements of “Implementation Science”

Contact Information www.pbis.org George Sugai University of Connecticut George.suagi@uconn.edu Rob Horner University of Oregon robh@uoregon.edu Tim Lewis University of Missouri LewisTJ@missouri.edu