1. 2018 SEA Title IV Part A Coordinator’s Meeting
Building Local Capacity to Implement Effective Practices:
Lessons Learned about building and using
Technical Assistance
Rob Horner, University of Oregon
OSEP and OSHS
TA Center on Positive Behavioral Interventions and Supports (PBIS)

2. Goals: Sandy: Practices, Systems, Data, Capacity
Acknowledge the growing expectations that SEAs will improve the capacity of local districts (LEAs) and regions to implement effective practices
Propose 10 core elements of “capacity building” that affect both initial and sustained implementation.
Summarize lessons learned about use of Technical Assistance from our experience with PBIS

3. Evolving role of State Education Agencies
Initial expectations
Leadership, Advocacy
Compliance
Flow-through funding
Emerging Expectations
Capacity Building
SEA, Region, County, District

4. What is School-wide Positive Behavior Intervention and Support (PBIS)?
School-wide PBIS is:
A multi-tiered framework for establishing the social culture and behavioral supports needed for a school achieve behavioral and academic outcomes for all students.
Evidence-based features of PBIS
Prevention
Define and teach positive social expectations
Acknowledge positive behavior
Arrange consistent consequences for problem behavior
On-going collection and use of data for decision-making
Continuum of intensive, individual intervention supports.
Implementation of the systems that support effective practices

5. School-wide Positive Behavioral Interventions and Supports (PBIS)
The social culture of a school matters.
A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families.
Effective practices with the systems needed for high fidelity and sustainability
Multiple tiers of intensity

6. Purpose of OSEP TA-Center on PBIS
The fundamental purpose of PBIS is to make schools more effective, efficient and equitable learning environments for all students.
The purpose of the OSEP TA-Center on PBIS is to define, develop, implement, and evaluate a multi-tiered approach to Technical Assistance that improves the capacity of SEAs, LEAs, and schools to establish, scale-up, and sustain School-wide Positive Behavioral Interventions and Supports (PBIS).

7. Installing Positive Behavioral Interventions and Supports
Step 1: Select effective practices that fit the needs and culture of the context? (ODR patterns, Attendance, Fidelity, School Climate Survey)
The most significant behavioral challenge is ____________. It occurs most often ___(where/when)____. The students most likely engaging in this behavior are _________________, and they appear to do this behavior to get _____________.
Student Outcomes
Academic, Social, Attendance, Emotional
Step 5: Use both fidelity and impact data to improve
(Adjust practices to achieve effect, adjust systems to improve efficiency and sustainability.
Data
(Information and procedures for effective decision-making)
Practices
(Changing Student Behavior)
Organizational Systems
(Supporting Staff Behavior)
Step 2: Combine Prevention and Clear Consequences:
Prevent: School culture that is predictable, consistent, positive, safe
Consequences: Accountable, function-based, fair, instructional
Step 4: Implement organizational systems that will sustain and scale effective practices
(policies, teaming, community of practice, data review, continuous improvement)
Step 3: Build a Multi-tiered Continuum of Practices:
Identification: Who
Assessment: Tailored Need
Comprehensive Supports:

8. Experimental Research on SWPBIS
Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, Bradshaw, C., Waasdorp, T., Leaf. P., (2012 )Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics, 130(5) Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2): Bradshaw, C. P., Pas, E. T., Goldweber, A., Rosenberg, M., & Leaf, P. (2012). Integrating schoolwide Positive Behavioral Interventions and Supports with tier 2 coaching to student support teams: The PBISplus Model. Advances in School Mental Health Promotion, 5(3), doi: / x Freeman, J., Simonsen, B., McCoach D.B., Sugai, G., Lombardi, A., & Horner, ( submitted) Implementation Effects of School-wide Positive Behavior Interventions and Supports on Academic, Attendance, and Behavior Outcomes in High Schools. Flannery, B., Fenning, P., Kato, M., & McIntosh, K. (2014). Effect of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly, 29,
SWPBIS Experimentally Related to:
Reduction in problem behavior
Increased academic performance
Increased attendance
Improved perception of safety
Reduction in bullying behaviors
Improved organizational efficiency
Reduction in staff turnover
Increased perception of teacher efficacy
Improved Social Emotional competence

9. Schools Implementing PBIS August, 2017
13,832,582 Students
3367
High Schools

10. Schools Implementing PBIS August, 2017
21 States with over 500 schools using PBIS

11. Proportion of Schools Implementing PBIS by State, August 2017
14 States with over 40% of schools using PBIS

12. Visibility and Dissemination Policy and Systems Alignment
Implementing PBIS: Implementation Drivers and Capacity Development (
Funding
Visibility and Dissemination
Policy and Systems Alignment
Personnel Selection and Readiness
Leadership Team
(Active Coordination)
Training & Orientation
Coaching
Evaluation and Performance Feedback
Content Expertise
Local Demonstrations

13. Visibility and Dissemination Policy and Systems Alignment
Implementing PBIS: Implementation Drivers and Capacity Development (
Funding
Visibility and Dissemination
Policy and Systems Alignment
Personnel Selection and Readiness
Leadership Team
(Active Coordination)
Training & Orientation
Coaching
Evaluation and Performance Feedback
Content Expertise
Local Demonstrations

14. Targeting Technical Assistance
Building Capacity at Multiple Levels of System

15. Lessons Learned About Technical Assistance
Technical Assistance involves multiple tiers of intensity.
Tier I: Websites, webinars, conferences, materials
Tier II: Planning, resource mapping, and coordination
Tier III: On-site training, coaching, problem solving
Use Technical Assistance to Invest in Systems as well as Practices
Selection of new practices/ Alignment
Funding/ Policy
Staffing
Data Collection and Use
Investing in District Capacity

16. Use a Systems Approach to Technical Assistance
State/Region are the units of facilitation
District is the unit of implementation
School is the unit of analysis
Student is the unit of impact

17. Establish Local Training and Coaching
Use the development of local “exemplars” to establish both Trainers and Coaches
Training: The supports needed to develop new skills and knowledge
Coaching: The supports needed to use new skills and knowledge in typical contexts
Prompting , Fluency Building, Performance Feedback, Adaptation to Culture and Organization
Local Trainers and Coaches are a key capacity for reducing the cost of scaling

18. Coach returns from leave
Example of the Impact of Coaching on Student Outcomes: Average Major Discipline Referrals per Day per Month
Coach returns from leave
Coach goes on leave
From Steve Goodman, Michigan

19. Establish the Systems needed for Sustainability as well as initial adoption
Build systems to support implementation for at least 10 years.
Policies, Funding, Communication, Staff Selection, Staff Orientation, Staff Evaluation
Use implementation Science

20. Measure Fidelity of “Practice Implementation” as well as Outcome Impact
Content Validity (Tier 1 .95; Tier II .93; Tier III .91)
Usability (12 of 14 > 80%) (15 min per Tier)
Inter-rater Agreement (.95; .96; .89)
Test-retest reliability (.98; .99; .99)
Factor Analysis
Available from OSEP TA-Center
No Cost
Assessors Training PowerPoint
and
Assessors Training Video at

21. Schools Reporting PBIS Tier I Fidelity August, 2017
14,324 Schools Reporting Tier I Fidelity
9,564 Schools Meeting PBIS Tier I Fidelity Criteria

22. Measure the effect of Technical Assistance on Local implementation Capacity
Use the District Capacity Assessment
Completed by District Team with
an External Facilitator
Facilitator orientation
Focus on a specific “practice”
Sub-scales
Team
Implementation Drivers
Trainers, Coaches, Selection
Data Systems

23. An example From One District in Oregon

24. Implementation of PBIS: One District Mean TFI (Total)

25. Build Technical Expertise: Implications for Selection & Training
Tier I can be implemented with minimal development of behavioral competence
Tiers II and III require knowledge of behavioral theory at the school level

26. Summary
SEAs are being asked to provide the technical assistance needed to build local district and region capacity.
Build the ability to provide “active” as well as “passive” TA
Focus on Districts as the unit of implementation
Measure and use Implementation Fidelity data as well as outcome data
Incorporate the elements of “Implementation Science”

27. Contact Information
George Sugai
University of Connecticut
Rob Horner
University of Oregon
Tim Lewis
University of Missouri