Scaling Up SW PBS Implementation: Different Journeys to the Same Destination Heather Peshak George, Ph.D. APBS Conference, St. Louis, MO: March 26, 2010

2 Florida’s PBS Project Our Mission Increasing the capacity of Florida’s school districts to address problem behavior using positive behavior support through a Response to Intervention framework. What we do… Provide training and technical assistance to districts across the state in the development and implementation of positive behavior supports at the school-wide, targeted group, classroom and individual student levels

3 Exploration Phase Is there a need for change? What current practices and initiatives exist that are facilitators or barriers? What is innovation and does it address our problem? How do we plan for implementation? Is the team ready to begin installation of innovation?

4 Installation Phase What individuals have authority to reallocate resources/facilitate implementation and connect with state improvement plan? Who will guide implementation? What does implementation of the innovation involve?

5 Initial Implementation Phase Who are initial implementers? How do we begin implementation? How do we monitor fidelity and outcomes?

6 Full Implementation Phase How do we build local training/coaching capacity? What resources do we need to add more schools? How do we sustain fidelity and positive outcomes over time?

7 Innovation and Sustainability Phase How do we braid innovation with new/existing initiatives? How do we become more efficient and effective? How do we share what we have learned?

9 Funding IDEA Part B Discretionary Grant Around 1997: $33,000/year Current: Almost $1.8 million/year

10 Visibility Before: Present at every state conference Special education Expanded across disciplines ABA, Safe Schools, RtI, Reading, School Psychologists, Superintendent, Evaluation Now: Present when requested or “invited”

11 Click Here…

12 Newsletter: Read directly from the website or download to print. Use to find useful tips, data, important dates, etc.

13 Political Support and Policy Expanded funding of project State RtI Implementation Plan Active member on the FL DOE’s State Transformation Team for RtI Title of Project to include RtI:B RtI Bullying & Harassment Policy Zero Tolerance

State Infrastructure State Management Team –Role: Provide leadership and facilitate policy-level changes to support implementation of effective educational practice State Transformation Team –Role: Analyze progress toward statewide efforts, recommend actions for improvement, and support District Based Leadership Teams DBLT to build the capacity of districts to implement evidence-based practices and to establish integrated RtI academic and behavior systems in each school District Based Leadership Team –Role: Provide leadership, advisement, and training at the district level and assist schools in their implementation efforts School Based Leadership Team –Role: Develop a school implementation plan. The school based team will become “trainers” and “coaches” for the school staff and will be responsible for school wide implementation Advisory Group –Role: Provide on-going stakeholder input –pp of the Statewide RtI Implementation Plan

District Responsibilities Based on self-assessment results, and in conjunction with the student progression plan and K-12 Comprehensive Reading Plan, develop an RtI implementation plan organized around building consensus, infrastructure, and implementation. Plans should also address: –How current resources will be used to implement RtI and identify additional resources needed –How stakeholders will be educated –How stakeholders will be involved p. 8 of the Statewide RtI Implementation Plan

Alignment with Existing Initiatives The basic elements of RtI are required by NCLB and IDEA, therefore, they are included in all broad-based initiatives for schools striving to meet (AYP), such as: –K-12 Reading Plan and Reading First –Florida’s Continuous Improvement Model (FCIM) –Florida’s Positive Behavior Supports (PBS) –PS/RtI Pilot Project –Bright Beginnings p. 1 of the Statewide RtI Implementation Plan

Tiered Model of School Supports & the Problem-Solving Process ACADEMIC and BEHAVIOR SYSTEMS Tier 3: Intensive, Individualized Interventions & Supports The most intense (increased time, narrowed focus, reduced group size) instruction and intervention based upon individual student need provided in addition to and aligned with Tier 1 & 2 academic and behavior instruction and supports. Tier 2: Targeted, Supplemental Interventions & Supports. More targeted instruction/intervention and supplemental support in addition to and aligned with the core academic and behavior curriculum. Tier 1: Core, Universal Instruction & Supports. General academic and behavior instruction and support provided to all students in all settings. FL RtI State Transformation Team, Dec. 2009)

18

19 Training School-Wide/Initial (Tier 1) Booster/Retraining (Tier 1) Classroom (Tiers 1 & 2) – online only Targeted Group (Tier 2) Individual Student (Tier 3) Team Leader Principal/Administrator SWIS Coaches’ Italics indicates both on-site & online modules available

20 Tier 1 Training Sequence Day 1 Intro to RtI:B Teaming Developing Expectations & Rules Developing a System for Teaching Day 3 Evaluation Classroom PBS Facilitated work time Day 2 Developing a Reward System Effective Discipline Procedures Definitions Forms Process Responses Implementing PBS

22 Number of Schools Trained (As of March, ) 6 Pre-K schools 387 Elementary schools 167 Middle schools 82 High schools 56 Alt/Center schools 50 Other (e.g. K-8) 748 TOTAL SCHOOLS

Coaching & Behavioral Expertise RtI:B Support Structure Project Contact  District Coordinator (District Leadership Team)  COACH  School-based Team Leader  PBS Team Members

25 Factors to Consider in Developing Comprehensive Evaluation Systems 1)Systems Preparation –Readiness activities 2)Service Provision –Training and technical assistance 3)Identification and Assessment of Behavior Problems –Possible data sources 4)Evaluation Process –Timelines, data systems 5)Evaluation Data (Across all three Tiers) –Implementation Fidelity, Impact on Students, Attrition, Client Satisfaction 6)Products and Dissemination –Reports, materials, presentations, etc. (modified from Childs, Kincaid & George, in press)

26 Florida’s Evaluation Model Training On-going technical assistance FLPBS ↓ Districts ↓ Coaches ↓ Schools End- Year Impact Outcome data (ODR, ISS, OSS) FL Comprehensive Assessment Test Benchmarks of Quality School Demographic Data PBS Walkthrough Daily Progress Reports Behavior Rating Scales Climate Surveys Implementation Fidelity PBS Implementation Checklist (PIC) Benchmarks of Quality (BoQ) Benchmarks for Advanced Tiers (BAT) School Demographic Data School-wide Implementation Factors Tier 3 plan fidelity checklist BEP Fidelity checklist Project Impact Attrition Survey/Attrition Rates District Action Plans Client Satisfaction School-Wide Implementation Factors District Coordinator’s Survey Training Evaluations  Annual Reports  Revisions to training and technical assistance process  National, State, district, school dissemination activities  Website  On-line training modules Systems Preparation Service Provision Evaluation Process Evaluation Data Products and Dissemination MidY ear I MidY ear II Identification/ Assessment  Discipline Records  ESE Referrals  Surveys  Walkthroughs  PIC  Classroom Assessment Tool  Student rank/rating  Teacher requests  Lack of response  BAT  Behavior Rating Scale  Daily Progress Report Charts District Action Plan District Readiness Checklist School Readiness Checklist New School Profile (includes ODR, ISS, OSS)

27 (1) Systems Preparation Tier 1 District Readiness Checklist District Action Plan –District baseline and goals School Readiness Checklist Baseline data Tier 2 School Readiness Implementation of Tier 1 School Infrastructure Tier 3 –District Action Plan Systems change Evaluation of products and processes Establish vision and goals

28

29 (2) Service Provision Training and ongoing technical assistance FLPBS Districts Coaches Schools Training Tier 1- District and multi-district on-site Tier 2 –District, multi-district, web-based Tier 3- Post assessment, goal setting, systems/process established

(3) Identification & Assessment Tier 1 –Discipline records, attendance, ESE referrals, baseline BoQ, action plans, climate surveys, coaches surveys, walkthrough (mini SET), PBS Implementation Checklist (PIC) Classroom –Discipline records, teacher requests, student rankings/ratings, ESE referrals, observations, Classroom Assessment Tool Tier 2 –Discipline records, teacher requests, student rankings/ratings (SSBD, TRF, etc…), lack of response to Tier 1, Daily Progress Reports, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT) Tier 3 –Above items, lack of response to Tier 2, Behavior Rating Scale, observation data, intervention fidelity checklist, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT)

31 Office Discipline Referrals

Student Initials Grade/Period I or E (Step 2) _________ ________________ Rank top 3 externalizing and top 3 internalizing students Check “YES” if personally taught expectations to the student Check “YES” if personally given a SW-PBS reward to student Teacher Nomination

Tier 2 Progress Monitoring

Behavior Rating Scale BehaviorDate Hitting 8 or more 6-7 times 4-5 times 2-3 times 0-1 times Profanity 16 or more times times 8-11 times 4-7 times 0-3 times Requesting Attention/ Assistance 55% or more 40-55% 25-40% 10-25% 0-10%

(4) Evaluation Process Timelines for State Evaluation –Baseline (due date varies) –Mid Year I – due 10/31 School Profile PBS Implementation Checklist (PIC) (Tiers 1-3) –Mid Year II – due 2/28 PBS Implementation Checklist (PIC) (Tiers 1-3) –End Year – due 6/15 Benchmarks of Quality (BoQ) (Tier 1) Benchmarks for Advanced Tiers (BAT) (Tiers 2-3) Outcome Data Summary School-wide Implementation Factors (SWIF) Web-based Data Entry and Reporting PBSES Statewide Student Database – Academic/Behavior

36 (5) Evaluation Data a)Implementation Fidelity –PIC (All Tiers) –BoQ (Tier 1) –BAT (Tiers 2-3) –SWIF (All Tiers) –Walkthrough (Tier 1) –Tier 2 & 3 intervention specific fidelity measures c)Attrition –Attrition Survey (All) b)Impact on Students –Outcome data (ODR, ISS, OSS) –Academic achievement –School Demographic Data (e.g. ethnicity) –Attendance –DPR charting –Behavior Rating Scale d)Client Satisfaction –SWIF –Climate surveys –Social validation

37 (a) Implementation Fidelity 1.Are schools trained in SWPBS implementing with fidelity? Across years? Across school types? –BoQ, BAT, School Demographic Data 2.What factors are related to implementing with fidelity? –SWIF survey, BoQ, BAT

38 Tier 1 Critical Element Implementation Level chart

39 PBS Implementation Level chart

40 High Implementing Florida PBS Schools (Scoring 70 or Above on BoQ) Average Score 66 Average Score 66 Average Score 72 Average Score 77 Average Score 76

BoQ Totals by School Type Across Years

Brief Walk-through

Benchmarks for Advanced Tiers Score Summary for BAT Sections Tier 1 T2&3:Commit T2&3:Stu Ident T2&3:Monitor T2:Support T2:Main Impl T2:Main Monitor T2:2nd Impl T2:2nd Monitor T2:3rd Impl T2:3rd Monitor T2:4th Impl T2:4th Monitor T3:Support T3:Assess T3:Monitor Total BAT Sections Percent Possible Score

44 (b) Impact on Student Behavior 1.Do schools implementing SWPBS decrease ODRs, days of ISS, and days of OSS? –ODRs, ISS, OSS 2.Do schools implementing SWPBS realize an increase in academic achievement? –FCAT scores 3.Is there a difference in outcomes across school types? –ODRs, ISS, OSS, FCAT scores, school demographic data 4.Do schools implementing with high fidelity have greater outcomes implementers with low fidelity? –BoQ, ODRs, ISS, OSS 5.Do teams that work well together have greater outcomes than those that don’t work as well together? –Team Process Evaluation, ODRs, ISS, OSS

higher implementing schools reported 37% fewer ODRs per 100 students

overall average difference of 40% across the five school years

overall average difference of 43% across the five school years

49 (c) Attrition 1.Why do schools discontinue implementation of SWPBS? –Attrition Survey

50 (d) Consumer Satisfaction 1.Are our consumers satisfied with the training, technical assistance, products and support received? –SWIF survey –District Coordinators survey –Training evaluation –Climate surveys

School-Wide Implementation Factors (SWIF) Higher Implementing Lower Implementing (70+ on BoQ) (-70 on BoQ) 90% + respondents from high implementing schools identified these factors as Helpful: 80%+ respondents from low implementing schools identified these factors as Helpful: Factors MOST Helpful to Implementation of SWPBS  Expectations and rules clearly defined  Administrator committed to PBS, willing to teach and model PBS, willing to reward students  Representative and committed PBS Team  Reward system works  PBS Coach’s guidance with process  Students responses to rewards and activities  Expectations and rules clearly defined  Administrator willing to reward students  Representative PBS Team 25%+ respondents from high implementing schools identified these factors as Problematic: 50%+ respondents from low implementing schools identified these factors as Problematic: Factors MOST Problematic to Implementation of SWPBS  Adequate funding  Team recognizes faculty participation  Staff stability from year to year  Student stability from year to year  Staff time for PBS  Staff belief about effectiveness of PBS  Staff philosophy  Staff consistency in teaching  Staff consistency in discipline procedures

53 (6) Products and Dissemination Annual Reports Revisions to Training Revisions to Technical Assistance process Dissemination activities: –National, state, district, school levels Revisions to Website On-line training modules

Model Schools 32% of eligible schools achieved model school status in Of the 133 schools = 27 Gold, 72 Silver, 34 Bronze

55 What’s New Train-the-Trainer Evaluation Process Expansion of Online Training Modules State-wide Behavioral Database Possible State RtI Center High Schools Further Research ???

56 Current Research PBS Implementation Checklist (PIC) Benchmarks of Quality (Revised) Benchmarks for Advanced Tiers (BAT) Walkthrough Tier 3 Model Demonstrations at Tier 2/3

57 Challenge

58 Contact Heather Peshak George, Ph.D. Phone: (813) Fax: (813) State Website: National Website: Association on PBS: