The Importance of Coaching in Implementation of Evidence-based Practices Rob Horner University of Oregon

Goals Current assumptions/research about coaching Define our experience with coaching in PBS implementation Implications for building district capacity

Coaching Defined Coaching is the active and iterative delivery of: ◦ (a) prompts that increase successful behavior, and ◦ (b) corrections that decrease unsuccessful behavior. ◦ Coaching is done by someone with credibility and experience with the target skill(s) ◦ Coaching is done on-site, in real time ◦ Coaching is done after initial training ◦ Coaching is done repeatedly (e.g. monthly) ◦ Coaching intensity is adjusted to need

Outcomes of Coaching Fluency with trained skills Adaptation of trained concepts/skills to local contexts and challenges  And new challenges that arise Rapid redirection from miss-applications Increased fidelity of overall implementation Improved sustainability  Most often due to ability to increase coaching intensity at critical points in time.

Training Outcomes Related to Training Components Training Outcomes Training Components Knowledge of Content Skill Implementation Classroom Application Presentation/ Lecture Plus Demonstration Plus Practice Plus Coaching/ Admin Support Data Feedback 10% 5% 0% 30% 20% 0% 60% 60% 5% 95% 95% 95% Joyce & Showers, 2002

Coaching within SWPBS Implementation Context: ◦ 9000 schools implementing SWPBS nationally Defining the Role  Internal vs External Selecting Coaches Training and support for coaches Assessing Impact

Leadership Team Funding Visibility Political Support TrainingCoachingEvaluation Local Demonstration Schools Active Coordination Behavioral Expertise

Coaching vs. Training Coaching involves active collaboration and participation, but not group instruction. ◦ Small group ◦ Build from local competence ◦ Sustainable

Who should be a coach? Internal vs External  Internal coaches are employed in the school where they provide support  External coaches are employed outside the schools where they provide support (e.g. by district, region, state).

Who should be a coach? Internal CoachExternal Coach AdvantagesKnowledge of school Staff relationships Regular access Independent Outside perspective Multiple schools experience DisadvantagesConflicting roles Narrow range of experiences Limited knowledge of school Limited relationships Less frequent access

Who should be a coach Coaching Competencies NecessaryPreferred Participate in team training Able to attend team meetings at least monthly Effective working with adults Knowledgeable about school operating systems Professional Commitment Knowledge about SWPBS Knowledge about behavior support practices (targeted, individual) Skilled in collection and use of data for decision-making.

What Coaches Do Work with team during initial SW-PBS training Meet with new teams monthly on-site Telephone/ contact as needed “Positive” nag  Self-assessment (EBS Survey, Team Checklist)  Action planning  Activity implementation  On-going evaluation  School self-evaluation efforts  State-wide Initiative evaluation efforts (SET) Guide State-wide initiative  Feedback to Taskforce

What Coaches Do Dissemination of outcomes and effects SWIS Facilitation ◦ Implement and support use of data-based decision making.

Commitment of Coaches Team Support ◦ First Year ( 1-2 teams) (participate in training and planning) ◦ Second Year ( Maintain initial teams, start 3-5 teams) ◦ Future Years ( teams total ) FTE commitment ◦ 20-50% Roles/Background ◦ Behavior Specialists, Special Education Teachers ◦ Consultants, Administrators ◦ School Psychologists, Counselors, Social Workers

Guiding Principles for Effective Coaching Build local capacity  Become unnecessary…but remain available Maximize current competence  Never change things that are working  Always make the smallest change that will have the biggest impact Focus on valued outcomes  Tie all efforts to the benefits for children Emphasize Accountability  Measure and report; measure and report; measure and report. Build credibility through:  (a) consistency, (b) competence with behavioral principles/practices, (c) relationships, (d) time investment. Precorrect for success

Specific Expectations ◦ Attend and participate in team training ◦ Meet with your team(s) at least monthly  Provide technical assistance as needed ◦ Monitor and report on team efforts  Team Checklist  EBS Survey/ SET/ ISSET  Annual Profile/Summary Data ◦ Present on School-wide PBS at district, state, national forums. ◦ Assist district to build capacity for sustained implementation (re-define your role over time) ◦ Meetings with Coordinator and Taskforce for purposes of state-wide planning

Assist Teams in Using Data for Decision-making Using Team-Checklist and EBS Survey data for Team Action Planning Using SET/ TIC data for evaluation Using ODR/ Academic (ORF) data for assessment, planning and reporting. Keeping faculty involved through regular data reporting.

Examples Illinois North Carolina Michigan

PBIS in Illinois July 17, 2008 Developing Local Systems of Care for Children and Adolescents with Mental Health Needs and their Families Training Institutes Nashville, TN Lucille Eber Ed.D. IL PBIS Network

PBIS Schools Over Ten Years: Trained & Partially or Fully Implementing

# IL PBIS Schools & # External Coaches June 30, 2008

The Organization of PBIS in Illinois 900 schools implementing SWPBS ISBE Coordination Chicago Coordinators North Coordinators Central Coordinators South Coordinators 46 Coaches (10) 33 Schools 495 Coaches (84) 525 Schools 193 Coaches (20) 203 Schools 105 Coaches (29) 127 Schools

Elementary Middle

Capacity Schools per Coach per Region

Illinois Suspension Rates per 100 PBS slope = Non PBS slope = -.37

Illinois Suspension Rates per 100 for Black and Hispanic Students PBS Slope = Non PBS Slope = -,34

North Carolina Positive Behavior Support Initiative Partners’ Update February 2009 Heather R. Reynolds NC Department of Public Instruction Bob Algozzine Behavior and Reading Improvement Center

State PBS Coordinator Heather R Reynolds North Carolina Positive Behavior Support Initiative

Office discipline referral data (majors) from schools implementing PBS in North Carolina [07-08] compare favorably with national averages.

North Carolina Positive Behavior Support Initiative Levels of behavior risk in schools implementing PBS were comparable to widely-accepted expectations and better than those in comparison schools not systematically implementing PBS.

North Carolina Positive Behavior Support Initiative [A]chievement causes [B]ehavior? [B]ehavior causes [A]chievement? [Context causes [A]chievement and [B]ehavior?.

Steve Goodman

Goals 1. Share information about Michigan’s Integrated Behavior and Learning Support Initiative (MiBLSi) 2. Provide examples of improving the quality and quantity of the data collected 3. Provide examples of acting upon project data to improve outcomes

Participating Schools 2004 Schools (21) 2005 Schools (31) 2006 Schools (50) 2000 Model Demonstration Schools (5)

Project Data: Outcomes, Process and System Development

Major Discipline Referrals per 100 Students per Year (Schools implementing > 80% on Team Implementation Checklist)

DIBELS Instructional Recommendations and Major Discipline Referral per Cohort per Year DIBELS Benchmark Major Discipline Referrals

Participating School Example: Fourth Grade Reading MEAP Results Began MiBLSi Implementation

Improving the quality and quantity of project data

Percent of Process and System Data Collected by Cohort

Improving the accuracy and Consistency of Recording Office Discipline Referrals

Developing Fluency with Discipline Referral Categories Example Exercise 2: Match the example situation below to the correct problem behavior on the discipline categories answer sheet. Write the letter in the column for Exercise 2.

Acting on the Data to Improve Classroom Management

Major Discipline Referrals by Location Began focusing on classroom management support

MEAP- 4 th grade Reading Assessment 29 Elementary Schools In Michigan Schoolwide: Over 55% of major discipline referrals from classroom Schoolwide: Under 55% of major discipline referrals from classroom Probability of scoring below 75% proficiency on 4 th grade MEAP (Reading):.78 Probability of scoring above 75% proficiency on 4 th grade MEAP (Reading):.75

Improving Targeted Student Intervention Interviews with staff and self assessment indicate a need to develop targeted support systems

Checklist for Individual Student Support Systems (CISS) Results from Cohort 4 (n=34 schools)

Improving Targeted Student Intervention Strategies Building Leadership Teams: ◦ “Quick Sort” process for identifying students and linking to interventions Focused training for practitioners: ◦ Using Behavior Education Program (check in- checkout)

Supporting Coaches Conducting Self-Assessment to identify needs Providing support based on results ◦ Coach training 2 – 4 time per year ◦ Coach manual ◦ Coach website ◦ Coach conference (March 13-14)

Coaches Self-Assessment adapted from: Sugai, Todd and Horner, 2006

Example of the Impact of Coaching on Student Outcomes: Average Major Discipline Referrals per Day per Month Coach returns from leave

Working with School Teams to use data Data Review/Action planning with building leadership teams Pre-training coaches/principals in the data review content Providing worksheets to guide data review process

Team Evaluation of Outcome, Process and Systems Data

EBS Team Implementation Checklist by Feature

Lesson Learned Implementation cannot be faster than your school staff capacity to implement Teams need to be taught how to analyze and use data Emphasis on directing resources to need and removing competing activities

Building Capacity Statewide Districts need capacity to: ◦ Deliver regular training on core content  Annual orientation  Staff development ◦ Incorporate expectations in regular staff evaluations ◦ Provide expertise for more intense support need requirements.  Regular meetings with building personnel around “emerging challenges.”

Avoid passing the planning buck State asks districts to build a plan Districts ask schools to build a plan Schools ask teachers to build a plan

“Even if you’re on the right track, you’ll get run over if you just sit there” - Will Rogers