This product was developed by Florida’s Positive Behavior Support Project through University of South Florida, Louis de la Parte Florida Mental Health Institute funded by the State of Florida, Department of Education, Bureau of Exceptional Education and Student Services, through federal assistance under the Individuals with Disabilities Education Act (IDEA), Part B. Evaluating School-wide Systems

We Cheer You!

Why is Program Evaluation Important? To gain an understanding of how the program is functioning – “Are we really doing what we think we are doing?” To document program effectiveness – “Is what we’re doing working?” To identify and examine strengths and weaknesses of the program – Celebrate success – Identify areas to improve

Areas of Evaluation PBS Team Functioning/Effectiveness PBS Elements The SW Plan Implementation Outcomes Discipline & Academic Data Staff, Student, and Parent Perceptions

Evaluation Procedures 2 critical evaluation points Mid-Year End-of-Year

Mid-Year Report Coaches Mid-Year Checklist Coaches Self-assessment For Each School: – School Profile – Team Update – Team Process Surveys

Coach’s Mid-Year Report Checklist Directions: Submit this cover sheet along with the documents listed to your district coordinator before you leave for Thanksgiving Holidays (or earlier as determined by your district coordinator). Items 2-4 are to be completed for each school you serve. Coach Name:________________________ Date:___________________ District:_________­­____________________School Year:_____________ Mid-Year Items Item 1: Coach’s Self-assessment School 1: _________________________________ Item 2: School Profile Item 3: School Team Update (Completed by the coach) Item 4: Team Process Evaluation (Completed by each team member. Send completed forms; PBS Project will tabulate and return a summary report for the team.)

School-wide PBS Coach’s Self-assessment Coach Name:__________________District:________________Date:_____________ Skill 1 – learning 2 – building but not fluent 3 – fluent/mastered DATA 1. Understand and use the school behavior data system 2. Supporting the team in use of data to make decisions TEAM 1. Facilitating team meetings 2. Assisting teams in problem solving IMPLEMENTATION 1. Understand the critical elements of School-wide PBS

SCHOOL PROFILE INFORMATION Date of Report __________________District____________________ Contact Information Name of school: Address: Six-digit DOE district/school number: Telephone: Fax: Administrative contact:Telephone: Coach:Telephone: Type of School (please circle) Elementary Middle High Other (Center, etc.)

School-wide PBS School Team Update School Name______________________________________Date:_____________ Team Activity 1 – Not True 2 – Somewhat True 3 – Very True DATA 1. Data are entered regularly and are up to date 2. Data are used at PBS Team meetings 3. Data are shared with faculty regularly TEAM 1. Team uses written documentation (agenda and/or minutes) 2. Team holds meetings monthly 3. Team meetings are effective

School-wide Positive Behavior Support Team Process Evaluation Date:______________Name of School:______________________________ Frequency of Team Meetings:  weekly  bi-monthly  monthly  other _______ Average Length of Team Meetings: ______________ The following items relate to the functioning and effectiveness of the PBS team throughout the year. Please rate each item on the following scale: Strongly Disagree DisagreeNot SureAgreeStrongly Agree Not Applicable 12345NA 1.The team shares common goals. 2.The team has a common vision for the school. 3.All team members actively participate effectively in the process. 4.Each team member’s goals for the school are recognized throughout the process and planning NA

End-of-Year Report Coaches End-of-Year Checklist For Each School: – School-wide Benchmarks of Quality – School Team Update – Outcome Data Summary

Coach’s End-of-Year Report Checklist Directions: Submit this cover sheet along with the items listed to your district coordinator before the last teacher workday (or earlier as determined by your district coordinator). Coach Name:________________________ Date:___________________ District:_________­­____________________School Year:_____________ End-of-Year Items School 1: _________________________________ Item 1: Benchmarks of Quality Scoring Form (see instructions for completing) Item 2: School Team Update Item 3: Outcome Data Summary (If using SWIS, include a copy of end/year report) Item 4: Describe Staff Satisfaction Assessment (Give a written description of the method used and a brief summary of the outcomes. You are welcome to include a copy of any written survey.)

School-wide Benchmarks of Quality – Lists over 50 benchmarks of quality school-wide PBS programs that address 10 critical elements – Completed by school teams on a yearly basis to assess how they score on a 100 point scale with regard to developing and implementing school-wide PBS – Useful in developing action plans for following year – One of the measures used by the PBS Project to determine model schools

3 Elements of the Benchmarks of Quality Team Members Rating Forms – Complete and return to coach Benchmarks Scoring Form – Completed by coach using Scoring Guide – Used for reporting back to team Benchmarks Scoring Guide – Describes administration process – Rubric for scoring each item

School-wide PBS School Team Update School Name______________________________________Date:_____________ Team Activity 1 – Not True 2 – Somewhat True 3 – Very True DATA 1. Data are entered regularly and are up to date 2. Data are used at PBS Team meetings 3. Data are shared with faculty regularly TEAM 1. Team uses written documentation (agenda and/or minutes) 2. Team holds meetings monthly 3. Team meetings are effective

Outcome Data Summary Attendance Data Total number of school days Average daily attendance (%) Academic Data (School Accountability/FCAT) (if available as of June 1) Percent of students making learning gains in reading Percent of lowest 25% making learning gains in reading School grade Total school points If your school uses SWIS, you may stop here and attach the end of the year report. If you do not use SWIS, please complete the remainder of the form. Behavioral Data

Staff Satisfaction Assessment May be developed by the district to assess how well staff are implementing, and their satisfaction with the PBS process Sample forms may be available form the PBS Project for adaptation by schools May be included in annual climate surveys Reported on Coaches’ End-of-Year Report

Benchmarks Practice Directions: – Read through the materials presented on “ RAM Elementary – Complete a Scoring Form using the team member ratings and Scoring Guide

Other Evaluation Instruments Training Evaluation – Staff and Student Trainings on School-wide PBS – The results of those evaluations will be used to adapt or revise the training materials and approach to be most effective Systems-wide Evaluation Tool (SET) – Used in model school process – Used in validation study of Benchmarks of Quality Oregon School Safety Survey

Using Evaluation Results Improve and expand PBS – Implement PBS at other levels (targeted group, classroom, individual) – Support the acquisition of additional resources for further school improvement – Share with other schools/districts – Identify and celebrate successes – Identify areas that need improvement

Benchmarks Feedback Areas of Discrepancy Item # Team Response Coach’s Score Scoring Guide Description Areas of Strength Critical Element Description of Areas of Strength Critical Element Description of Areas in Need of Development Areas in Need of Development

Staff Satisfaction Results

Using Staff Feedback CONCERNS about PBS:CHANGES to PBS: Not enough serious consequences for actions/felt discipline was too lenient (5) · Administration is aware of concerns. · May be the result of not always knowing the consequences – improve communication with teachers through . Infractions weren’t taken seriously by students (3)  Refer to minors as “Classroom referrals” · AP will follow-up when parent contact is unsuccessful Market should be held more frequently (12) · Markets will be monthly, regularly scheduled in advance and listed on calendar (approx. every 4 wks)

Using Staff Feedback CONCERNS about PBS: CHANGES to PBS: Rewards too juvenile for intermediate students (3) · Reinforcer inventory will be given out to 3rd-5 th grades and suggested items purchased to include in the market. · Add more seasonal activities (ex. movie day, kickball game day, etc Need more faculty involvement/never felt whole school was part of the team (2)  Faculty/PBS team Communication liaison established · to send input on a daily basis...no longer have to wait until surveyed. · Channels of communication established (ex. PBS reps at primary/intermediate articulation meetings, instructional council, etc)

Action Planning for Coming Year Use combined results to identify ways improve the PBS process – Benchmarks - program elements – Staff feedback - issues of relative importance – Team process – effectiveness/efficiency Establish new Action Plan for coming year

Evaluation Resources School-wide Readiness and Evaluation Tools Chart Florida’s PBS Project Website District Coordinator

Evaluation of PBS Project TA Technical Assistance Survey – Mailed in Spring – Option for Anonymity