TEMPLATE DESIGN © DE-PBS Key Features Evaluation: Matching Philosophy & Measurement Sarah K. Hearn, M.Ed., Delaware Positive Behavior Support Project Development GoalsItem Breakdown Revisions & Findings to Date Contact Information Fully align external evaluation with DE-PBS Key Features Include qualitative & quantitative data Link to Professional Development Include multiple key perspectives in process Differentiate implementation levels of SWPBS to better describe schools and provide appropriate supports Revisions : Modify questions based on consistent scoring concerns Condense and streamline rubric areas Revisit rubric definitions, especially on essential items, to ensure clarity and support consistency Explore level parameters due to large percentage of Developing scores Findings: More schools reaching Proficient and Exemplary levels in Program Development 50% of schools at Exploring in Correcting Problem Behaviors = Identified need for additional professional development Development Process Rubric development to map out components of each of the 10 Key Features of School-wide Positive Behavior Support in Delaware Inclusion & expansion of some School-wide Evaluation Tool (SET) components Determine information sources Question development Peer review & rubric reorganization to match existing tools Spring 2010 – Question pilot Spring 2011—2013 – School-based pilot phases & revisions Phase 1 – 2 schools Phase 2 – 7 schools Phase 3 – 28 schools Rubric Samples On-site visit by DE-PBS Staff Sources of Information: Interviews with administrator, SW team leader, teachers/staff, students Review of SW PBS process documents Schoolwide observations Data: School Climate Surveys, DE Assessment of Strengths & Needs for PBS Use of iPads or Paper/Pencil Scoring & Reporting Information from multiple sources used to score all rubric components Scale = 3, 2, 1, 0 Per rubric, a set of items identified as “Essential Items” – Defined as key system elements & illustrate evidence of implementation fidelity Scoring based on Essential Items Reporting provided on all rubric areas Narrative summary organized by the 4 main evaluation areas which include: Areas of strengths Recommendations for improvement, professional development opportunities and resources Report review tool & action planning guide OPTIONAL LOGO HERE Tool Revisions: Analyze data set with all Spring 2013 data Finalize level system & notify schools Finalize all items and questions Testing for tool validity and reliability Finalize Filemaker application for iPad DE-PBS Project Plans Support schools to better implement the programs developed Focus on Correcting Behavior Problems in training and technical assistance – at district coach and school levels Explore implications for categorizing schools using new level system Evaluation Procedures School-wide PBS Tier 1: Program Development & Evaluation  Data  Problem-Solving Teams  Professional Development & Resources Prevention: Implementing Schoolwide & Classroom Systems  Positive Relations  Expectations/Teaching/ Acknowledgement  Safety Correcting Problem Behavior Developing Self Discipline Next Steps LevelParametersNameDefinition % of 2s and 3s earned on Essential Items Exploring Few elements of implementation 2 51%-75% of 2s and 3s earned on Essential Items Developing Early phase of implementation; some elements adequately in place % of 2s and 3s earned on Essential Items; No 0s earned Proficient Elements in place and implemented 4 100% of 2s and 3s earned on Essential Items Exemplary Implementation shows evidence of innovation and sustainability SW-PBS Implementation Levels – Under Refinement SectionTotal Items Essential Items School-wide PBS Tier 1: Program Development & Evaluation 188 Prevention: Implementing Schoolwide & Classroom Systems 179 Correcting Problem Behavior63 Developing Self Discipline73 Structure School-wide PBS Tier 1: Program Development & Evaluation Office Discipline Referral data pulled by a designated person, reviewed monthly by the School-wide team, and shared with entire staff. Source: Team Leader Interview - Does the PBS team have a designated data person who is able to pull/print reports for monthly meetings (Big 5)? Are ODR data collected and reviewed in summary form monthly by the PBS team? How often are the Big 5 data summaries shared with all staff during the school year? How are they shared? Prevention: Implementing Schoolwide & Classroom Systems Students are recognized for their good behavior (e.g., verbal praise, coupon, privilege), AND students can state the reason for the recognition. Source: Student Interview - Have you received any recognition from a teacher/adult in school for good behavior during the past week? If yes, why did you receive this recognition? Correcting Problem Behavior Correction is viewed not just as use of punishment or consequences but also as opportunity to help develop social problem-solving/decision-making skills of self-discipline and prosocial behavior. Source: Staff Interview - Beyond classroom management strategies, when correcting misbehavior, how do you help develop problem-solving & decision-making skills? Developing Self Discipline Social Emotional Lessons are infused throughout the school curriculum. Source: Staff Interview - Do teachers or other staff teach lessons intended to help develop social/emotional competencies? Levels by Evaluation Section N=22 Sarah K. Hearn –