1. BoQ Critical Element: Data Entry & Analysis Plan Established

2. Critical Element D: Data Entry & Analysis Plan Established 13. Data system to collect and analyze ODR data 14. Additional data collected (attendance, grades, faculty attendance, surveys) 15. Data analyzed monthly (minimum) 16. Data shared with team and faculty monthly (minimum)  Data are regularly shared with families (at least monthly)  Data shared is monitored for confidentiality  Data gathered is disaggregated by race and ethnicity

3. BoQ Scores by Element All Colorado PBIS Schools *as of 5/30/14

5. Effective Procedures for Dealing with Discipline Data System Definitions Referral Form Referral Process

6. What is an Office Discipline Referral (ODR)?  What it IS:  Kid - Staff Member - Administrator interaction  Underestimation of actual behavior  Piece of information used to make decisions  Data point

7. What is an Office Discipline Referral (ODR)?  What it IS NOT:  Punishment  A reflection on teacher’s skills  A way to change or re-teach behavior  A first attempt at correcting behavior

8. Why use ODRs in a PBIS school? Simply too cumbersome to collect all positive recognitions if given in the 5:1 ratio!

9. Characteristics of an Effective Referral Form The following categories must be included on the form:  Student’s Name  Date  Time of Incident  Student’s Grade Level  Referring Staff  Location of Incident  Problem Behavior  Possible Motivation  Others Involved  Administrative Decision  Other Comments

10. Office Discipline Referral (ODR) Forms  In formatting the referral form, you must make sure to answer the following questions: WhoWhy WhatWhen Where  Clarity on the referral form takes the guess work out of the data entry person’s job  Data will be more reliable and accurate as judgment calls are minimized

11. Some Examples…

12. Effective Procedures for Dealing with Discipline Data System Definitions Referral Form Referral Process

13. Data Driven Decision Making As coaches and ICs, we solemnly vow:  to never require meaningless data,  to help bring meaning to seemingly meaningless data,  to provide access to your data so you don’t have to worry about losing it,  to guide your PBIS team in the interpretation and presentation of data.

14. Reasons to Collect and Use Data  Essential for good decision-making  Professional accountability  Decisions made with accurate data are more likely to be: (a) implemented (b) effective

15. Improving Decision Making with Problem Solving Step 1—Define the problem What is the problem? Step 2—Problem Analysis Why is it occurring? Step 3—Plan Implementation What are we going to do about it? Step 4—Evaluate Is it working? CO MTSS Problem Solving Process

16.  Fidelity of Implementation Benchmarks of Quality (BoQ) School-wide Evaluation Tool (SET) Self-Assessment Survey Team Implementation Checklist  Progress Monitoring Office Discipline Referrals (ODR) Suspension/Expulsion Attendance Staff/Parent/Student surveys Two Primary Forms of Data: Outcomes: Is what we’re doing making a difference and benefiting students? Systems: Are we doing what we said we’d do?

17. Progress Monitoring for Behavior  Identify and use meaningful and efficient data sources  Determine trends and patterns  Implement interventions to identified need  Evaluate effectiveness of interventions  On-going

18. Considerations to Support Data-based Decision-making  Determine efficient collection procedures  Gathered continuously  Embedded as part of school cycle, not something “extra”  Easily accessible  Readable displays  Regular review  The people who collect and summarize the data see it is being used

19. Systems for Analyzing Data Utilize an established protocol for analyzing data:  Have clear decision rules for determining needs across all three Tiers  Review monthly with administrators and PBIS team  Share with staff at least quarterly (monthly is best)

20. What Data to Collect? Use what you have… Office discipline referrals/detentions Suspensions/expulsions Referrals by student behavior, staff behavior, and administrative context Office referrals per day per month Climate surveys Attendance Referrals to team responsible for problem solving around students needing more targeted and intensive support Referrals to special education programs

21. When to Enter Data Daily recommended, but at least weekly (at a minimum) With a school-wide data collection system, schools are better prepared to respond proactively to situations Schools can begin to identify problems and generate solutions Monitor and evaluate success of interventions

22. When to Analyze Data  Weekly or monthly (minimum)  Allows the team to see if specific interventions are working  Some data on particular students are reviewed more frequently for counseling and family partnering  Reviewing specific behavior incidents more frequently provides further clues regarding effective interventions

23. How to Analyze Data  How many referrals are there:  per day each month?  based on location?  based on the type of behavior?  by student?  by time of day?  by student motivation?  by ethnicity?  What is the range of consequences provided based on the type of behavior exhibited? “The Big 5”

24. Questions to Guide Data Analysis Every time the PBIS team reviews the “BIG 5” graphs, ask these questions… 1. What is going well? 2. Do we have a problem? If so, what is it? 3. Where? When? How often? Who? 4. What other information do we need? 5. What are our best guesses about what is happening? 6. What is our plan or intervention? 7. When and how will we evaluate the effectiveness?

25. Meaningful Data Can your current data system calculate the following……

26. Average Referrals Per Day Per Month

27. Referrals by Location

28. Referrals by Time of Day

29. Referrals by Problem Behavior

30. Referrals by Student

31. Referrals by Day of the Week

32. Referrals by Grade 32

33. Referrals by Motivation 33

34.  SWIS – School-wide Information System  Tracks discipline referrals for instructional decision making  Provides information for school-wide planning  Provides information for targeted group and/or individual student needs  www.swis.org www.swis.org A Brief Note about SWIS…

35. Using Data within Multi-tiered System of Supports FunctionLiteracy/ReadingBehaviorNumeracy/Math Universal Screening DIBELS/ EasyCBM SWIS; SSBD; SSRS Under Development Progress Monitoring DIBELS/ EasyCBM SWIS; CICO-SWIS; ISIS- SWIS Under Development Standardized Assessment and Accountability State Assessment Measures State Assessment Measures (Suspension/ Expulsion Data, Graduation Rates, etc.) State Assessment Measures SWIS Facilitator Training 2012

36. How does my school get SWIS?  Identify one SWIS facilitator per district  SWIS facilitator training in Denver (Nov.)  Talk to your regional TAC  Complete SWIS Readiness Checklist  Cost = $250 per school per year

37. SWIS Harassment/Bullying  Data collection to specify Harassment/ Bullying type  Supports data-based decision making and intervention planning around Harassment/ Bullying  School-wide  Individual Students  Example categories:  Racial  Sexual  Gender/Sexual Orientation  Ability  Cyber  Intimidation  Hazing  Other

38. Gibb, A.C. & Skiba, R. (2008). Using Data to Address Equity Issues in Special Education. Bloomington, IN: Center for Evaluation & Education Policy. “ Data are a critical first step in addressing sources of inequity in education: in order to solve a problem, we must first be able to describe it.”

39. SWIS Ethnicity Reports  High priority issue  SpEd specifically  “ Big 7” – adding Motivation & Ethnicity reports when working with teams  Ethnicity reports within SWIS allow us to progress monitor for prevention  CO PBIS Initiative will be offering webinars specifically around understanding Ethnicity reports

41. In addition to discipline data…. Look at % of students by ethnicity …  in AP & Honors courses  on Honor Rolls  with passing grades  in higher level math, setting course for MS & HS higher math and calculus  Attendance & Tardy rates  In SPED

42. Helpful Data  Mini-SET Mini-SET  Wheeling Data Wheeling Data  Become familiar with School View (on CDE website)  Evaluate overall academic performance and disaggregated by race & ethnicity  Know your school district’s graduation rates and also know disaggregated rates  Watch trends from year to year

43. ”The Law of the Scoreboard” John Maxwell  The scoreboard is essential to understanding. It provides a snapshot of the game at any given time.  The scoreboard is essential to evaluating.  The scoreboard is essential to decision making.  The scoreboard is essential to adjusting.  The scoreboard is essential to winning.

44. Team Time  Determine effectiveness of current data system and process  Review SWIS Readiness Checklist and determine what needs to be accomplished  Determine how data will be shared with staff and families (look at your Mini-SET!)  Complete action plan section (Data Entry & Analysis) identifying goals and tasks to be completed Outcomes

45. Team Time  Review your current Office Discipline Referral Form and check for compatibility with SWIS  Revise or create form to include SWIS- required categories  Determine how to present this discipline referral form to the staff  Complete action plan section (Discipline Procedures) identifying goals and tasks to be completed Outcomes

46.  What is the current data system used by your team? (ex. Infinite Campus, SASI, SWIS, Powerschool)  Can the data system be used to assist your team in making data-based decisions?  How often does your team look at the data?  What data does your team look at?  Do you think that the team understands how to use data to make decisions?  Can you access data disaggregated by race and ethnicity?  Is your discipline data an accurate reflection of what is happening across campus? Team Time Reflection Questions

47.  Can your Data System answer the following questions:  how many referrals are there per day each month?  how many referrals are there a based on location?  how many referrals are there based on type of behavior?  how many referrals are there by individual student?  how many referrals are there based on time of day?  how many referrals are there based on motivation?  What needs to change in your data system, entry or output to help your team make decisions? Team Time Reflection Questions

48. The contents of this training were developed under a grant from the US Department of Education, #H323A090005. However, these contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Project Officer, Jennifer Coffey, PhD.