Using Data for Effective Team Meetings within PB4L Rob Horner University of Oregon

Goals Overview of Team-Initiated Problem Solving (TIPS) Using data to define a problem with precision Building comprehensive solutions Using data to monitor, evaluate and adapt

Team-Initiated Problem Solving Meeting Foundations Clear purpose and authority Efficient Process o Start, Stop, Schedule o Before/ During/ After o Respectful, Diligent Defined Roles o Facilitator o Minute Taker o Data Analyst o Active Member Electronic Meeting Minutes Problem Solving Define problem with precision o What, when, where, who, why, & how often o Define Goal that will guide solution building Build functional solutions o Prevent, teach, reward, consequences, safety, data use Build Action Plan for implementation Who will do what by when? Date collection plan for both fidelity and impact Monitoring & Adaptations * Use data to measure status of implementation and benefits to students * Adapt solutions and action plans in response to those data

Implement Solution with High Integrity Implement Solution with High Integrity Identify Goal for Change Identify Goal for Change Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution and Compare against Goal Monitor Impact of Solution and Compare against Goal Make Summative Evaluation Decision Make Summative Evaluation Decision Meeting Foundations Team-Initiated Problem Solving (TIPS II) Model Identify Solution and Create Implementation Plan with Contextual Fit Identify Solution and Create Implementation Plan with Contextual Fit Collect and Use Data What, Who, When, Where, and Why, and How Often? How do we want the problem to change? What are we going to do to bring about desired change? Did we implement with fidelity? Has the problem been solved? Maintain, Adapt, Stop?

General Flow of Meeting Call meeting to order – Who is present? Review agenda for today Discuss current status and previously defined problem(s) – Were solutions implemented? Discuss current data and relation to goal. Better? Worse? Was goal reached? What next? Discuss administrative tasks and any general issues Discuss any new problems – Identify precise problems, develop solution plans (what, who, when), identify goals, determine fidelity and outcome data needed Wrap up meeting – Review date/time for next meeting and evaluate present team meeting.

Problem Solving Define Problem o Identify problems early o Identify problems with PRECISION before building solutions Establish Goal o Where do we want to be Build Solutions that are comprehensive and locally relevant Prevent, teach, extinguish, reward, monitor Select solutions based on local strengths, culture, organization Build Action Plan for Implementing Solutions

Problem Solving What is a problem? o Any difference between what exists and what is desired. What its the difference between problems that are significant enough to warrant a solution and those that can be tolerated? o Social values o Academic expectations o Political and organizational expectations TIPS II Training Manual (2014) 8

Problem Solving Regular review of valued outcomes o Reading o Math o Writing o Behavior Compare current status with expected/desired status o Requires some “standard” for comparison Transform “basic problems” in to “precise problem statements” o Do NOT solve problems that are poorly defined. TIPS II Training Manual (2014) 84 Identify Problem with Precision Identify Problem with Precision

Problem Solving Define Problem o Identify problems early o Identify problems with PRECISION before building solutions Establish Goal o Where do we want to be Build Solutions that are comprehensive and locally relevant Prevent, teach, extinguish, reward, monitor Select solutions based on local strengths, culture, organization Build Action Plan for Implementing Solutions TIPS II Training Manual (2014) 10

Precise Problem Statement How often is a behavior occurring? Where is the behavior most likely to occur? What is the behavior of concern? Who is doing the behavior? When is the behavior most likely? Given Who, What, Where, When: o WHY does the behavior keep happening in this context? TIPS II Training Manual (2014) 11

Start with Primary Problem Statements Look at the Big Picture. Then use data to refine the problem to a Precise Problem Statement. Move to Precise Problem Statements Office discipline referrals for 3 rd graders are above national median for schools our size. Referrals for physical aggression among third grade students from 11:30-12:30 in the cafeteria are occurring at 2-3 per day and increasing over time. It is believed that this is happening due to insufficient supervision and access to peer attention.

Precise Statement? PreciseNot Precise Indicates a difference between what is happening and what is desired. What, Who, Where, When, Why, and How Often

Not Precise or Not Precise? Students on bus #43 are physically aggressive to gain peer attention at least 5 times per bus ride Three 5 th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention. This is occurring at least 5 times a week. Boys are engaging in sexual harassment.

Not Precise or Not Precise ? Darin uses inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school. Students are being aggressive on the bus.

Not Precise or Not Precise? Minor disrespect and disruption are occurring at a rate of 1.5 per day/month, ( increasing over time) and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7 th and 8 th grades, involves many students, and appears to be maintained peer attention.

Not Precise or Precise? Danielle is 9 years old and refuses to follow parental requests to pick up toys, help with dishes, or do homework. She will sulk and pout until we get frustrated and send her to her room. Danielle is 9 years old and 3-4 times per day refuses to follow parental requests to do minor tasks like pick up toys, help with dishes or do homework at home. She will sulk and pout. The behaviors are maintained by escape from non-preferred activities.

Your Turn to practice a. Make up a PRIMARY problem statement b. Transform that description into a PRECISE problem statement. Who What Where When Why How Often Write down Primary and Precise statements Be ready to share your hypothetical example with group 5 minutes

Using Data to Define Problems with Precision Major ODRs per day per month o Level o Trend o Pattern (peak months?) Compare with national median (or average) Compare with local norms TIPS II Training Manual (2014) 19

SWIS summary (Majors Only) 4858 schools; 2,476,989 students; 3,054,592 ODRs Grade RangeNumber of Schools Mean Enrollment per school Mean ODRs per 100 stud/ school day Median ODRs per 100 per stud/ school day 25 th Percentile ODR/100/ school day 75 th Percentile ODR/100/ school day K (.40) (.67) (1.04) PreK (.41) PreK (1.41)

Monitoring Data; Major ODRs per day per month Although the months are different, an average referral per day per month count allows for true comparisons. Average Referrals Per Day Per Month

Transforming Data into Information Tell the story that is present in your data. How often are problems occurring? Is the trend increasing, decreasing, stable over time? How does this level compare to last year? Are there months with unusually high levels (peaks)? How do our patterns compare with others? How do our patterns compare with prior years? How do our patterns compare with what our students, families and staff believe is appropriate/acceptable?

Do we have a Problem? Review ODR per day per month o What is the overall level? o What is the pattern? o What is typical? o What is possible? o What is needed? If there is a problem… then define with precision

Elementary School with 150 Students 24 Questions to Ask of the Data What is happening? What is typical? What is possible? What is needed? Questions to Ask of the Data What is happening? What is typical? What is possible? What is needed?

What is the narrative? Questions to Ask What is happening? What is typical? What is possible? What is needed? Questions to Ask What is happening? What is typical? What is possible? What is needed? 75 th Percentile Median 25 th Percentile

Elementary School Example Median

Elementary School Example Median

Middle School Example Median 765 Students

High School Example Median

Multi-Year Example Median Year One Year Two

What When Who Why Designing Effective Behavior Support Where Define Problems with precision TIPS II Training Manual (2014) How Often

Primary Problem Statements Data examined at the SW level or with just one graph allow for “primary” problem statements only, such as – Our school’s… o Average referrals per day per month for All Referrals & Minors are above the 75 th percentile for 5 of 6 months o Average referrals per day per month for Majors show an increasing trend o Referrals for Defiance totaled 77 instances of the last three months o Referrals in Classrooms have been too high for the last two months o Referrals for each month of this year are higher than for the corresponding month of the previous year TIPS II Training Manual (2014)

Implement Solution with High Integrity Identify Goal for Change Identify Problem with Precision Monitor Impact of Solution and Compare Against Goal Make Summative Evaluation Decision Meeting Foundations Team-Initiated Problem Solving II (TIPS II) Model Identify Solution and Create Implementation Plan with Contextual Fit Collect and Use Data What is the problem? Who? What? Where? When? Why? TIPS II Training Manual (2014)

Classroom Playground Where? Questions: 1.What location(s) are associated with the most ODRs? 2.Sort by “structured” settings and “non-structured” settings (Classroom & Gym vs. Commons, Cafeteria, Hall, Playground)

Where

Where

Defiance What Behavior(s) Questions: 1. Are most common behavior problems (a) Student- Student, or (b) Adult-Student related? 2. Are problem behaviors MAJOR or MINOR or BOTH?

What Behavior(s)

Who Questions: 1.Are there many, a few, or one student associated with the problem? 2.Are there more students emerging?

Who

11:45-12:00 When?When? Questions: 1.Are problem behaviors more likely at some times of the day? 2.What is happening during periods when problems are most likely?

When

When

When

When: Middle School

WHY?—the hardest question What is perceived as maintaining the problem behavior? Always assess motivation AFTER you have defined: o Who? o What? o Where? You always ask WHY students misbehave in the context. Look for “primary” motivation if there are multiple possibilities.

Why? Obtain positives o Attention from peers o Attention from adults o Personal satisfaction o Physical objects o Preferred activity Avoid negatives o Avoid peers o Avoid adults o Avoid work o Avoid undesirable activities o Avoid undesirable objects/food/ sounds/ etc Don’t Get too Big Revenge Prestige Power Status Happiness

Avoid Work Why?Why

Why Motivation for many elementary students engaging in disruption in the classroom

Why Motivation for middle school students’ non-compliance and insubordination in the classroom

Implement Solution with High Integrity Implement Solution with High Integrity Identify Goal for Change Identify Goal for Change Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution and Compare against Goal Monitor Impact of Solution and Compare against Goal Make Summative Evaluation Decision Make Summative Evaluation Decision Meeting Foundations Team-Initiated Problem Solving (TIPS II) Model Identify Solution and Create Implementation Plan with Contextual Fit Identify Solution and Create Implementation Plan with Contextual Fit Collect and Use Data

Identify Solution and Create Implementation Plan with Contextual Fit What and When

Implement Solution with High Integrity Implement Solution with High Integrity Identify Goal for Change Identify Goal for Change Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution and Compare against Goal Monitor Impact of Solution and Compare against Goal Make Summative Evaluation Decision Make Summative Evaluation Decision Meeting Foundations Team-Initiated Problem Solving (TIPS II) Model Identify Solution and Create Implementation Plan with Contextual Fit Identify Solution and Create Implementation Plan with Contextual Fit Collect and Use Data

Identify Solution & Create Implementation Plan with Contextual Fit Objectives o Use solution option categories to brainstorm solution actions for your precise problem statement o Define the scope of necessary solutions SW, specific setting, grade/group/individual student o Define an action plan for each solution action Who does what by when TIPS II Training Manual (2013) 56

How to Build a Solution Plan Use solution elements to guide discussion Brainstorm options for solution elements Select a set of actions (one plan) that o Require the least amount of effort that will produce desired changes o Strengthen what you already do well o Fit with the precision statement and priority o Are efficient & feasible o Are likely to have the desired impact TIPS II Training Manual (2013)

Solution Action ElementsSolution Action Elements Defined Prevent Focus on prevention first. How could we reduce the situations that lead to these behaviors? Teach How do we ensure that students know what they SHOULD be doing when these situations arise? Reward How do we ensure that appropriate behavior is recognized? Extinguish How do we work to ensure that problem behavior is NOT being rewarded. Correct How will you correct errors? Safety Are additional safety precautions needed? Solution Implementation Plan Elements TIPS II Training Manual (2013)

Solution Action ElementsPossible Generic Solution Actions Prevent What can we do to prevent the problem? Adjust physical environment. Define & document expectations and routines. Assure consistent & clear communication with all staff. Teach What do we need to teach to solve the problem? Explicit instruction linked to school wide expectations. Teach what to do, how to do it and when to do it. Model respect. Reward What can we do to reward appropriate behavior? Strengthen existing school wide rewards. Include student preferences. Use function-based reinforcers Extinguish What can we do to prevent the problem behavior from being rewarded? Use ‘signal’ for asking person to ‘stop’. Teach others to ignore (turn away/look down) problem behavior. Correct What will we do to provide corrective feedback? Intervene early by using a neutral, respectful tone of voice. Label inappropriate behavior followed by what to do Follow SW discipline procedures Safety Do we need additional safety precautions? Separate student from others if he/she is unable to demonstrate self- control. Make sure adult supervision is available.

How much effort is needed to bring about desired change? When deciding how many solution elements to implement consider: o Priority for change Severity of the problem Intensity of the problem Frequency of the problem Potential safety concerns o Impact of solution implementation Confidence that solution implementation will make a difference o Feasibility/ Availability of resources Professional development, support, time, tools Existing skills and capacities of implementers Is this doable? Is it the most efficient approach? TIPS II Training Manual (2013) 60

Is Solution Appropriate for the context? Solution actions should fit the context o Consider your school’s culture & expectations o Consider your school’s improvement plan goals o Consider the school calendar o Consider impact of solution implementation on staff work load & classroom instruction schedules Implementation time & effort need to be worth it by meeting the goal o Consider the function of the problem behavior TIPS II Training Manual (2013) 61

Your Turn Too many ODRs (3.4 per day) for aggression and fighting during 3 rd period recess by many boys (and some girls) wanting access to the playground equipment. Prevent?Extinction? Teach?Correction? Reward?Safety? Data System Needed?

Your Turn Precise Problem: Solution o PreventExtinction o TeachCorrection o RewardSafety o Data System

Action Planning What will be doneWho will do itBy When Data Collection 1.Did we implement the solution? 2.Did the solution benefit students?

Teacher Completed Fidelity Scales Establish a fidelity check routine that relates to Implementation A 1-5 scale is used for questions Up to 3 questions per week Share scale as a poster in the faculty room, an electronic survey, or paper and pencil We agreed to provide “high-five greetings” to all students entering your class on time in the morning this week? How did we do? No Yes How many days (we agreed to 4) during the week did you review with students the procedures for passing in the hall? ✓✓ ✓ ✓ ✓ ✓ ✓

Summary Solve Problems Build “meeting foundations” (roles, minutes, systems) Use data to define problems with precision o Define problems early Use data to build solutions Include “Action Plans” with each solution Use data to ask (a) did we implement, and (b) did the plan benefit students?