1. Using Data for Decision-making
Rob Horner, Anne Todd, Steve Newton,
Bob Algozzine, Kate Algozzine

2. Main Ideas
Decisions are more likely to be effective and efficient when they are based on data.
The quality of decision-making depends most on the first step (defining the problem to be solved)
Define problems with precision and clarity

3. Main Ideas
Data help us ask the right questions…they do not provide the answers: Use data to
Identify problems
Refine problems
Define the questions that lead to solutions
Data help place the “problem” in the context rather than in the students.

4. Main Idea The process a team uses to problem solve is important:
Roles:
Facilitator; Recorder; Data analyst; Active member
Organization
Agenda; Old business (did we do what we said we would do); New business; Action plan for decisions.
What happens BEFORE a meeting
What happens DURING a meeting
What happen AFTER a meeting

6. Main Ideas Build “decision systems” not “data systems”
Use data in “decision layers”
Is there a problem? (overall rate of ODR)
Localize the problem
(location, problem behavior, students, time of day)
Get specific
Don’t drown in the data
It’s “OK” to be doing well
Be efficient

8. Using Data Do we have a problem?
Refine the description of the problem?
What behavior, Who, Where, When, Why
Test hypotheses
“I think the problem on the playground is due to Eric”
“ We think the lunch period is too long”
“We believe the end of ‘block schedule” is used poorly”
Define how to monitor if solution is effective

9. Problem Solving Foundations
Review
Status and
Identify
Problems
Team Initiated Problem Solving (TIPS) Model
Develop and
Refine
Hypotheses
Evaluate and
Revise
Action Plan
Collect
and Use
Data
Discuss and
Select
Solutions
Develop and
Implement
Action Plan
Problem Solving Foundations

10. Identifying problems/issues
What data to monitor
ODR per day per month
OSS, ISS, Attendance, Teacher report
Team Checklist/ SET (are we doing what we planned to do?)
What question to answer
Do we have a problem?
What questions to ask of Level, Trend, Peaks
How do our data compare with last year?
How do our data compare with national/regional norms?
How do our data compare with our preferred/expected status?
If a problem is identified, then ask
What are the data we need to make a good decision?

11. Total Office Discipline Referrals as of January 10

13. Using Data to Refine Problem Statement
The statement of a problem is important for team-based problem solving.
Everyone must be working on the same problem with the same assumptions.
Problems often are framed in a “Primary” form, that creates concern, but that is not useful for problem-solving.
Frame primary problems based on initial review of data
Use more detailed review of data to build “Solvable Problem Statements.”

14. Solvable Problem Statements (What are the data we need for a decision
Solvable problem statements include information about the five core “W” questions.
What is problem, and how often is it happening
Where is it happening
Who is engaged in the behavior
When the problem is most likely
Why the problem is sustaining

15. Primary versus Precision Statements
Primary Statements
Too many referrals
September has more suspensions than last year
Gang behavior is increasing
The cafeteria is out of control
Student disrespect is out of control
Precision Statements
There are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.

16. Primary versus Precision Statements
Primary Statements
Too many referrals
September has more suspensions than last year
Gang behavior is increasing
The cafeteria is out of control
Student disrespect is out of control
Precision Statements
There are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.

17. Precise or Primary Statement?
Children are using inappropriate language with a high frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school
James D. is hitting others in the cafeteria during lunch, and his hitting is maintained by peer attention.

18. Precise or Primary Statement?
ODRs during December are higher than in any other month.
Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by escape from work (but may also be maintained by peer attention… we are not sure).

19. Precise or Primary Statement?
Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention and possibly unsophisticated sexual expression.
Boys are engaging in sexual harassment

20. Organizing Data for Decision-making
Compare data across time
Moving from counts to count/month

21. 11/12/2018
SWIS summary (Majors Only) 3,410 schools; 1,737,432 students; 1,500,770 ODRs
Grade Range
Number of Schools
Avg. Enrollment per school
National Avg. for Major ODRs per 100 students, per school day
K-6
2,162
450
.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days
6-9
602
657
.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days
9-12
215
887
1.27 = more than 1 Major ODR per school day, or about 127 every 100 days K-
(8-12)
431
408
1.06 = about 1 Major ODR per school day, or about 106 every 100 days
On August 15 of every year, download the most current SWIS data summary from . Each of the examples might change given changes in the national average per 100 students per day
Newton, J.S., Todd, A.W., Algozzine, K, Horner, R.H. & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon unpublished training manual.
Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B

22. Start with the ODR/Day/Month Graph
Use the information in the data to build a narrative that draws the team into problem solving.
Be descriptive
Link local data to national patterns
Tie the data back to local conditions/events.

23. Elementary School 465 students (465/ 100 = 4.6 X .34= 1.56)
Our rate of problem behavior has been above the national average for schools our size for 9 of 10 months this year. There has been a decreasing trend since Dec.

24. Elementary School 1000 Students (1000/100 =10 X .34= 3.4)
The rate of problem behavior has been at or below the national average schools our size for 6 of 10 months. The past 4 months have been below the national average

25. Middle School 765 students (765/100 = 7.6 X .85= 6.46)
The rate of problem behavior has been at or below the national average schools our size for 9 of 10 months. The past 8 months have been below the national average with a decreasing trend

26. Describe the narrative for this school

27. Describe the narrative for this school

28. Describe the narrative for this school

29. Describe the narrative for this school

30. Using Data to Build Precision
Given that we know we have a problem
What problem behaviors
Where are they occurring
When are they occurring
Who is involved
Why do they keep happening

31. What questions to ask about the patterns of problem behaviors?
Do we have one problem behavior situations or more than one?
Do we have many problem behaviors or just a few problem behaviors?
Do we have clusters of problem behaviors?
What school wide expectations do we need to re-teach?

32. Data should allow asking the right question… not supplying the answer.
If many referrals in class
Which classes?
Which students?
When?
If many referrals in cafeteria
What times? (beginning or end of lunch period?)
What problem behaviors?

33. Disrespect is our most frequent problem behavior.
We also have incidents of fighting and harassment
What are next questions?
Who, When, Why?

34. What are next questions?
Our most frequent problem behavior is disrespect, followed by inappropriate language, disruption and tardy
What are next questions?
Who, When, Why?

35. We have many instances of disrespect, aggression/fighting
We have many instances of disrespect, aggression/fighting. technology violations, tardy, harassment, lying, skipping, and inappropriate language

36. What questions to ask about Referrals by Location
Where are the problems occurring?
Are there problems in many locations, clusters of locations, or one location?

37. Many problem behaviors in class
Many problem behaviors in unstructured settings (hall, playground, parking lot, bathroom)

38. Many problems in the cafeteria, hallway, common area, class, bathroom
Many problems in the cafeteria, hallway, common area, class, bathroom. Where is the ‘unknown’ location?

39. What questions to ask about Referrals by Time
When are the problem behaviors occurring?
How do those times match with the daily activities?
How does this information match up to Referrals by Location?

40. Most problems are occurring between 9:45-10:45.
Other problematic times are 8-8:45 and 11:30.

41. Most problems are occurring at noon

42. Many problems at 12:15, 7:45-8:30, 10:00-10:45

43. What questions to ask about Referrals by Student
What proportion of students has 0-1 ODR?
What proportion of students has 2-5 ODRs?
What proportion of students has 6+ ODRs?
Do we have systems of support that increase student success?

44. Student # 121 needs individualized support.
8 students are likely candidates for some type of Tier II support.
87% of our students have received 0-1 ODR

45. 14 students are likely candidates for some type of Tier II support
14 students are likely candidates for some type of Tier II support. Student #119 needs individualized support

46. 93% of our students have received no more than one ODR
We have 11 students who are likely candidates for some type of Tier II support
93% of our students have received no more than one ODR

47. What questions to ask about Referrals by Perceived Motivation
What is perceived as maintaining the problem behavior?
Are there one or more perceptions?

48. We have many incidents with unknown motivation
The problem behaviors are most likely maintained by task avoidance and peer avoidance.
We have many incidents with unknown motivation

49. Problem behaviors appear to be maintained by peer and adult attention

50. Problem behaviors appear to be maintained by escaping adult attention

51. Using the TIPS Minutes to guide Problem Solving
Write the precision problem statement in the left hand column below
Define a goal and write it in the right hand column

52. Using Data to Build Solutions
Prevention: How can we avoid the problem context?
Who, When, Where
Schedule change, curriculum change, etc
Teaching: How can we define, teach, and monitor what we want?
Teach appropriate behavior
Use problem behavior as negative example
Recognition: How can we build in systematic reward for desired behavior?
Extinction: How can we prevent problem behavior from being rewarded?
Consequences: What are efficient, consistent consequences for problem behavior?
How will we collect and use data to evaluate (a) implementation fidelity, and (b) impact on student outcomes?

53. Solution Development Prevention Teaching Reward Extinction
Corrective Consequence
Data Collection

54. Trevor Test Middle School
565 students
Grades 6,7,8

55. Trevor Test Middle School Is there a problem? If so, what is it?

56. Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)

57. Precise Problem Statement & Hypothesis Development
Many students from all grade levels are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attention
A smaller number of students engage in skipping and noncompliance/defiance in classes, (mostly in rooms 13, 14 and 18), and these behaviors appear to be maintained by escape.

58. Solution Development Prevention Teaching Reward Extinction
Corrective Consequence
Data Collection

59. Solution Development: For disruption in hall and cafeteria
Prevention
*Teach behavioral expectations in cafeteria
*Maintain current lunch schedule, but shift classes to balance numbers.
Teaching
Reward
Establish “Friday Five”: Extra 5 min of lunch on Friday for five good days.
Extinction
Encourage all students to work for “Friday Five”… make reward for problem behavior less likely
Corrective Consequence
Active supervision, and continued early consequence (ODR)
Data Collection
Maintain ODR record and supervisor weekly report

60. Using Data For Decision-Making
Phoenix Elementary
Using Data For Decision-Making

61. You are the Behavior Support team for Phoenix Elementary
You are the Behavior Support team for Phoenix Elementary. 265 students k-5
Do you have a problem?
Where?
With Whom?
What other information might you want?
Given what you know, what considerations would you have for possible action?

62. Examples Phoenix Elementary What is national comparison?
Absolute level compared with last year, compared with teacher/staff impressions, compared with family impressions, compared with student impressions.
Where, what, when, who , why
Hypotheses?
Solutions

63. Year 2
Year 1

64. Year 1
Year 2

65. Major ODRs Year 2 Only

67. Problem Statement Do we have a problem?
Build a precise problem statement

68. Solution Development Prevention Teaching Reward Extinction
Corrective Consequence
Data Collection

69. Langley Elementary School
478 Students
K-5

72. Precision Statement/Hypothesis
What
Where
When
Who
Why
What other info needed?
Possible Solutions?

73. Solution Development Prevention Teaching Reward Extinction
Corrective Consequence
Data Collection

74. Sandhill High school 354 students

75. Sandhill High School Is there a problem? If so, what is it?

76. Sandhill High School 02/01/2007 through 04/30/2007 (last 3 mos.)

77. Precision Statement/Hypothesis
What
Where
When
Who
Why
What other info needed?
Possible Solutions?

78. Solution Development Prevention Teaching Reward Extinction
Corrective Consequence
Data Collection

79. Summary
Establish information systems for decision-making… not data systems for reporting.
Use data to identify problems
Use data to clarify/refine problems
Use knowledge of context and content to build solutions.
Use data to monitor impact of solutions.