1. Early Warning Systems
Screening, Intervening & Progress Monitoring in High School
Jill Koenitzer & Michelle Polzin, Technical Assistance Coordinators, WI RTI Center
Jill Sharp, Principal, Wittenberg-Birnamwood High School
Kara Muthig, School Psychologist, Wittenberg-Birnamwood School District
The Wisconsin RtI Center/Wisconsin PBIS Network (CFDA #84.027) acknowledges the support of the Wisconsin Department of Public Instruction in the development of this PowerPoint and for the continued support of this federally-funded grant program. There are no copyright restrictions on this document; however, please credit the Wisconsin DPI and support of federal funds when copying all or part of this material
PBIS Forum 2014

2. Maximizing Your Session Participation
When Working In Your Team
Consider 4 questions:
Where are we in our implementation?
What do I hope to learn?
What did I learn?
What will I do with what I learned?

3. Where are you in the implementation process
Where are you in the implementation process? Adapted from Fixsen & Blase, 2005
We think we know what we need so we are planning to move forward (evidence-based)
Exploration & Adoption
Let’s make sure we’re ready to implement (capacity infrastructure)
Installation
Let’s give it a try & evaluate (demonstration)
Initial Implementation
That worked, let’s do it for real and implement all tiers across all schools (investment)
Let’s make it our way of doing business & sustain implementation (institutionalized use)
Full Implementation

4. Leadership Team Action Planning Worksheets: Steps
Self-Assessment: Accomplishments & Priorities
Leadership Team Action Planning Worksheet
Session Assignments & Notes: High Priorities
Team Member Note-Taking Worksheet
Action Planning: Enhancements & Improvements

5. Agenda Introduce Early Warning Systems (EWS)
Establish purpose for implementing Early Warning Systems (EWS)
The Early Warning System Process
Wittenberg-Birnamwood High School EWS experiences and lessons learned

6. Who is here today? Special Education Directors Pupil Services Teachers
Principals
Family Representatives
Interventionists

7. How familiar are you with
Fist to Five
Unfamiliar Somewhat Familiar Very Familiar
How familiar are you with
Early Warning Systems?
How familiar are you with Early Warning Systems?

8. Wisconsin’s Vision in detail...
Early Warning Systems (EWS)
Overlay animation PBIS slide over RtI graphic
Purpose: Early Warning Systems help us collect and use integrated behavioral and academic student data that is part of a balanced assessment system.

9. Early Warning Systems
Early warning systems (EWS) rely on readily available existing data housed at the school to:
Predict which students are at-risk for dropping out of high school or not moving to next level
Target resources to support off-track students while they are still in school, before they drop out
Predict students who are not performing up to ability or are not college & career ready
Examine patterns and identify school climate issues
Jill Sharp Note….some kids taking courses that are too easy for them and not challenging themselves…..

10. Existing Data Considerations
How do we identify students who are not likely to graduate?
How do we identify students who are not college and career ready?
What questions do we answer when using existing data?
Read slide.

11. Attendance Behavior Coursework Risk Indicators—Use your ABCs
Screening measures
Risk Indicators—Use your ABCs
Attendance
Attendance/tardies
Chronic absenteeism
Mobility
Engagement
Participation
Behavior
Social-Emotional
Office referrals
Suspensions
Behavioral screening
Internalizing behaviors
Developmental assets
At-risk support
Behavioral plan
Family stressors
Coursework
Academic screening
Common assessments
Standardized testing
Grades
Retention
Accelerated learning
Interventions, Title I
Head Start, Summer School
Not static data, factors we influence
Bold ones are power indicators
Sometimes other measures explain why a student is not performing at or above benchmark. Using an ABCD method may help remind your team to dig deeper and look at multiple measures. A represents attendances, which is crucial to student success and is a measure of engagement with school. B represents behavior. C represents performance on coursework and in class performance.
Attendance is one area that schools have a wealth of existing data. While traditionally, we have used truancy as a measure for engagement and schools have interventions in place for truancy, the research from Dr. Robert Balfanz (The Importance of Being in School: A report in Absenteeism in the Nation’s Public Schools) indicates that Chronic absenteeism is a far more responsive and preventive cut score for attendance.
Chronic absenteeism is defined as any student who misses 16% or more of school for any reason. (Wisconsin DPI Tech Guide)
Attendance is a measure of student engagement, and according to a recent Gallup poll, student engagement with school drops every year. 8 in 10 elementary students qualify as engaged, 6 in 10 MS students, and 4 in 10 HS students (Truancy and Attendance: Interventions for High School). In fact, the best predictor for students dropping out of school is frequent absenteeism for any reason. Students who are engaged are less likely to drop out.
According to Allensworth and Easton, Attendance is the best predictor of school success.
Behavior as measure of social emotional needs is generally measured using office discipline referrals, suspensions, expulsions or other qualitative measures. Don’t forget behavior may or may not be the precursor to a student performing below benchmark. The student struggling with academic skills may be the precursor to a below benchmark performance on behavior measures like office discipline referrals. Distinguishing between the two is important to student success. Behavior is not the focus of this training, but you need to consider how behavior impacts academics.
Coursework, as a further measure of academic skills or social emotional needs, is another multiple measure to look at. You can consider additional screening measures or existing course grades, WKCE performance, other measures. Retention as monitored via an overage report is a crucial piece of data to have for examining academic deficits. According to Roderick in Phi Delta Kappan, 75% of students retained in K – 3 drop out of school, and 90% of students retained in 4 – 6 grade drop out of school.
AIMSWeb, EasyCBM, FAST, MAP, or…
Fountas and Pinnell Benchmark Assessments, or…
Reading or Math series benchmark assessments, or…
Locally-developed benchmark assessments
*See handouts : Predictors of Post Secondary Success and Risk Indicators for Not Graduating

12. Establishing Purpose for Implementing EWS
Michelle slides

13. Show of Hands or Percent of Americans with a high school diploma 85%
Percent of students who repeat 9th grade that graduate
15%
Percent of crimes in U.S. committed by a high school dropout
75%
Percent of black dropouts that have spent time in prison
60%
Percent of Hispanic dropouts that were due to pregnancy
41%
Percent of U.S. jobs a high school dropout is NOT eligible for
90%
Which group has a higher dropout rate:
Black
Hispanic
93%
35%
53%
90%
25%
75%
9.6%
17.6%

15. 4-Year Graduation Rates by Student Groups in 4 Large Cities
Note that while we as a state are focusing in on African American because of our status of being worst in the nation for achievement gap between Caucasian and African American, we need to pay close attention to the graduation rates of Hispanic students as well.

16. Unemployment & Work Force Participation by Education Level 2011

17. 2014 Federal Poverty Levels— Monthly Income
Before we go to income based on level of education, let’s visit the federal poverty level guidelines for annual and monthly income for a point of reference….
Source: Wisconsin DHS

18. Average Estimated Income for Wisconsin Workers 2011
Approx.
$12,000
Difference
An extra $1000 per month makes a huge difference in quality of life!!!

19. Incarceration Averages Applied to Wisconsin’s Prison Population
Estimated Incarceration if Dropouts were Graduates
Wisconsin has 354,411 males between the ages of 16 and 24. If you limit this group to men aged in order to discount struggling students that may still return to school, you end up with 285,749 adults.
Of these young men, 40,588 have no diploma. If you apply Sum's imprisonment averages across this group, you end up with 3,789 likely inmates who are dropouts between the ages of 18 and 24.
Unfortunately, Sum's study doesn't report on the incarceration rates of high school graduates. For that, we'll have to use Lochner and Moretti's estimates. This is an imperfect comparison thanks to the specific age groups that are singled out and the difference in time frame for data collection, but it should still be enough to provide a conservative estimate of how many fewer inmates between the ages of 18 and 24 Wisconsin would have if all students graduated.

20. Medicare Coverage by Education Level in Wisconsin
Dropouts are 200% more likely to need Medicare and 4x as likely to require government assistance (i.e.: welfare, food stamps, subsidized housing, etc….).

21. Potential Savings to Wisconsin Tax Payers if Dropouts were Graduates
Imagine the good we could do with that kind of money!!
K-12 Education Funding
College Tuition Reduction
Career Training Programs
Based on 2011 Graduation Rates

22. Agenda 2017: Wisconsin DPI We know we need to change! Outcomes
We have an agenda in Wisconsin to ensure every student graduates on time and is ready for college or a career when we send them out into the world.
We know we need to change!

23. Current Reality vs. Desired Future
Academic & Behavioral Systems
Do the math:
Are 80% of students responding to the Universal System?
Are you set up to provide Tier 2/Selected Interventions to 15% of your student body?
Tier 3/Intensive supports to 5%?
1-5% Tier 3/Tertiary Interventions
Individual students
Assessment-based
Intense, durable procedures
5-15% Tier 2/Secondary Interventions
Some students (at-risk)
High efficiency
Rapid response
Small group interventions
Some individualizing
Based on info from the national center for PBIS, in any given year, we can expect that up to 15% of any student body would benefit from a targeted, group intervention, and only 5 % would demonstrate need for a Tier 3, highly intensive intervention.
Look at your numbers; what is 15 % of your student body? What is 5 % of your student body?
2 minutes talk with a shoulder partner
Group share out:
Any Ahh Ha’s or Oh No’s?
80-90% Tier 1/Universal Interventions
All settings, all students
Preventive, proactive
Illinois PBIS Network, Revised May 15, Adapted from “What is school-wide PBS?” OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. Accessed at

24. Getting Started with Early Warning Systems
Download
The
Guides

25. Follow
7-Step
Process

26. Establish Roles and Responsibilities
STEP 1
Can be a new or pre-existing team
Team must receive PD to use tool effectively
Early identification of students at risk
Assign interventions & monitor progress
Meet at regular intervals
within first 20 days of school
after every grading period (i.e.: 8x/year)
The EWS team may be established as a new team or may build on or be integrated into existing teams (e.g., school improvement team, response to intervention team, student support team).
It is not necessary to create an entirely new team for EWS work, but an existing team that takes on the responsibility to use the tool for dropout prevention efforts should include a broad representation of staff within the school and, ideally, the district (e.g., principals, teachers,district administrators, specialists).
Additionally, the EWS team must receive professional development on the process and EWS Tool v2.0 and subsequently be given adequate time to implement the EWS process.
it is vital that the EWS work be a main priority of the designated team.
EWS teams should meet regularly. At a minimum, EWS teams should meet after the first 20 or 30 days of school and shortly after the end of each grading period. (example: mid-quarter and quarter, so 8x per year).The focus of EWS team meetings is to review and discuss the information available in the tool, particularly about individual students who have been identified as at risk for dropping out.
The EWS team considers interventions for each identified student and then continues to use the EWS data to closely monitor students to ensure that the assigned interventions are adequately supporting each student.
Continuous monitoring of students who display indicators of risk will improve the EWS team’s ability to match appropriate interventions to these students and will allow mid-course corrections if a particular student does not seem to improve after being assigned to an intervention.
Can be used in conjunction with Intervention Tracking Tool, which is introduced in later session.

27. Download and Use the EWS Tool
STEP 2
Set up thresholds for flagging students
Download available student data
Ensure data is entered at regularly scheduled intervals
Reports available and accessible to all EWS team members

28. Localize Risk Indicators in the Tool Settings
Update with Wittenberg’s tool settings

29. STEP 3 Review the EWS Data
Understand patterns in student engagement and academic performance
Identify students at-risk for dropout
Organize and sort students into groups based on risk factors
Consider allocation of resources for flagged students (needs vs. available interventions)

30. Find, Organize and Sort Students

31. STEP 4 Why? Interpret EWS Data
Dig deeper into complex causes of student disengagement and academic failure
Examine additional information not included in EWS tool
Additional teacher input
Additional assessments
Student & parent interviews
Why?
Michelle slides 32-43
indicators of risk are merely signs of deeper and likely more complex problems related to student disengagement with school and academic failure.
look more closely at the characteristics of flagged students by examining additional data that are not included in the EWS Tool v2.0 but are available in other information data systems or from individuals who interact with these students.
Additional teacher input
Assessments to dig deeper
Student interviews
Parent interviews
Remember the problem solving steps….Who, What, Where, When and Why???? Step 4 improves the team’s understanding about why students are displaying indicators of risk. If we understand WHY, we are better able to provided effective interventions

32. Assign & Provide Interventions
STEP 5
Map interventions from school, district and community
Organize interventions into tiered levels of support and determine entry/exit decision rules
Match students to interventions based on need
What You Need for Step 5
• An understanding of students needs in the school, by priority, based on the review of EWS data and additional information
• An inventory of available interventions
• Leadership buy-in and support for interventions and strategies to assist off-track students
Other Resources
• Appendix A–2. Dropout Prevention Intervention Mapping

33. Create a continuum of support
(ordered by intensity of intervention & effort needed to implement)
Tier 1/Universal
School-Wide Assessment
School-Wide Prevention Systems
Drop-in Tutoring
Check-in/ Check-out (CICO)
Tier 2/ Secondary
Tier 3/
Tertiary
Lunch & Learn
Social/Academic Instructional Groups (S/AIG) Academic Seminar
Academic Seminar
Group Intervention w. Individualized Feature (e.g., Check and Connect and Mentoring)
Formal Tutoring
Take a look at these interventions. School-side PBIS, the anchor for all further work with children is done by ALL – all staff, all students, all environments. The interventions that you see on the this triangle are the evidence based interventions that we train in tier 2 and 3. If your clinical staff is responsible for all of these interventions, along with all of their other responsibilities, they are going to burn out. Fast.
The lower on the triangle the intervention, the more expertise and time needed to provide, and the smaller the number of students who will need to access. THUS, you will need to use your resources wisely, when planning for tier two and three. Are there people in the school who “get kids” and would have a flexlible enough schedule that they could be trained in an intervention, and freed up for a bit of time each day or week, to provide the intervention? Who would be the best fit for facilitating a complex FBA/BIP or a WRAP plan?
Small Group Comm. Arts
or Math Interventions
Brief Functional Behavior Assessment/
Behavior Intervention Planning (FBA/BIP)
Read 180
Complex or Multiple-domain FBA/BIP
Adapted from Illinois PBIS Network, & T. Scott
PBIS Forum 2014
Wraparound

34. Intervention List & Descriptions

35. Monitor Student Progress
STEP 6
Monitor students who are participating in interventions
Determine effectiveness of intervention overall for groups of students
Identify student needs that are not being met
Identify new interventions to meet student needs
Communicate with and engage family members
The team has an ongoing process for reviewing student intervention data.
Student response is compared to guidelines for success (decision rules)
Interventions are evaluated to determine if they are effective for student population and if not, why…
Not correct intervention for students
Fidelity of implementation issues
Research and create new interventions to meet needs of students who are not responding positively

36. Evaluate and Refine EWS Process
STEP 7
Team evaluates EWS process on annual basis
Evaluate risk indicators & thresholds
Evaluate decision rules & interventions
Evaluate impact with student outcome data
Reflect on strengths and challenges
Make recommendations for improving the process

37. Examples of Multi-Year Data

38. Establish Roles and Responsibilities
Step 1
Establish Roles and Responsibilities
Step 3
Review the EWS Data
Step 4
Interpret the EWS Data
Step 5
Assign & Provide Interventions
Step 6
Monitor Students
Step 7
Evaluate & Refine EWS Process
Step 2
Use the EWS Tool

39. Wittenberg-Birnamwood High School
Early Warning Systems Implementation

40. WBHS Demographics
Wittenberg & Birnamwood are rural communities in central Wisconsin.
WBHS enrollment has been declining over the past five years. Current enrollment = 376
88% White, 5% American Indian, 3% Hispanic
35% Low SES
15% Students with disabilities

41. WBHS History We had a lot of data but didn’t use it effectively
Learned that we had to schedule collaboration and not assume that staff knew how to do this
Having a decision making model has helped us to focus rather than jumping from initiative to initiative
Sixth year of PBIS. Has become part of the culture of our school.

42. District Goals
All students will improve literacy skills in reading and writing across the K-12 curriculum in alignment with the common core standards
All students will improve math literacy skills through the study of and alignment with the common core standards
The District will improve student learning with aligned curriculum, assessment, and instruction.
The District will ensure that students have a safe, welcoming and productive learning environment.

43. EWS Connection to District Decision Making Model

44. WBHS Goals & EWS
The use of the EWS High School Tool and data wall to provide a resource framework that identifies students that are off track for graduation or underachieving, so that staff-designed, evidence-based programs and practices, based on data, can be put into place early and systematically in order to prepare all students for post-secondary and career readiness.

45. Teams
STEP 1
Teams were chosen with the intent to represent all departments.
9th Grade Team: Principal, School Counselor, Math Teachers, ELA Teacher, Science Teacher, Health/PE Teacher, Social Studies Teacher, Special Education Teacher, School Psychologist
10th Grade Team: Principal, School Counselor, PE Teacher, Math Teacher, Social Studies Teacher, Science Teacher, ELA Teacher, Specialist, Special Education Teacher, School Psychologist
11th Grade Team: Principal, School Counselor, PE Teacher, Math Teacher, Social Studies Teacher, Science Teacher, ELA Teacher, Specialist, Special Education Teacher, School Psychologist

46. Teams
STEP 1
Norms: Stay on task, Start and end on time, Everyone contributes, Non-judgmental conversations, Comments are solutions-based
Roles: Principal facilitates meetings, but all participate.
When: Teams meet monthly during the school day for two class periods each. Substitute teachers cover classes.
Where: Teams meet in the data room.

47. STEP 2 Using the EWS Tool Who enters the data What data is entered?
How are students flagged?
How is the data presented to the team?
Data is downloaded into the EWS tool prior to every team meeting by Dean of Students
Data sheets are handed out on confidential data sheets at each team meeting
Data is also visually represented on data wall in locked data room
Student Flags

48. STEP 3 Reviewing the Data
Helps group stay on track about talking about groups of students instead of individual students….individual kids are discussed at a different time (morning meetings). Hitting a bigger percentage of students by using protocol

49. STEP 3 Reviewing the Data Regularly Review & Update: MAP scores
ACT Aspire
Smarter Balanced
WKCE scores
DEWS scores
Attendance
Behavioral referrals
Grades
Connection survey
*Will also be using ACT data at end of year

50. STEP 3 Reviewing the Data Information on student data cards:
Special Education
ELL
Booster/Intervention Classes
Attendance Risk
Behavioral Risk
Grade/Credit Risk
DEWS Risk

51. STEP 3 Reviewing the Data Who is at risk? Team looks for patterns.
First we look at whole group and small group concerns
Next we look at individual student concerns
Team focuses on MAP scores after current benchmark dates
Team focuses on grades/attendance/behavior in-between benchmark assessments

52. STEP 4 Interpreting the Data
Team discusses individual and/or groups of students who are at risk. Staff share any additional information that may help with decision making.
Parents are often contacted when discussing risk of individual students.
School counselor or school psychologist may interview the student to problem solve.
indicators of risk are merely signs of deeper and likely more complex problems related to student disengagement with school and academic failure.
look more closely at the characteristics of flagged students by examining additional data that are not included in the EWS Tool v2.0 but are available in other information data systems or from individuals who interact with these students.
Additional teacher input
Assessments to dig deeper
Student interviews
Parent interviews
Remember the problem solving steps….Who, What, Where, When and Why???? Step 4 improves the team’s understanding about why students are displaying indicators of risk. If we understand WHY, we are better able to provided effective interventions
*Use caution in assigning tasks to pupil services staff

53. STEP 4 Interpreting the Data Examples:
Team dug deeper into MAP scores of students near the benchmark. The team found a pattern that almost all of those students struggled most with the strand “Informational Text”.
Team hypothesized that some students might not be trying their best on the MAP test and therefore scores may be inaccurate of true skills.
Team hypothesized that individual students are struggling with mental health issues, relationships, connections at school, AODA issues, skill deficits, etc.
indicators of risk are merely signs of deeper and likely more complex problems related to student disengagement with school and academic failure.
look more closely at the characteristics of flagged students by examining additional data that are not included in the EWS Tool v2.0 but are available in other information data systems or from individuals who interact with these students.
Additional teacher input
Assessments to dig deeper
Student interviews
Parent interviews
Remember the problem solving steps….Who, What, Where, When and Why???? Step 4 improves the team’s understanding about why students are displaying indicators of risk. If we understand WHY, we are better able to provided effective interventions

54. STEP 5 Assign & Provide Interventions Tier 1: Core curriculum
District RtI Team
WBHS PBIS Team
Daily Shout Outs
8th Period Advisory (IE System)
Tiers 2 & 3:
CICO
Individual & Small Group Counseling
Mentoring
Academic Booster/Intervention Classes (8th period)
RtI/PBIS Tier 2&3 Meetings
Descriptions of IE System
Non-negotiable interventions
Teachers made decision rules about who goes into IE….Jill share document she used to prompt thinking in teachers for creating decision rules (started with flagging decision rules)

55. STEP 5 Assign & Provide Interventions Entry/Exit Decision Rules:
Interventions during IE (Teacher created)
CICO (SWIS Data and Attendance)
Counseling (Parent, Teacher or Self Referral)
Mentoring (EWS and Tier 2/3 Teams assign based on data)
Do we have enough available interventions to meet all student needs?
Would like to do more mentoring
Would like to do more small group counseling
Shortage of mental health resources in the community
Everyone is engaged
Teachers are reteaching, giving time to complete tests
Students all have higher percentage of engaged time

56. Intervention/Enrichment (I/E)

57. Enrichment Classes Use to create exposure Decision Rules
Teacher proposals for enrichment classes
Enrichment Calendar
Clubs & Organizations
Parents have access…live on website

58. Enrichment Calendar

59. STEP 6 Monitor Student Progress Data is collected throughout the year.
We need to work on adding progress monitoring tools for interventions (still working on creating…both for progress of individuals as well as the intervention as a whole)
Academic Behavior Measures
Decision rules/rubric for intervention classes
Need decision rules for non-academic interventions
Path Driver
Teacher Buy-In….There hasn’t always been a direct connection between the PM assessment results and what they are teaching
The team has an ongoing process for reviewing student intervention data.
Student response is compared to guidelines for success (decision rules)
Interventions are evaluated to determine if they are effective for student population and if not, why…
Not correct intervention for students
Fidelity of implementation issues
Research and create new interventions to meet needs of students who are not responding positively

60. Data Wall—Student Growth
Fall 2013:
Targeted middle group
Fall 2014:
Evidence of growth

61. Data Wall—Fall 2014
Fall 2014: Freshman
PBIS Forum 2014

62. STEP 7 Evaluate & Refine Process
Team evaluated EWS process end of year
Evaluated risk indicators & thresholds
Evaluated decision rules & interventions & made changes
Evaluated impact with student outcome data
D/F list
MAP Scores
Attendance
Growth on School Report Card
Reflected on strengths and challenges

64. Evaluate & Refine Implement an E/I time 8th period
Increase instructional time: move meetings to after school, investigate alternative scheduling options (block, modified block, etc.)
Continue to work toward core plus more for ALL students below grade level
Refine the art and science of teaching through deeper investigation of the GRRM
Learn - attend institutes, conferences, read, collaborate (EE)
Reflect - evaluate ourselves and identify weaknesses so that we can continue to improve professionally (EE)
Expand EWS so that all grade levels meet and review data (Request from staff to add 11th grade meetings for )

65. Lessons Learned Things take time
Build a culture of meeting kid’s needs
Know your staff
Give all staff a voice in making decisions and creating the system supports
There are things you have to give up in order to do something different
Communication is key
Use what didn’t work and learn from it to change for the better
There are a lot of things done that work, but when things don’t work, that is okay…just learn from it and change what you are doing

66. Lessons Learned
Staff perceptions of impact of system changes with PBIS when connected to bigger picture
Connect to Life Skills
Not just about how to go through lunch line
We have to TEACH kids what academic behaviors look like

67. What are our teachers saying?
There is still never enough time
We need to include all staff somehow
Would like to focus more on groups of students rather than individuals
We will have more data as we move forward about effectiveness of EWS
Staff feel more ownership
Systems approach is more effective
Better awareness of student needs due to data and collaboration
Problem solving is positive

68. Early Warning Systems Wittenberg-Birnamwood High School
Jill Koenitzer,
Michelle Polzin,
Jill Sharp,
Kara Muthig,

69. Early Warning System Resources
National High School Center
Early Warning Systems in Education
YouTube Channel webinars: NHSCenterMedia
PBIS Forum 2014