Using Early Warning Data to Improve Student Outcomes

Massachusetts Department of Elementary & Secondary Education What are Early Warning Data and the Early Warning Indicator System (EWIS)? 2

Massachusetts Department of Elementary & Secondary Education 3 What are Early Warning Data?  General term that refers to data used to predict, diagnose, and monitor student risk for not meeting academic milestones  Early warning data includes the EWIS student risk levels and other data found in the EWIS reports  Also includes local data sources which include monitoring indicators and diagnostic data

Massachusetts Department of Elementary & Secondary Education 4 What is the EWIS?  Massachusetts Early Warning Indicator System (EWIS)  Created by the Massachusetts Department of Elementary and Secondary Education  Identifies students who are at risk of missing key academic milestones in grades 1–12  Provides student risk levels at the beginning of each school year  Student-level and aggregate EWIS information is available through five reports in Edwin Analytics

Massachusetts Department of Elementary & Secondary Education 5 EWIS Four Key Academic Milestones Grade Level GroupingsAcademic Milestones Statewide Results for Milestone Grades 1-3Reading by the end of third grade: Reaching proficiency or higher on 3 rd grade ELA State Assessment 57% Grades 4-6Middle school ready: Reaching proficiency or higher on 6 th grade ELA and math State Assessments 55% Grades 7-9High school ready: Passing grades on all 9 th grade courses 79% Grades 10-12High school graduation: Meeting all local and state graduation requirements 87%

Massachusetts Department of Elementary & Secondary Education 6 What are EWIS Student Risk Levels?  Students who attended a Massachusetts public school in the year prior is assigned a level of risk of low, moderate, or high  A student’s risk level indicates their likelihood of missing the key academic milestone for their grade level  E.g., A 5 th grade student that is high risk is considered to be off track for meeting the academic milestone of proficient or higher on the 6 th grade ELA and math MCAS  EWIS student risk levels are based on statistical formulas that are applied to multiple sources of readily available data from the prior year

Massachusetts Department of Elementary & Secondary Education 7 EWIS is / is not… == The Massachusetts EWIS is…The Massachusetts EWIS is not… A tool to better target interventions and supports at the individual, small group, and whole school levels. A “life sentence” or a reason to label students. A systematic way to flag students that need to be further examined to determine what additional support are necessary. A diagnostic tool that provides details on the exact reason(s) why a student is at risk for not meeting an expected academic goal. Related to measures that are included in the state’s school and district accountability system (PPI), but is not itself an accountability measure. An accountability measure. Based on data and risk models that the Department will reexamine each year to continually improve the system. A stagnant system that will remain exactly the same in future years.

Massachusetts Department of Elementary & Secondary Education 8 Local Early Warning Data  Monitoring indicators:  Rely on data that are available during the school year – such as attendance, academic performance, and behavior  Complement the use of EWIS student risk levels, schools/districts can use monitoring indicators to flag students who are at risk and track student progress at multiple times throughout a single school year  Diagnostic Data:  Supplemental data sources that can help schools and districts better understand underlying causes of student risk, and track student progress in interventions

Massachusetts Department of Elementary & Secondary Education EWIS Report Options in Edwin Analytics 9

Massachusetts Department of Elementary & Secondary Education 10 EWIS Reports in Edwin Analytics  There are five EWIS reports in Edwin:  EWIS District View (EW301)  EWIS Student List (EW601)  EWIS Graphical View (EW302)  EWIS Subgroup Analysis (EW318)  EWIS Risk Level Indicator Analysis (EW317)  All reports may be filtered numerous way (e.g., specific schools, grade levels, student subgroups)  EWIS Risk Level is also a filtering option in new classroom (teacher) reports

Massachusetts Department of Elementary & Secondary Education 11 Edwin Analytics: EWIS Reports

Massachusetts Department of Elementary & Secondary Education 12 EWIS Report: Sample Prompt Page 12

Massachusetts Department of Elementary & Secondary Education 13 EWIS District View (EW301)  Which grade levels or schools have the most students that need targeted support in order to be on track to meet the next academic goal?  What is the breakdown of high, moderate, and low risk levels in a particular student subgroup (e.g. among students with disabilities)?  Where does the district need to focus additional resources to get students back on track for meeting the next academic goal?  Tip: This report is a great first place to start reviewing EWIS data!

14

Massachusetts Department of Elementary & Secondary Education 15 EWIS Student List (EW601)  What are the current risk levels of my students?  What are the prior risk levels of my current students?  What data points (indicators) went into the calculation to determine the risk level?  Tip: Avoid the temptation to go straight to the Student List report when first reviewing EWIS data… it is usually more helpful to start with the other reports to get the lay of the land

Massachusetts Department of Elementary & Secondary Education 16 School Name Gr ad e Last Name First Name SASID Risk Level Attend Rate Susp ensio n Retain ed Scho ol Move Low Income SPED Level Gender Urban Reside nce Overag e Title1 School wide ACCES S for ELLs ELA MCAS 2014 Math MCAS 2014 Math Course Perf ELA Course Perf Sci Course Perf SS Course Perf Non- Core Course Perf Previo us Year Risk Level Race/E thnicit y ELL ELL Progra m Special Educat ion Placem ent Any High School 9AardvarkArnold High98%0No FreeNoneFemaleYesNoYes1.8--Pass N/AHispani c or Latino YEnrolled in sheltere d English immersi on Not a Special Educati on Student, age 6-21 Any High School 9BaileyBilly High90%0No FreeModerat e FemaleYes NoneNeeds Improve ment WarningPass Fail HighWhiteNN/AFull Inclusio n special educatio n services outside the general educatio n classroo m less than 21% of the time Any High School 9ColoradoCarlos High88%0No ReducedNoneFemaleYesNoYes2.6--Pass N/AWhiteYEnrolled in sheltere d English immersi on Not a Special Educati on Student, age 6-21 Any High School 9DentonDaisy High99%0No FreeNoneMaleYes NoneProficie nt Needs Improve ment Pass FailPassFailHighHispani c or Latino NN/ANot a Special Educati on Student, age 6-21 Any High School 9EverettElena High74%0NoYesFreeNoneFemaleYesNoYesNoneNeeds Improve ment WarningPass FailHighWhiteNN/ANot a Special Educati on Student, age 6-21

Massachusetts Department of Elementary & Secondary Education 17 EWIS Graphical View (EW302)  How does a selected group of students compare to the district and state?  What are the previous risk levels of the currently enrolled students over the past 5 years?  E.g., what are the previous risk levels of my current 4 th grade male ELL students?  What are the risk levels of a specific student group over the past 5 years?  E.g., what are the risk levels of 4 th grade male ELL students each year over the last 5 years?

18

Massachusetts Department of Elementary & Secondary Education 19 EWIS Subgroup Analysis (EW318)  How do EWIS risk levels within a school distribute over different race/ethnicities or gender?  Is there a noticeable difference in EWIS risk levels for certain student subgroups (i.e. SWD, ELL, Low Income, etc.)?  How do the EWIS risk levels of a particular subgroup of students compare to the state as a whole?  Are there certain groups of students who could benefit from additional services or interventions based on their risk levels?

20

Massachusetts Department of Elementary & Secondary Education 21 EWIS Risk Level Indicator Analysis (EW317)  How do attendance or suspension rates differ between risk levels?  Which courses do high risk students Pass or Fail at a consistent rate?  What are the areas of concern among high risk students that need to be further investigated?  Are there particular areas within a grade where universal (Tier I) changes may help students?

22

Massachusetts Department of Elementary & Secondary Education 23 Coming Soon! Expanded EWIS Predict readiness for postsecondary education 1.Enrollment : likelihood to enroll in postsecondary education 2.Developmental/Remedial: readiness for credit- bearing college courses 3.Persistence: likelihood for students to persist from the first to second year

Massachusetts Department of Elementary & Secondary Education Early Warning Implementation Cycle 24

Massachusetts Department of Elementary & Secondary Education 25 What is Early Warning Implementation?  Multi-step process that districts and schools can follow to effectively use early warning data to:  identify at-risk students  explore underlying causes for risk  provide interventions and supports  monitor student progress  make schoolwide and student level improvements  Tip: This type of multi-step process is a key foundation for implementing tiered systems of support

Massachusetts Department of Elementary & Secondary Education 26 Early Warning Implementation Theory of Action EWIS, monitoring indicators, and diagnostic data Early Warning Implementation Guide and other resources, training, and supports District and/or school-based team focused on using early warning data to support students COMPONENTS D Improved likelihood for students in need to receive apt supports and get back on track A Improved use of data B Improved assignment of resources OUTCOMES C Improved progress monitoring E Improved student success in reaching desired academic milestones INTERMEDIATE OUTCOMES

Massachusetts Department of Elementary & Secondary Education 27 6 Step Early Warning Implementation Cycle

Massachusetts Department of Elementary & Secondary Education 28 For more information…  Watch the Edwin website for updates, including materials and professional development opportunities:  Follow Edwin on Edwin_DESE  Questions can be submitted to:  If you need access to Edwin Analytics – contact your district directory administrator:  For info on how your school and district are doing on the four academic milestones…  3 rd & 6 th grade MCAS and PARCC results – Profiles:  9 th grade course passing & high school graduation rate – Success After High School DART:  Video highlights of Massachusetts schools and districts, created by the national Data Quality Campaign (DQC):

Massachusetts Department of Elementary & Secondary Education 29 Early Warning Implementation Guide  Section 1. What are Early Warning Data? Includes overview of EWIS & definitions of key terms  Section 2. The Six-Step Early Warning Implementation Process. Each subsection briefly describes the step, anticipated outcomes, and the tasks and processes schools and districts can use to implement the step  “Notes from the Field” anecdotes derived from Massachusetts districts and schools  “Build off Your Foundation” includes prompts that encourage schools and districts to build upon current practices  Each step includes with a set of guiding questions  Each step concludes with a checklist and other tools

Massachusetts Department of Elementary & Secondary Education 30 Early Warning Implementation Guide  Section 3. Additional Resources. This section provides information and links to additional Massachusetts and other resources that may be beneficial  Section 4. Appendixes. This section provides additional tools and information to support implementation and understanding of early warning data