A Briefing for the Bill & Melinda Gates Foundation By: Mary Beth Celio Northwest Decision Resources September 9, 2009 AMDG

A brief review of the goals of the cohort study To address the reality that many young people leave high school without a diploma, and thus enter adulthood with a handicap, the Seattle ‘06 Cohort Study was designed to: Develop middle school/high school early warning indicators—the best combination of student characteristics to be used to predict withdrawal from high school without a diploma, Identify ‘tipping points’ –critical times/events that predict imminent withdrawal from school, and Segment the potential dropouts according to the nature and timing of indicators so that interventions can be tailored and targeted. 2

Why develop Seattle-specific tipping points, early warning indicators and segmentation? Early warning indicators Why? Can identify the level of dropout risk for students from 6 th grade up so that preventive or remedial programs can be designed according to the components of risk. What? Student academic, behavioral or personal characteristics in middle school or high school that, together, are strongly predictive of dropping out of high school. Tipping points Why? Can be used to build “triggers” into the data system to notify school personnel of students who are close to a critical decision point What? Any events during the middle school or high school years that have been found to signal imminent withdrawal from school. Segmentation Why? Can help design and target services and programs to students who need assistance, when they need it. Who? Early Strugglers, HS Off-track, In-place dropouts – and the Unpredictables. 3

First step: Identifying the cohort Start with a comprehensive data base containing all available personal, academic and behavioral variables for students scheduled 6,905 students met all the requirements for initial cohort inclusion 5,241 students were eligible for analysis Determine an outcome for each student in the cohort based on the (sometimes conflicting) information available Students who transferred out of district schools and did not return were dropped from the cohort at the time they left the district for the final time Regular graduates (on-time or within 2 years of expected graduation) Non-graduates: Students in the cohort who dropped out of school and were recognized as dropouts by SPS Students who left the school system after attending four or more years without earning a diploma Students who earned a GED Students who left the school district without providing evidence of transfer to another out-of-district school 4

A Challenge to all: Students come and go in waves. 5

A “class portrait” of the Class of

Graduation rates are the new metric for school district success... 7

... but confusion reigns. 8

Why an early warning indicator? In the absence of specific indicators of risk for individual students, schools have sometimes relied on racial/economic profiling: “poor kids, especially poor kids of color, are the likely dropouts.” Although dropout rates are higher among poor children of color, this stereotype is neither accurate nor prescriptive: Race, sex and free lunch status alone can predict only a small proportion of future dropouts. Knowing only race, sex and free lunch status does not provide adequate information on which to build strategic interventions. 9

What are the requirements for an early warning indicator and “tipping points”? Quantifiable student characteristics that... Have been found by research to be associated with leaving high school without a diploma; Are available in the current student information system; and Can be easily accessed by school administrators/teachers. A statistical method (logistic regression) to identify the combination of characteristics that most accurately and parsimoniously predict which students leave school without a diploma. 10

What happens to students who were retained/demoted? Students may have been retained in grade for a number of reasons, but most likely for lack of academic progress. Students who were retained in any earlier grade are significantly less likely than other students to graduate. Students who “catch up” with peers are twice as likely to graduate as those who don’t. 11

Unexcused absences are extremely important as early warning indicators/tipping points. It is not clear whether unexcused absences (aka skipping) causes or is a result of poor school performance. Either way, they are highly predictive of eventual dropout. Days of unexcused absence are less frequent in middle school, but still highly predictive. 12

To be specific—unexcused absences (in groups of five) in any grade are strongly predictive of eventual high school failure. The graduation rate drops percentage points after 5 unexcused absences in any school year. 13

Fs in core courses are strongly predictive of high school failure, and their effect is cumulative. 14

GPAs are common currency—and can provide additional information Some students never receive an F but still fail. They pass, but with a low GPA that predicts leaving high school without a diploma. GPAs in middle school are less reliable than those in high school, but a very low middle school GPA is highly predictive of later dropping out of high school. Students with cumulative GPAs below 1.5 at any grade are about half as likely to graduate as students with GPAs at or above

There is a weak relationship between meeting standards on 7 th and 10 th grade WASL tests and getting to graduation. 16 Note: Although not predictive on their own, WASLs can provide additional information and/or can act as surrogates for missing GPAs for students transferring into the district. Scores in reading at 7 th and 10 th grades are more powerful than scores in other subjects.

Adding towards a useful model for early identification of risk Major predictors of high school withdrawal include: Earning one or more Fs in any year of middle school (doubles risk) or in the 9 th or 10 th grade (doubles risk again) Has unexcused absences in excess of 5 per school year, in any middle school or high school year GPA below 1.5 in middle school Earning very low scores on WASL tests at 7 th and 10 th grades More than one out-of-school suspension during middle or high school 17

Longitudinal analysis clarifies/displays timing and level of risk Answers two questions about students in the SPS: When are students most at risk of dropping out? What events/behaviors increase risk? Taking into account race, gender and free lunch status, risk for students coming from SPS middle schools is: 1.5 times higher if repeated a grade, doubles again with 1+ core course Fs in MS, and doubles again with 6+ unexcused absences in MS. 18

Segmentation: Definitions Four groups of dropouts are distinguished by timing and performance; the fourth group defies prediction. Early strugglers: Students who have academic or behavioral risk factors in middle school that continue into high school. HS Off-track: students who enter SPS without evidence of risk or enter after 9 th grade and then get off-track—usually within the 1 st year. In-place dropouts: students who have relatively low measures of risk and remain in school through 12 th grade (or longer) and then disappear or drop out. The unpredictables 19

The Subgroups/Segments of SPS Dropouts 20 Early strugglers are more likely than other dropouts to have been retained, to be male, to be older than peers Off-trackers tend to run into academic difficulties almost immediately In-place dropouts don’t hit tipping points; under the radar

Characteristics of different segments provide insight into location/profile of students at risk for dropping out. 21 Early Strugglers can be identified in or before middle school and are much more likely than other dropouts to be poor children of color—who are probably concentrated in a limited number of schools. Multivariate analysis indicates that performance trumps demographics in terms of prediction, but demographics are clearly important.

Importance of unexcused absences: some notes, implications and options Criteria for unexcused/excused absences need to be clear and consistent across schools. Follow-up of unexcused absences should be immediate (c.f. Sprint ads!) Record keeping on absences must be reliable and immediately available to counselors/teachers. The reasons for the unexcused absences need to be identified and addressed on an individual basis. A process for catching up should be immediately available. 22

Importance of failures in core courses: some notes, implications and options Core course Fs in middle school are less frequent than in high school, but just as predictive of dropping out later. Failures in 9 th grade are particularly dangerous, but can be predicted for many students from their middle school grades. Such failures can be anticipated. Students transferring into the district from outside are more likely to fail core courses than long-time students, with less warning. Orientation to the district should go beyond setting schedules. Mid-term grades could be used as the “red flags” for intervention. Fs are dangerous; multiple Fs can be fatal. 23

Possible uses/next steps for Seattle ‘06 cohort data/findings Revise student data system to format key data items for use in an early warning system Use tipping points to build in triggers in the data system Create intervention strategies around triggers (e.g., when a student has an F in a core course in the middle of 1 st semester, 9 th grade, the emergency retrieval team goes into action) Assess risk early and often using SPS-specific risk prediction equations Create targeted interventions for different segments of at-risk students (e.g., create tutoring/relearning program for students with very low assessment scores and/or grades in 7 th grade; develop “catch-up” programs for late entry strugglers.) 24