Avoidable Losses Rice University Creekmore Symposium April 29, 2008

The Problem 135,000 youth are lost from Texas high schools every year prior to graduation. These are disproportionately African-American, Latino, and English Language Learners

High Stakes Accountability More than a decade of reform Aimed at:  increasing student achievement  closing the achievement gap Became the model for the nation

Avoidable Losses: Our Findings The high stakes accountability system is directly connected to the severity of the dropout problem This occurs when administrators are in compliance with the accountability system  School ratings go up as weak students exit the system  Students are viewed as assets or liabilities to the school ratings

How did we discover this connection? Data from 271,000 students over 7 years. Analysis from inside 7 schools Multi-year case study of a school working to comply with the accountability system Student interviews to learn their perspectives on life in school under this system

Avoidable Losses: The Study Accountability Miracle in Brazos City Educate or Comply The Culture of Accountability: Setting Kids Up to Fail

Accountability Miracle in Brazos City Julian Vasquez Heilig, Ph.D. University of Texas at Austin

Texas Miracle In 1993, the Texas Legislature enacted a ground-breaking statute that mandated the creation of the Texas public school accountability system to rate school districts and evaluate campuses During the 2000 presidential campaign, George W. Bush touted the “Texas educational miracle” as a model for the rest of the nation George W. Bush and Rod Paige, two primary arbiters of NCLB, lassoed their ideas for federal education policy from Texas

The Miracle Closing achievement gap Low dropout rate Soaring Exit test scores Rising graduation rates Rising accountability ratings

Closing the Achievement Gap?

BCSD Reported Dropout by Year

Graduation Rates and Exit BCSD reported: Graduation rate had soared 21 percentage points over five years, from 54 percent in 1997 to 75.3 percent by 2002 Local Newspaper characterized the high school achievement gains with the headline: “Sophomores soar in Exit TAAS exam”

TEA Accountability Ratings for BCSD High Schools ( )

Considering BCSD’s closing achievement gap and rising accountability ratings (ultra low dropout and high graduation rates) it might be suggested that BCSD schools dramatically improved urban minority students’ academic and enrollment success. Indicative of an education miracle- worthy of national acclaim and replication into a comprehensive national educational policy (NCLB).

Accountability Miracle Reconsidered Testing analysis Longitudinal cohort analysis Grade retention Graduation

BCSD Testing: High-stakes vs. Low- stakes On average, about 12% of African American and Latino elementary and middle school students were excluded from the TAAS (high-stakes) but not Harcourt (low-stakes) testing from In contrast to the TAAS, there is little difference in test-taking patterns on Harcourt tests by race/ethnicity. As a result, greater proportions of Latino and African American students’ test scores were excluded from TEA accountability ratings

Mean Scores by Participation Those excluded from the English TAAS, the basis of the state and district accountability rankings, scored significantly lower on the Stanford-9 than those who took the TAAS across all racial/ethnic groups (p<.001)*. * A GLS regression analysis examined the effects of a dummy variable representing inclusion or exclusion in the English TAAS (controlling for race / ethnicity, language status, income status, and school-level proportions of at-risk students).

BCSD High School Cohort Progression

BCSD High School Cohort Progression: Latinos and African Americans

BCSD Grade Retention by Year

BCSD 9th Grade Retention by Race/Ethnicity

BCSD Cohort Testing on the Exit Retention?: Only 209 of 3,489 retained students (about 8% of the total) ever became eligible to graduate by passing all three subjects on the spring Exit TAAS.

BCSD Graduation Rate of th Grade Cohort (5-Yr) Large number of students are being retained and otherwise not advancing with their peers through school Many students not testing on the Exit Cohort graduation rates should be correspondingly low

BCSD Graduation Rates Vast discrepancy between the cohort graduation eligibility and status rates and publicly released rates District hemorrhaged students, especially between the 9th and 10th grades A counselor at King high school estimated from her experience, “I would think that the graduation rate is closer to 40-45%, not 85%.” She related, Ultimately what’s happening is that we’re letting kids down. We’re using some kind of system to disguise where they are. If you’ve got 600 kids in your school and 300 graduate….they’re somewhere...

Left Behind? Is it possible that high-stakes testing and accountability may not be as good as advertised? Voices from BCSD high schools on their response to the pressure of high-stakes testing and accountability

Inferential Regression Analyses Do actions that impact student progress spur increases in high-stakes test scores and TEA accountability ratings? GLM Random-Effect Regression  Increasing retention and disappearance of students increases average Exit scores Multinomial Logistic Regression  By increasing 9th grade retention, schools increase and maintain accountability rating Robustness of GLM random-effect and multinomial logistic regressions suggest that manipulating student progress can increase Exit scores and accountability ratings

Summary of Findings High-stakes test scores and accountability ratings steadily rise as students steadily leave Incentives to game and reduce educational opportunity for low- performing students High-stakes testing and accountability policies led to an escalation of deleterious outcomes disproportionately impacting minority students

To Comply or To Educate Judy Radigan, Ph.D.

A Case Study in Compliance Edgeview H S, representative of Brazos City high schools with losses of low-income and minority youth 75% Latino, 20% African American, 4% White, 10% English Language Learners Reforming school working to personalize the learning environment with small learning communities Spring 1997 – Principal targeted to improve scores markedly or face TERMINATION

Ninth Grade Waiver Based grade promotion from 9 th grade on passing of core courses: English, social studies, math, science rather than number of credits gained All the schools had it. In fact, we were one of the last schools to accept the waiver because philosophically we thought [to do so] would be cheating. We were testing everybody. We felt we had no choice but to move to that waiver in order to save face and get our scores up. A lot of the kids were defeated, and they dropped out.

Texas Miracle TAAS98/9999/0000/0101/02 All Reading Writing Math Edgeview’s accountability rating moved from “acceptable with acceptable progress” to “recognized with exemplary progress.”

Withdrawal from High School

9 th Grade Retention

Structural Change Small Learning Communities

Accountability Incentives: From System to Principal Federal State school District school vendorsconsultants

Texas Miracle? It’s not a miracle to manipulate things. A miracle is saving kids actually, in reality—that’s what miracles are. It’s not to manipulate things so that it appears—it’s a facade.

The Culture of Accountabilty: Setting Up Kids to Fail Eileen M. Coppola, Ed.D.

Expanding the Lens/ Drilling Down into Student Experience How does what we’ve seen at Edgeview become systematically embedded? How does the operation of the accountability system become “normal” or unquestioned? How do students experience school under this system? How does their experience affect their motivation and ability to stay in school?

Focus groups and individual interviews with school staff and students in 7 Brazos City High Schools. 3 schools majority African-American; 4 majority Latino; all majority low SES. 14 Administrators, 24 Math and English Teachers. 122 students selected by teachers from senior English classes and volunteers from a program for former dropouts. Data

Other Components of the Accountability System Zero-tolerance attendance policy. Grade retention. Curriculum degradation.

Beneath the Surface: Connecting Dropping Out with the Accountability System Extensive grade retention leads to frustration. Texas’ 90% rule on attendance mean that students are penalized and lose credit for courses. The demands of high-stakes testing degrades the curriculum over time. Punitive culture and rules conflict with cultural norms and adult status of students.

Grade Retention Considerable research shows an increased likelihood of dropping out each time a student is retained in grade. High levels of frustration, disorientation, and alienation reported by students, leading to other choices than continuing school.

(Focus group with former dropouts who returned to school) Jose:I’ve repeated ninth grade four – four times. Arturo:Same thing with me. I repeated ninth grade four years because I just slacked off. I was lazy and – Int: You repeated ninth grade four years? Arturo: Yes. …Well part of my problem was my freshman year it was – if I didn’t know anything I wouldn’t do it, or I wouldn’t go to class because, you know, I was afraid if they’d call on me and have me go up I wouldn’t know how to work it so I just wouldn’t go to class. So that was just maybe the reason. The Effects of Retention

The Effects of Retention, Continued Paolo: Oh, yeah, they had me taking Algebra for, like, three years straight. I passed the first year, so in the second year I just decided not to go. I tried to get it fixed, but they wouldn’t fix it. So after the third week trying to get it fixed I just stopped going. Int: So did they count you as absent? Paolo: Yeah, and when they tried doing that I just got more aggravated and stopped coming to school period.

Attendance Policies 90% rule Loss of credit Evolution over time/credit recovery courses Intervention of Justice of Peace and Juvenile Court – ticketing Yet, many reasons for attendance issues: family, working, need for flexibility.

(Focus group with former dropouts who returned to school) Carla:I went to three elementary schools…then Middle School. Then I came to ninth grade. When I got to ninth grade, you know, I just made it as a fun thing, you know. I had more freedom than what I had in other schools. Learned – made new friends, you know. Well I flunked ninth grade and back to ninth grade again. Kept going in circles and circles, and my mom used to get tickets and tickets so, you know, it’s just left for me to drop out instead of me just giving my mom nothing but tickets. So I dropped out, and I stayed, like, a whole year without being in school. Int: So what did they actually say – I mean, what happened? Carla: They sent me a truancy letter saying that if I missed one more day of school that I would be fined….A $500 ticket and a court date. The Effects of Attendance Policy

Lavone:Per absence. Carla: Per absence, and I was – but I had my excuse. But most students – Int: See, I was wondering if that $500 fine, if they would (that would make you really stop coming to school? Lavone:That would be a lot. Sean: Some people – rich people really don’t care. For that reason they know if they’re still going to miss another day of school they’ll probably okay with it. Carla: I might as well drop out. Sean: I might as well drop out. Carla: And no one’s going to pay $500 a day. Effects of Attendance Policy, Continued

Curriculum Degradation Curriculum that is increasingly fragmented and shaped by the forms and priorities of the standardized tests themselves. This fragmentation leads to inauthentic curriculum that further alienates students, who become bored and frustrated. Schools continued to have limited capacity to help struggling students. Students complain that they struggle, and worry about the tests, but cannot understand the work.

Example: Curriculum Degradation Irma: Instead of teaching us the real life things that we going to need for college and stuff, they started zeroing in just on that test. So it makes everybody nervous, and it threw everybody off. So, like our curriculum is thrown off, ‘cause what they originally were teaching us in the subjects, all of the sudden they switched and then they were just zeroing into this test.

Fernando: Math is – its not that it’s boring, it’s just hard. I’ll get it in the beginning, and I’ll get lost. I won’t know what’s going on…[The teacher] helps but, you know, it just – John: He [the teacher] doesn’t break it down a lot. Fernando: Yeah, he don’t – and then his examples are wack. He takes it out of his mind. Int: How does he decide what math you guys should learn? Fernando: Right now he’s teaching us the TAKS. John: Yeah, right now he’s preparing us for TAKS…. John: They teach us what’s on the test – what’s going to be on the test. Fernando: You need that to graduate. Second Example Curriculum Degradation

Culture of Accountability Culture of accountability:  alienates students  can degrade curriculum so it is overly oriented to tested skills  does not address instructional capacity of schools or faculty to undertake challenging teaching  Is technical, bureaucratic, and punitive, not developmental

Has NCLB Made a Difference? NCLB has exacerbated the pressure for increased school ratings. Schools at risk of losing funding NCLB provisions raise the stakes by putting the school at risk of closure.

Conclusions These losses are avoidable by rebuilding a system that:  is less rigid, less punitive, more driven by developmental concerns, makes school engaging places for young people  dramatically improves the capacity of schools to serve the students most at-risk of dropping out  conceives accountability as running in both directions: communities must adequately fund and support schools  communities become totally involved in thinking about what we want from our schools

Avoidable Losses: High-Stakes Accountability and the Dropout Crisis Education Policy Analysis Archives January,