Accountability and Assessment: From “A Nation at Risk”  NCLB  Race to the Top

Framing Questions: Who can and should be held educationally accountable, for what, to whom, and why? How can we measure student learning in a way that both reflects and promotes concerns about equity? How did No Child Left Behind (NCLB) change educators’ practices and national conversations about these issues? How are Race to the Top and current ESEA reauthorization negotiations influencing the national conversation, on the one hand, and educational policy and practices, on the other?

1980: Department of Education created (at Cabinet level) 1965: Elementary and Secondary Education Act (ESEA), including Title I funding for disadvantaged children. Requires reauthorization every 5 years. 1917, 1946: Federal aid to schools for vocational, agricultural, and home ec education 1958: National Defense Education Act (response to Sputnik) funds improvements in science, math, and foreign language instruction 1964: Title VI of Civil Rights Act 1972: Title IX of Education Amendments 1973, 1975: Section 504 of Rehabilitation Act, Education for All Handicapped Children Act 1980s – 2000s: States and professional groups develop content standards, assessments, and accountability mechanisms prohibit race, sex, special needs discrimination in education 1983: A Nation at Risk 2002: ESEA reauthorized as No Child Left Behind Act of : Individuals with Disabilities Education Act revamped 2009-??: Race to the Top, ESEA reauthorization 2010: Common Core Standards

Assessment and Accountability: The Early Years 1980s – early 1990s “Off the shelf” tests: Stanford, ITBS, California Achievement Test Norm-referenced Minimum competency tests: e.g. Texas Assessment of Basic Skills (1979), Texas Educational Assessment of Minimum Skills (1984)

Assessment and Accountability: The Early Years 1980s – early 1990s “Off the shelf” tests: Stanford, ITBS, California Achievement Test Norm-referenced Minimum competency tests: e.g. Texas Assessment of Basic Skills (1979), TX Educational Assessment of Minimum Skills (1984) Mid-to-late 1990s Achievement tests: e.g. Texas Assessment of Academic Skills (1993), Texas Assessment of Knowledge and Skills (1999) Criterion-referenced Tests start to be aligned with state standards

Criticisms of State Assessment Systems Wildly variable in quality and rigor Often not aligned with state standards (state standards didn’t always exist) Student-level accountability (e.g. must pass to graduate) but no teacher, principal, or school-level accountability Not used to guide instruction

Accountability as…

Pause and Think Pause and think:  How have state assessment and accountability systems changed over the past 30 years?  Do these changes represent progress, regress, or some of both?  What purposes do you think state-level assessment and accountability systems should play in education, if any?  What questions do you still have?

NCLB (No Child Left Behind)

Major NCLB Requirements: All students (100%) at “proficiency” by 2014 – AYP determined for each subgroup: Failure to make AYP in any subgroup = overall school failure to make AYP Progressive sanctions, some including increased funding Reading and math tests every year grades 3-8, once in grade 9 or 10, science at three diff. times Graduation and attendance rates now part of AYP (100%-current % proficient) (2014-current year) % increase expected=

How NCLB Promotes Equity: Disaggregates scores Judges schools by their least successful students, not their most successful Establishes clear and common achievement standards; eliminates between-school and even between-district variation in standards for success Provides strong incentives for school improvement focused on student achievement Purportedly offers students/families in failing schools additional options, including to transfer to a non-failing school

Challenges to Equity: The Threat of Perverse Incentives States Schools Students

Perverse Incentives for States: Lower Standards to Increase Passing Rates Source for following slides: National Center for Education Statistics (2007). Mapping 2005 State Proficiency Standards Onto the NAEP Scales (NCES ). U.S. Department of Education. Washington, DC: Author.

Perverse Incentives for Schools: Don’t Let Students Get to 10 th Grade (unless they can pass the test)

BostonDistrict Results Academy of Public Service Brighton High Charlestown Community Academy Madison Park Monument High Fenway High Social Justice Academy CambridgeDistrict Results NewtonDistrict Results SomervilleDistrict Results LawrenceDistrict Results District Name School Name Gra de 1 Gra de 2 Gra de 3 Gra de 4 Gra de 5 Gra de 6 Gra de 7 Gra de 8 Gra de 9 Gra de 10 Gra de 11 Gra de 12 District Retention Rates: Evidence of the Urban Ninth Grade Bulge Updated Apr. 16, Accessed December 15, 2009.

District Retention Rates Retention Rates by Race/Ethnicity (All Numbers are Percentages) District Name Number of Retentions Enrollment in Grades 1-12 Retention RateWhiteBlackAsian Native American Native Hawaiian Multi-Race, Non- HispanicHispanic Boston3,36750, Cambridge934, Lawrence52111, Newton3610, Somerville2524, District Retention Rates by Race/Ethnicity ( )

Perverse Incentives (or at least outcomes) for Students: If You’re Not Passing, Drop Out

Retention and Dropouts Repeating any grade correlated with and even clearly contributes to dropping out Persistence to 12th grade dramatically lower for students repeating grade 9 (TX & Philadelphia) Up to 40% of ninth grade students in cities with the highest dropout rates repeat 9 th grade; only 10– 15% of those repeaters go on to graduate 40% of dropouts in low–income high schools left after ninth grade vs. 27% in low–poverty schools Balfanz and Legters 2004,

Cohort year Graduation Rates in Massachusetts: Source: * Limited English Proficient Graduates Non-Graduates 2009 Cohort # 4-Year Rate Difference from 2008 Still in School Non-Grad CompleterGED Dropped Out Expelled All Students77, % %0.8%2.1%9.3%0.1% Female37, % %0.8%2.0%7.9%0.0% Male39, % %0.8%2.3%10.6%0.2% LEP*4, % %0.8%4.9%22.9%0.2% Special Education14, % %1.7%2.0%16.1%0.1% Low Income29, % %1.5%3.1%17.3%0.1% African American6, % %1.8%2.0%15.0%0.2% Asian3, % %1.2%0.9%5.8%0.1% Hispanic10, % %2.1%2.6%22.6%0.2% Multi-race, Non- Hisp. 1, %0.06.7%0.7%2.5%9.5%0.1% Native American % %0.0%0.9%15.1%0.0% Pacific Islander7269.4% %1.4%2.8%11.1%0.0% White54, % %0.4%2.1%6.3%0.1% Urban26, % %1.5%3.0%17.3%0.1%

Pause and Think Pause and think:  What are the strengths of NCLB both overall and particularly with respect to the achievement of educational equity?  What are the weaknesses of NCLB both overall and particularly with respect to the achievement of educational equity?  Is it possible to retain the strengths while overcoming the weaknesses, especially those resulting from perverse incentives? How?  What would you do if you were in the position of reauthorizing ESEA? What would you tell your senator or congressperson, and why?

ESEA Reauthorization: Points of Agreement Make AYP (or its equivalent) more nuanced, not just yes/no Measure individual student growth rather than cohort comparisons: “value-added” measurements (VAM) Incorporate multiple measures into accountability system: e.g. dropout and retention rates, possibly higher-ed access Do everything possible to avoid perverse incentives Provide resources (“opportunity to learn”) and not just consequences Use data to guide instruction and not just guide sanctions and rewards Promote complex teaching for complex thinking

Value Added Measurements Track individual student growth under one classroom or in one school, rather than comparing cohorts Many concerns about wild fluctuations in VAM assessments of teacher effectiveness year-to-year ~30-40% predictive power: about 1/3 of teachers in top group one year will be in top group next year But multi-year averages may provide reliable data on individual teacher effectiveness, especially when combined with other data sources Sources: Prof. Andrew Ho and

VAM in Los Angeles: Data-driven teacher accountability at work Los Angeles Times used seven years of district math and English data to conduct a VAM analysis of LAUSD teachers in Summer 2010 “Grading the Teachers. Who’s teaching L.A’s kids?” ( ) published with teacher names and pictures Full searchable database available to the public online Protests, rallies, and threatened teacher’s union boycott of Los Angeles Times followed LA Unified School Board voted soon after to include VAM in teacher evaluations Male elementary school teacher committed suicide in Sep Public VAM data was blamed in media. The Times issued their condolences. Data are still available online.

Race to the Top (RTTT)

$4.3 billion in incentives to get states to:  Adopt standards and assessments that prepare students to succeed in college, the workplace, and the global economy  Build data systems that measure student growth and success and inform teachers and principals about how they can improve instruction  Recruit, develop, reward, and retain effective teachers and principals especially in high-need schools  Turn around lowest-achieving schools Race to the Top: Goals

States must demonstrate their ability to: Participate in a consortium of states that are working toward adopting a common set of K-12 standards that are internationally benchmarked Develop & implement common, high- quality assessments aligned with the common standards RTTT: Standards & Assessments Common Core Multi-state assessments

States must demonstrate their ability to: Implement a statewide longitudinal data system; Make the data available to those working with local instructional improvement systems RTTT: Data Systems Value Added Measurements Research into risk factors, causal relationships Formative assessment Data- Driven Instruction

States must demonstrate their ability to: Allow alternative routes to certification; Design and implement evaluation systems for teachers and principals that take into account data on student growth; Develop a plan to distribute “effective” teachers and principals equitably; Link measures of “effectiveness” to preparation programs and professional development RTTT: Teachers & Leaders

States must demonstrate their ability to: Intervene directly in low-performing schools and districts; Identify the lowest-performing schools; Implement one of four school intervention models: RTTT: Low-Achieving Schools  Turnaround  Restart  School Closure  Transformation

Who can and should be held educationally accountable, for what, to whom, and why? How can we measure student learning in a way that both reflects and promotes concerns about equity? How did No Child Left Behind (NCLB) change educators’ practices and national conversations about these issues? How are Race to the Top and current ESEA reauthorization negotiations influencing the national conversation, on the one hand, and educational policy and practices, on the other? Pause and think: