Forty Years of Public School Reform: What the Data Tell Us Randy Keyworth Jack States The Wing Institute

A value of 1.0 = perfect flatness Pancake =.957 Kansas =.999 Texas State University Arizona State University Is Kansas Flatter Than a Pancake?

National Assessment of Educational Progress (NAEP) Only nationally representative and continuing assessment of what America’s students know and can do in: math, reading, science, writing, the arts, civics, economics, geography, and U.S. History. Common metric for all states Stays the same each year with carefully documented changes Administered by National Center for Education Statistics (NCES) within the Institute of Education Sciences within the U.S. Department of Education Provides a clear picture of student academic progress over time Tests are continually scrutinized for reliability and validity by panels of technical experts within NCES and by external groups.

A value of 1.0 = perfect flatness Pancake =.957 Kansas = years NAEP reading data =.996 Is Kansas Flatter Than a Pancake?

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1992, 1994, 1998, 2000, 2002, 2003, 2005, and 2007 Reading Assessments.

Grade 4 Grade 8 Grade 12

A value of 1.0 = perfect flatness Pancake =.957 Kansas = years NAEP reading data = years IES graduation rate data =.992 Is Kansas Flatter Than a Pancake?

IES, National Center for Education Statistics

Today’s Question: What’s going on????

Operated a large nonpublic school in SF Bay Area six campusesadult programs residential programsemployment supportive services public school consultationteacher training campus CAPSES Board Member President Regional Coordinator GAC Committeelocal SELPA liaison CAPSES “Street Cred”

The Wing Institute present independent, non-profit operating foundation promote evidence-based education policies and practices act as a catalyst to facilitate communication, cooperation and collaboration between individuals and organizations currently engaged in evidence based education

The Wing Institute’s Strategic Vision Identify exemplars in evidence-based education Develop networks to facilitate collaboration Provide support for new ideas, research, and publications Facilitate cross-discipline cooperation

Today’s Discussion What does performance monitoring data tell us about the effectiveness of 40 years of school reform? What does education research data tell us about the effectiveness of 40 years of school reform?

REACHING AMERICA'S HEALTH POTENTIAL: A STATE-BY-STATE LOOK AT ADULT HEALTH Commission to Build a Healthier America May 2009 U.S. Census Data: American Community Survey (2007) U.S. Census Data: Behavioral Risk Factor Surveillance System Survey Data ( )

SOURCE: Department of Health and Human Services (2003)

SOURCE: Department of Health and Human Services (2003)

U.S. Census Bureau, 2004

University of Maryland, Department of Sociology

Source: U.S. Department of Justice (2003)

Education and Delinquency The odds of delinquency, given low academic performance, were about 2.1 times higher than those given high academic performance. Intervention Studies showed that improvements in academic performance co-occurred or followed improvements in the prevalence of delinquency. Maguin & Loeber, 1996

SOURCE: U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), October 1967 through October 2007.

IES, National Center for Education Statistics

SOURCE: U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), October 1967 through October 2007.

Education is the civil rights issue of our generation. Arne Duncan Secretary of Education 2009

PERFORMANCE FEEDBACK Performance feedback is one of the most important tools we have to assess and correct performance at all levels of the education system: Student performance Staff performance Systems performance

STUDENT PERFORMANCE FEEDBACK No Child Left Behind: Adequate Yearly Progress (AYP) graduation rates state test scores in reading and math States can establish standards and goals

The Education Trust (2005) 127,292 high school students dropped out....70% were black or hispanic 1,252,396 high school students dropped out....53% were black, hispanic, or native american

Comparison of State AYP Requirements THOMAS B. FORDHAM I NSTITUTE (2008)

STUDENT PERFORMANCE FEEDBACK Department of Education Initiatives Adopting common standards and assessments Building longitudinal data systems that measure student growth and success to support instruction

STUDENT PERFORMANCE FEEDBACK Standardized Graduation Rate calculation DOE amended regulations require states to adopt the “cohort model” by : four-year adjusted cohort graduation rate disaggregated by subgroups

STUDENT PERFORMANCE FEEDBACK Standardized Tests Nation-wide standardized tests 43 States (so far)

Race to the Top Build robust data systems to aggregate and disaggregate student outcome data link teachers to students outcomes -- to distinguish between effective and ineffective teachers to link effective teachers to their colleges of education – to distinguish between effective and ineffective programs

STAFF PERFORMANCE FEEDBACK 1.Performance measures must be reliable and valid. 2.Staff should know the variables being measured 3.Skills must be identified to attain these measures 4.Staff feedback should be frequent and specific 5.Feedback should trigger and guide effecting coaching.

Hiring, Assignment, and Transfer in Chicago Public Schools Report from The New Teacher Project July 2007

Only 3 out of 1,000 teachers rated unsatisfactory 93% of teachers received superior or excellent ratings Hiring, Assignment, and Transfer in Chicago Public Schools Report from The New Teacher Project July 2007

87 Schools met criteria for being identified as “failing schools” 69 (79%) of these schools did not issue a single “unsatisfactory rating” Hiring, Assignment, and Transfer in Chicago Public Schools Report from The New Teacher Project July 2007

STAFF PERFORMANCE FEEDBACK TRADITIONAL: Performance measures must be reliable and valid. Staff should know the variables being measured. Skills must be identified to attain these measures. Staff feedback should be frequent and specific. Feedback should trigger and guide effecting coaching. No

STAFF PERFORMANCE FEEDBACK LAUSD Performance measures must be reliable and valid. Staff should know the variables being measured. Skills must be identified to attain these measures. Staff feedback should be frequent and specific. Feedback should trigger and guide effecting coaching. Yes* No

SYSTEM PERFORMANCE FEEDBACK Most systemic school reform strategies have focused on “structural fixes”

Structural Fixes….Commonly Held Beliefs SCHOOL RESOURCES increase funding for students higher pay for teachers TEACHER QUALITY more “Highly Qualified Teachers” (NCLB) more teachers with credentials more teachers with advanced degrees more teachers with professional certifications (NBPTS) comprehensive induction for new teachers SCHOOL MODELS smaller class size school choice charter schools SCHOOL REFORM INITIATIVES A Nation at Risk Goals 2000 No Child Left Behind

Structural Fix: Increased Funding for Education SOURCE: U.S. Department of Education, National Center for Education Statistics. (2009). Digest of Education Statistics, 2008 (NCES ), Chapter 2 and Table 179.

IES National Center for Education Statistics Digest of Educational Statistics: 2008 Structural Fix: Increased Pay for Teachers

Structural Fix: More Highly Qualified Teachers

Structural Fix: More “Credentialed” Teachers

SOURCE: National Education Association, Status of the American Public School Teacher, (This table was prepared August 2003.) Structural Fix: Advanced Degrees for Teachers Master’s Degrees Bachelor’s Degrees

Structural Fix: More Board Certified Teachers

Smith & Ingersoll, 2004 Structural Fix: More Induction Support for Teachers Structural Fix: More Induction Support for Teachers

Structural Fix: Class Size Reduction 32 States now have class size reduction programs in law: Texas (1983)California (1996) Tennessee (1989)New York (1997) Minnesota (1990)Florida (2002)

Structural Fix: Class Size Reduction

Structural Fix: School Choice TRENDS IN THE USE of SCHOOL CHOICE: 1993 To 2007, IES

There are currently over 5,000 Charter schools in the United States U.S. Department of Education, National Center for Education Statistics Structural Fix: More Charter Schools Structural Fix: More Charter Schools

A History of Structural Fixes SCHOOL RESOURCES increase funding for students higher pay for teachers TEACHER QUALITY more “Highly Qualified Teachers” (NCLB) more teachers with credentials more teachers with advanced degrees more teachers with professional certifications (NBPTS) comprehensive induction for new teachers SCHOOL MODELS smaller class size school choice charter schools SCHOOL REFORM INITIATIVES A Nation at Risk Goals 2000 No Child Left Behind ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1992, 1994, 1998, 2000, 2002, 2003, 2005, and 2007 Reading Assessments.

National Assessment of Educational Progress (NAEP) Basic denotes partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade. Proficient represents solid academic performance. Students reaching this level have demonstrated competency over challenging subject matter. Advanced represents superior performance. Grade 4Grade 8Grade 12 Basic = 208Basic = 243Basic = 265 Proficient = 238Proficient = 281Proficient = 302 Advanced = 268Advanced = 323Advanced = 346

1.2 million students below basic 2.4 million students below proficient

347,900 Students

Structural Fix: Class Size Reduction

Legislatively mandated four year comprehensive study. Major CA research orgs: AIR, RAND, PACE, WestEd, EdSource Conclusions (Final Report): 1.The relationship of CSR to student achievement was inconclusive…attribution to any gains in scores to CSR is unwarranted. 2.Students in CSR received more individual attention, but similar instruction and curriculum. 3. CSR remains very popular with parents and teachers. CSR Research Consortium

PERFORMANCE FEEDBACK we have had lousy performance feedback systems student performance teacher performance system performance we have have been focusing on the wrong things structural fixes haven’t worked enormous time, energy and money has been wasted there is hope on the horizon better student longitudinal data systems standardized formulas for graduation rate data uniform standardized tests better tools for evaluating teacher performance What’s going on????

THE END

The Education Culture Education has been a non-evidence-based culture: Teaching is an “art”, not a “science” Constructivism, anti-science, anti-measurement Ruled by ideology, preferences, hunches, fads The teaching profession has been stuck in a “guild” mentality Teacher tenure Resistance to performance appraisal Most school reform strategies are not based on scientific evidence Lack of fundamental research into teaching, curriculum, systems Special interests drive initiatives The culture drives simplistic, commonly held “beliefs” Lack of performance feedback

What is the relationship between teacher experience and effectiveness? Kane, Rockoff, Staiger, 2006

What is the relationship between teacher experience and effectiveness?

STUDENT PERFORMANCE FEEDBACK Sample of Graduation Rate calculations: counting only seniors for graduation rates (not freshman entering high school) percentage of kids who got their diplomas in four years or less number of graduates compared to 12 th grade enrollment on the last day of school

Structural Fix: Major School Reform Initiatives A Nation at Risk (1984) Goals 2000 (1994) No Child Left Behind (2001)

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1978, 1982, 1986, 1990, 1992, 1994, 1996, 1999, 2004, and 2008 Long-Term Trend Mathematics Assessments.

What’s Going On???? TOOLS: Roadmap Roadmap: a model for identifying critical components of evidence-based education Data-mining: Data-mining: a mechanism for identifying, analyzing, and disseminating evidence

Research Replicability Sustainability Evidence-based Education Roadmap Practice

Keys to Evidence-Based Education RESEARCH selecting interventions PERFORMANCE MONITORING evaluating interventions

The Wing Institute Services Publications Research Public Policy Analyses Information Clearinghouse The Evidence-based Education Knowledge Network Professional Forums