American Education Research Association April 2004 Pete Bylsma, Director Research/Evaluation/Accountability Office of Superintendent of Public Instruction Impact of NCLB Requirements in Washington State Challenges and Successes

Overview of Presentation Process used to set policies Overview of state AYP policies First year results and reactions Changes proposed

Context of NCLB in Washington NCLB is long and complex Implementation is complicated and still evolving Previous state accountability system not well developed More work in less time with higher stakes

Process Used to Establish Policies Many analyses of alternative policies projecting results based on 2002 data Widespread stakeholder input and review of data Peer review went very smoothly –Plan was relatively simple –Impact data available to support all key decisions Making policies operational required lots of thought –Eliminating ways to “beat the system”

Overview of State AYP Policies Annual goals on “straight-line” to 100% in 2014 Separate results for tests in grades 4, 7, and 10 N of 30 for accountability (10 for reporting) Continuous enrollment from October 1 through the testing period (ends in mid-May) Standard error at 95% confidence level On-time graduation rate goal = 73% or 1 point above previous year (85% goal in 2014) Unexcused absence rate goal = 1% or a reduction from previous year

AYP Yearly Targets Grade 4 Grade 7 Grade 10 YearReadingMathReadingMathReadingMath Baseline

Percent Meeting Standard ReadingMathematics Increment of 4.0 needed Increment of 5.9 needed GRADE 4 YEARLY TARGETS AYP can be made if the percent meeting standard is below the yearly target either via safe harbor or when the standard error is included in the total. (Increments are rounded)

Summary of State AYP Results State made AYP in 78 of 111 categories (70%) Of the proficiency categories, the state made 21 of 54 (39%) All Students, Asian/Pac. Is., and White groups made AYP in all grades in both reading and math American Indian, Black, and Low-Income groups made AYP in just one of six proficient categories (Grade 4 Math) Hispanic, Special Education, and Limited English groups did not make AYP in any proficiency category Graduation rate initially 79%, later changed to 66% Unexcused absence rate 0.5%

First Year Results Total number of districts and schools not making AYP was very close to projected numbers (42% and 22%) But far above previous results (0% and 3%) Safe harbor helped very little Not making AYP is a function of the N (Secondary schools less likely to make AYP due to larger enrollment in tested grades) Most students are in districts that did not make AYP

Responses to NCLB Much more attention being given to data quality and achievement gap (motivation to change) Widespread cynicism about law and lack of funding to meet federal mandates Negative impact on public perception of testing in general Narrowing curriculum to tested subjects Some focus on beating the system rather than serving students – temptation to lower standards

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2003 figures include results from the alternate assessment, include the standard error, and are based on results of continuously enrolled students 60.6% Grade 10 Percent meeting standard 52.9% 2003 Goal Grade % 56.2% Grade % 35.9% “All Students” Category in Reading State 2003 Results – Adequate Yearly Progress

2003 Goal 35.6% State 2003 Results – Adequate Yearly Progress

Unexcused Absence Rates Among 296 Districts Percent Unexcused Absences 35 districts (11.8%) with greater than 1%

AYP Results on the Web

296 Districts District 2003 AYP Results (All Grades) Made AYPDid Not Make AYPToo small to evaluate (N<30) Number of districts

296 Districts Percent of Students in Districts by AYP Result (All Grades) 79.3% 19.4% 1.3% 0% 20% 40% 60% 80% 100% Made AYPDid Not Make AYPToo small to evaluate (N<30)

0% 20% 40% 60% 80% 100% < >4250 Number of students in a district Districts making AYP Odds of a District Making AYP Declines as Enrollment Increases Less than 30 in a grade

District 2003 AYP Results by Grade Grade 4Grade 7Grade 10 Number of districts Made AYPDid Not Make AYPToo small to evaluate (N<30) 5 via safe harbor 2 via safe harbor 3 via safe harbor

Percent of Students in District by Grade 16.1% 24.1% 18.3% 74.2% 82.6% 80.8% 1.7% 0.9% 1.4% 0% 20% 40% 60% 80% 100% Grade 4Grade 7Grade 10

Example District AYP Results Olympia School District, ~9000 students Grade 4 Reading All studentsAmerican Indian Asian/ Pac Is BlackHispanicWhiteSpecial Education Limited English Low Income Percent meeting standard Students meeting standard *SEP Interval * For students continuously enrolled from October 1 NA (<30) Goal = 56.2 NA (<30) NA (<30)

District AYP Results, 2003 Grade 4 Reading by Group All Students American Indian AsianBlackHispanicWhiteSpecial Education Limited English Low Income Number of districts Made AYPDid Not Make AYP

District 2003 AYP Group Results All Grades in Reading 63.6% 97.6%98.0% 89.9% 47.3% 15.1% 17.9% 70.7% 66.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% All Students American Indian AsianBlackHispanicWhiteSpecial Education Limited English Low Income Percent of cells with 30 or more students making AYP

District AYP Results, 2003 Number of Groups Not Making AYP in the 123 Districts 111 possible

School 2003 Results (All Grades) Adequate Yearly Progress Made AYPDid Not Make AYPToo small to evaluate (N<30) Number of schools

Percent of Students in Schools by AYP Result (All Grades) 43.0% 53.8% 3.1% 0% 20% 40% 60% 80% 100% Made AYPDid Not Make AYPToo small to evaluate (N<30)

School 2003 Results (By Grade Level) Adequate Yearly Progress Grade 4Grade 7Grade 10 Number of schools Made AYP Did Not Make AYP Too small to evaluate (N<30)

Percent of Students in Schools by AYP Result and Grade 43.3% 84.7% 34.1% 11.8% 54.1% 62.5% 3.5% 3.4% 2.6% 0% 20% 40% 60% 80% 100% Grade 4Grade 7Grade 10

School AYP Results, 2003 Number of Categories Not Making AYP All Grades Number of cells not making AYP Number of schools

Total of 432 Schools Not Making AYP 51 Schools in “School Improvement” 13 schools made adequate yearly progress for the second year in a row and were removed from “school improvement” status. 44 of the 51 schools receive Title I funds 5 of the 51 made AYP in 2003 and remained in school improvement status Year 1 (Alert)Year 2/Step 1Step 2Step 3 Improvement Stage Number of schools 164 (43%) of these schools receive Title I funds

Proposed Changes Revise on-time graduation rate goals More accountability for small schools (N of 10-29) Increase N for LEP and special education Increase confidence level to 99% Use new flexibility for LEP testing, participation Add recognition system Develop criteria to differentiate need/assistance Clarify appeals process Use new student information system