1. American Education Research Association April 2004 Pete Bylsma, Director Research/Evaluation/Accountability Office of Superintendent of Public Instruction Impact of NCLB Requirements in Washington State Challenges and Successes

2. Overview of Presentation Process used to set policies Overview of state AYP policies First year results and reactions Changes proposed

3. Context of NCLB in Washington NCLB is long and complex Implementation is complicated and still evolving Previous state accountability system not well developed More work in less time with higher stakes

4. Process Used to Establish Policies Many analyses of alternative policies projecting results based on 2002 data Widespread stakeholder input and review of data Peer review went very smoothly –Plan was relatively simple –Impact data available to support all key decisions Making policies operational required lots of thought –Eliminating ways to “beat the system”

5. Overview of State AYP Policies Annual goals on “straight-line” to 100% in 2014 Separate results for tests in grades 4, 7, and 10 N of 30 for accountability (10 for reporting) Continuous enrollment from October 1 through the testing period (ends in mid-May) Standard error at 95% confidence level On-time graduation rate goal = 73% or 1 point above previous year (85% goal in 2014) Unexcused absence rate goal = 1% or a reduction from previous year

6. AYP Yearly Targets Grade 4 Grade 7 Grade 10 YearReadingMathReadingMathReadingMath 200252.229.7 30.117.3 48.624.8 200356.235.635.924.252.931.1 200460.241.441.831.157.237.3 200564.247.347.638.061.543.6 200668.153.153.444.965.749.9 200772.159.059.251.870.056.1 200876.164.965.158.774.362.4 200980.170.770.965.578.668.7 201084.176.676.772.482.974.9 201188.182.482.579.387.281.2 201292.088.388.486.291.487.5 201396.094.194.293.195.793.7 2014100.0 Baseline

7. 56.2 60.2 64.2 68.1 72.1 76.1 80.1 84.1 88.1 92.0 100.0 35.6 41.4 47.3 53.1 59.0 64.9 70.7 76.6 82.4 88.3 100.0 52.2 96.0 29.7 94.1 0 10 20 30 40 50 60 70 80 90 100 2002200320042005200620072008200920102011201220132014 Percent Meeting Standard ReadingMathematics Increment of 4.0 needed Increment of 5.9 needed GRADE 4 YEARLY TARGETS AYP can be made if the percent meeting standard is below the yearly target either via safe harbor or when the standard error is included in the total. (Increments are rounded)

8. Summary of State AYP Results State made AYP in 78 of 111 categories (70%) Of the proficiency categories, the state made 21 of 54 (39%) All Students, Asian/Pac. Is., and White groups made AYP in all grades in both reading and math American Indian, Black, and Low-Income groups made AYP in just one of six proficient categories (Grade 4 Math) Hispanic, Special Education, and Limited English groups did not make AYP in any proficiency category Graduation rate initially 79%, later changed to 66% Unexcused absence rate 0.5%

9. First Year Results Total number of districts and schools not making AYP was very close to projected numbers (42% and 22%) But far above previous results (0% and 3%) Safe harbor helped very little Not making AYP is a function of the N (Secondary schools less likely to make AYP due to larger enrollment in tested grades) Most students are in districts that did not make AYP

10. Responses to NCLB Much more attention being given to data quality and achievement gap (motivation to change) Widespread cynicism about law and lack of funding to meet federal mandates Negative impact on public perception of testing in general Narrowing curriculum to tested subjects Some focus on beating the system rather than serving students – temptation to lower standards

11. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2003 figures include results from the alternate assessment, include the standard error, and are based on results of continuously enrolled students 60.6% Grade 10 Percent meeting standard 52.9% 2003 Goal Grade 4 67.5% 56.2% Grade 7 48.8% 35.9% “All Students” Category in Reading 2002-03 State 2003 Results – Adequate Yearly Progress

12. 2003 Goal 35.6% State 2003 Results – Adequate Yearly Progress

15. Unexcused Absence Rates Among 296 Districts 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 11223344556677889100111122133144155166177188199210221232243254265276287 Percent Unexcused Absences 35 districts (11.8%) with greater than 1%

16. AYP Results on the Web http://reportcard.ospi.k12.wa.us/

17. 296 Districts District 2003 AYP Results (All Grades) 123 90 83 0 20 40 60 80 100 120 140 Made AYPDid Not Make AYPToo small to evaluate (N<30) Number of districts

18. 296 Districts Percent of Students in Districts by AYP Result (All Grades) 79.3% 19.4% 1.3% 0% 20% 40% 60% 80% 100% Made AYPDid Not Make AYPToo small to evaluate (N<30)

19. 0% 20% 40% 60% 80% 100% < 350 350- 1000 1001- 1650 1651- 2300 2301- 2950 2951- 3600 3601- 4250 >4250 Number of students in a district Districts making AYP Odds of a District Making AYP Declines as Enrollment Increases Less than 30 in a grade

20. District 2003 AYP Results by Grade 104 107 110 93 102 89 72 45 97 0 25 50 75 100 125 150 Grade 4Grade 7Grade 10 Number of districts Made AYPDid Not Make AYPToo small to evaluate (N<30) 5 via safe harbor 2 via safe harbor 3 via safe harbor

21. Percent of Students in District by Grade 16.1% 24.1% 18.3% 74.2% 82.6% 80.8% 1.7% 0.9% 1.4% 0% 20% 40% 60% 80% 100% Grade 4Grade 7Grade 10

22. Example District AYP Results Olympia School District, ~9000 students Grade 4 Reading 0 20 40 60 80 100 All studentsAmerican Indian Asian/ Pac Is BlackHispanicWhiteSpecial Education Limited English Low Income Percent meeting standard Students meeting standard *SEP Interval * For students continuously enrolled from October 1 NA (<30) Goal = 56.2 NA (<30) NA (<30)

23. District AYP Results, 2003 Grade 4 Reading by Group 181 7 34 22 32 173 22 10 78 18 4 0 7 33 1 56 29 30 0 50 100 150 200 All Students American Indian AsianBlackHispanicWhiteSpecial Education Limited English Low Income Number of districts Made AYPDid Not Make AYP

24. District 2003 AYP Group Results All Grades in Reading 63.6% 97.6%98.0% 89.9% 47.3% 15.1% 17.9% 70.7% 66.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% All Students American Indian AsianBlackHispanicWhiteSpecial Education Limited English Low Income Percent of cells with 30 or more students making AYP

25. District AYP Results, 2003 Number of Groups Not Making AYP in the 123 Districts 111 possible

26. School 2003 Results (All Grades) Adequate Yearly Progress 386 1177 432 0 200 400 600 800 1000 1200 Made AYPDid Not Make AYPToo small to evaluate (N<30) Number of schools

27. Percent of Students in Schools by AYP Result (All Grades) 43.0% 53.8% 3.1% 0% 20% 40% 60% 80% 100% Made AYPDid Not Make AYPToo small to evaluate (N<30)

28. School 2003 Results (By Grade Level) Adequate Yearly Progress 220 863 152 113 167157 215 231 210 0 200 400 600 800 1000 Grade 4Grade 7Grade 10 Number of schools Made AYP Did Not Make AYP Too small to evaluate (N<30)

29. Percent of Students in Schools by AYP Result and Grade 43.3% 84.7% 34.1% 11.8% 54.1% 62.5% 3.5% 3.4% 2.6% 0% 20% 40% 60% 80% 100% Grade 4Grade 7Grade 10

30. School AYP Results, 2003 Number of Categories Not Making AYP All Grades 67 36 47 21 31 7 26 5 222 190 0 50 100 150 200 123456789101112 Number of cells not making AYP Number of schools

31. Total of 432 Schools Not Making AYP 51 Schools in “School Improvement” 13 schools made adequate yearly progress for the second year in a row and were removed from “school improvement” status. 44 of the 51 schools receive Title I funds 5 of the 51 made AYP in 2003 and remained in school improvement status 17 4 381 30 0 50 100 150 200 250 300 350 400 450 Year 1 (Alert)Year 2/Step 1Step 2Step 3 Improvement Stage Number of schools 164 (43%) of these schools receive Title I funds

32. Proposed Changes Revise on-time graduation rate goals More accountability for small schools (N of 10-29) Increase N for LEP and special education Increase confidence level to 99% Use new flexibility for LEP testing, participation Add recognition system Develop criteria to differentiate need/assistance Clarify appeals process Use new student information system