1. Impact of NCLB Adequate Yearly Progress on District Accountability in Colorado Carolyn Haug, Measured Progress Jonathan Dings, Boulder Valley School District

2. AERA Annual Meeting 4/12/04 Presentation Overview §Evaluation Framework §AYP Impact in Boulder Valley School District §Colorado Context of Multiple Accountability Systems §Toward an Improved Accountability System

3. AERA Annual Meeting 4/12/04 Accountability System Evaluation Framework (Baker & Linn, 2004) §builds staff capacity; §affects resource allocations; §supports high-quality instruction; §promotes student equity access to education; §minimizes corruption; §affects teacher quality, recruitment, and retention; and §produces unanticipated outcomes.

4. AERA Annual Meeting 4/12/04 Consequences §Of AYP Test-Score Driven Accountability §Apart from Consequences of Comprehensive Federal Title Funding Changes §Apart from Consequences of Colorado’s 3 rd -10 th Grade State Testing Program, which predates NCLB’s AYP

5. AERA Annual Meeting 4/12/04 Results of Incentives §Slight Increase in Attention to Student Groups in Improvement Planning, Testing All; District, Schools Already Engaged §Bookkeeping for Disaggregation and AYP Status Calculation (350 person-hours) §Morale(?)

6. AERA Annual Meeting 4/12/04 AYP-Triggered Funding Impact §Apart from Broader NCLB/Title Funding Changes, Professional Development, Parent Engagement, Homeless Services §Expected Net Decrease in Discretionary $ §Further Professional Development §Dollars Reserved for Transportation, A Doubtfully Effective Use §Fewer Schools and Students Served

7. AERA Annual Meeting 4/12/04 District Groups Not Making AYP in 2003 % of Targets Missed Overall 0 Asian 0 White 0 Black 0 American Indian 0 Hispanic33 English Language Learners83 Students With Disabilities33 Free/Reduced Lunch* (not in AYP) 17

8. Effects of Publicizing AYP Results Schools Fail Targets Banner Headline from Boulder Daily Camera Newspaper, 11/19/03

9. AERA Annual Meeting 4/12/04 Impact of AYP Ratings on the Media, Public, and Parents §Schools Fail Targets; Excellent Failures; Excellence Fails to Impress Feds (Boulder Daily Camera articles) §AYP status created dissonance about previously-held beliefs about some BVSD schools §Statewide, similar confusion prevailed: “The great power of AYP is that it doesn’t let Colorado’s best schools cover up with overall good scores those students being left behind.” (emphasis added)

10. AERA Annual Meeting 4/12/04 AYP Competes with Other Colorado Accountability Systems: School Accountability Reports (SAR) and Accreditation §As a result of three separately-enacted laws, Colorado schools are subject to three different school accountability mechanisms: l AYP: federal law, enacted January 2002 l SAR: state school reform legislation, enacted July 2000 l Accreditation: state school reform legislation, enacted July 1998 §Subsequently, schools face potentially three different school ratings

11. AERA Annual Meeting 4/12/04 Complementary or Contradictory Systems?

12. AERA Annual Meeting 4/12/04 SAR and Accreditation Ratings for Schools Failing AYP SAR Rating Based on Spring 2003 School Accreditation Status, 2002-2003 School Year School AAverageFully Accredited School BHighFully Accredited School CHighFully Accredited School DLowAcademic Watch School EHighFully Accredited School FAverageFully Accredited School GExcellentFully Accredited School HLowFully Accredited School ILowFully Accredited

13. AERA Annual Meeting 4/12/04 Size Mattered: Most Large Schools Failed AYP SAR Rating Based on Spring 2003 School Accreditation Status, 2002-2003 School Year School AAverageFully Accredited School BHighFully Accredited School CHighFully Accredited School DLowAcademic Watch School EHighFully Accredited School FAverageFully Accredited School GExcellentFully Accredited School HLowFully Accredited School ILowFully Accredited

14. AERA Annual Meeting 4/12/04 Conclusion: Toward an Improved Accountability System §Accountability is good when it accurately identifies schools and when consequences are reasonable §Results from 3 systems are not synthesized, which leads to serious confusion §Multiple systems yield multiple measures that could be combined to form one contextual, cohesive synopsis of school performance

15. AERA Annual Meeting 4/12/04 Next Steps §The next task is to design a useful, valid methodology for integrating data from AYP, SAR and accreditation that meets the intended purposes of each of the 3 systems: l providing schools with useful feedback about performance in order to improve the school, and l school accountability. §Rather than solely a school-shopping device, a school’s rating would provide information, and therefore opportunities for improvement.