1. University of Colorado at Boulder National Center for Research on Evaluation, Standards, and Student Testing Challenges for States and Schools in the No Child Left Behind Act of 2001 Robert L. Linn Presentation at The Western Knight Center for Specialized Journalism, USC, February 1, 2003

2. Major Challenges Adequate Yearly Progress (AYP) Student performance standards Keeping number of schools categorized as “needs improvement” and facing sanctions to a manageable and reasonable number Year-to-year volatility of school-building results

3. Adequate Yearly Progress (AYP) Central to the Accountability System of the No Child Left Behind (NCLB) Act of 2001 States required to define AYP for the State, school districts, and schools in a way that enables all children to meet the State’s student achievement standards by 2014

4. Some Key Criteria for State’s AYP Definitions Same high standards of academic achievement to all public elementary school and secondary school students in the State Statistically valid and reliable Results in continuous and substantial academic improvement for all students

5. Some Key Criteria for State’s AYP Definitions (Continued) Annual measurable AYP –Mathematics –Reading/language arts –Substantial and continual progress All students considered as a whole Subgroups of Students

6. Subgroups of Students Identified for AYP Economically disadvantaged students Major racial and ethnic groups Students with disabilities Students with limited English proficiency

7. AYP Starting Point Starting point defined in 2001-2002 The larger of either 1.The percentage of students in the lowest scoring subgroup who achieve the proficient level or higher, or 2.The percentage proficient or higher in the school at the 20 th percentile, based on enrollment, among all schools ranked by the percentage of students at the proficient level or higher.

8. Annual Measurable Objectives Separate for mathematics and reading/language arts State specified increases from starting points in 2221-2002 to 100% in 2013-2014 May combine across grade levels within a subject for school, district, or State

9. High Performance Standards “Proficient” academic performance not defined in NCLB States must establish standard of proficient performance Many states already have quite ambitious performance standards NAEP Proficient level provides ambitious benchmark

10. Percentages of Students at the Proficient Level or Above and the Advanced Level on NAEP in 2000 PercentProficientPercentAdvanced Grade ReadingMathematicsReadingMathematics 4322683 8332735 12401762

11. TIMSS Grade 8 Mathematics Results for Selected Countries Equivalent NAEP Achievement Level <= Advanced <= Proficient <= Basic

14. Number of States Participating in Both Years of NAEP that Showed Gains of One Point Per Year or More at the Proficient Level or Higher SubjectGradeYearsNumber of States Reading492 to 983 of 34 Math492 to 0015 of 34 Math890 to 0018 of 29

16. Michigan Trends on MEAP and NAEP Year 1: the percentage satisfactory on MEAP fell between the NAEP trend lines for the percent proficient or better and percent basic or better MEAP gains faster that NAEP gains for several years then levels off near NAEP percent basic or above trend line

18. Maryland Trends on MSPAP and NAEP Trend lines are at different levels reflecting different definitions of standards Slopes of trend lines are quite similar

20. Texas Trend Lines on TAAS and NAEP TAAS trend steeper than NAEP trend TAAS more of a minimum-competency test while NAEP tests more demanding subject matter TAAS has been replaced by a more demanding test

21. Illustrations of AYP Targets Starting points 52% proficient or above in reading/language arts and 40% proficient or above in mathematics Annual gains required –Reading/language arts: 4% = (100% - 52%)/12 –Mathematics: 5% = (100% - 40%)/12 2003-2004 targets: –Reading/language arts: 60% = 52% + 2(4%) –Mathematics: 50% = 40% + 2(5%)

22. Schools Have Many Ways to Fail to Meet AYP Objectives Reading Mathematics Less than 95% student participation in testing White, African American, and Hispanic subgroups Economically disadvantaged students Limited English proficient students Students with disabilities

23. “Safe Harbor” Exception If one subgroup fall short of AYP target school can still avoid being placed in needs improvement category if 1.The percentage of students who score below the proficient level is decreased by at least 10% from the year before, and 2.There is improvement for that subgroup on other indicators

24. Variability in Stringency of Progress Requirements for Different Schools: 2003-2004 AYP Target is 50% Percentage of students proficient or above in 2001- 2002 School A: 30% School B: 45% School C: 75% Needed Increases in percent proficient or above by 2003-2004 School A: 20% School B: 5% School C: could decline by up to 25%

25. Scatterplot of Percent Proficient or Above for Schools on Colorado Grade 4 Reading Assessments in 1997 and 1999

26. Number of Schools in Needs Improvement Category State and districts need to provide assistance Schools face serious sanctions if they remain in category Risk that substantial majority of schools will be identified Need to keep number manageable and reasonable

27. Five States with Approved Plans Colorado, Indiana, Massachusetts, New York and Ohio Each state has plan that is expected to keep number of schools in needs improvement category within reasonable bounds Other states likely to follow similar routes

28. Illustrations of Approaches of Three States with Approved Plans Colorado: Lower proficient performance standard for NCLB purposes Massachusetts: Use index score rather than percent proficient or above Ohio: Set targeted increases in percent proficient or above that start modest and increase in later years

29. Colorado Has had four levels of performance (advanced, proficient, partially proficient, and unsatisfactory) for state accountability. Will continue to use four levels for state and local purposes. Defines proficient for NCLB to equal the combined categories of partially proficient and proficient. Results in high values for starting percentages proficient or above

30. Massachusetts Proficiency Index Measures How Close A School Is To All Students Being Proficient/Advanced Failing/Warning Needs Improvement Proficient Advanced 240 220 200 260 0 points 100 points 25 points 50 points 75 points Target For All Students

32. Volatility of School-Level Results Absolute performance reasonably stable from year to year for most schools Changes in performance from one year to the next are much less stable Schools that gain a lot from year 1 to 2 tend to lose from year 2 to year 3 Small schools less stable than large ones Disaggregation exacerbates the volatility

33. Correlations of Annual School Changes in the Percent Proficient or Above in Grade 4 Reading on CSAP Change ‘97 to ‘98 Change ‘98 to ‘99 Change ‘99 to ’00 Change ‘97 to ’98 1.00-.489.02 Change ‘98 to ’99 -.4891.00-.492 Change ‘99 to ‘00.02-.4921.00