2. Purpose of the study This study examines the NCLB accountability systems for 28 states. We took 36 real schools from around the nation (18 elementary, 18 middle) and asked: - How many would make AYP in each state? - What explains the variation? 2

3. AYP provisions examined 1. Proficiency cut scores in math and reading for 2006 2. Annual performance targets (“AMOs”) for 2008 3. Minimum subgroup sizes (“n sizes”) for 2008 4. Margins of error (“confidence intervals”) for 2008 3

4. Limitations of the study We were not able to consider: - NCLB’s “safe harbor” provision - Growth models - Nuances related to the exclusion of special education students and English Language Learners 4

5. Finding #1: Huge variation in AYP ratings for our elementary schools Almost all our sampled schools failed to make AYP in some states, and nearly all of these same schools made AYP in others. In Massachusetts, only 1 of 18 elementary schools made AYP. In Wisconsin, 17 of these same 18 schools made AYP. Same kids, same academic performance, same schools—and very different results. 5

6. Number of sampled elementary schools that made AYP in 2008, by state 6

7. Finding #2: There was less variation for our middle schools because most did poorly everywhere In 21 states, two or fewer middle schools made AYP. In no state did even half of the 18 middle schools make AYP. This is mostly because of the larger size of middle schools which means they are held accountable for the performance of more subgroups. 7

8. Finding #3: “Cut scores” are important, but so are annual targets Some states (like California) have high cut scores but still permit nearly all schools, including poor performers, to make AYP because of low annual targets. But other states that have been criticized for their low NCLB cut scores (e.g., Colorado), have tough annual targets that seem reasonable relative to their tests. 8

9. Overall proficiency rates of the elementary school sample in math 9

10. Math proficiency rates of the elementary school sample relative to each state’s 2008 targets 10

11. Finding #4: State determinations around “subgroup size” are critical Generally:  the lower the state’s subgroup size (“n size”),  the more subgroups for which the typical school is accountable,  and the more separate targets that school must hit. 11

12. Case in point: High-performing Chaucer Middle School 12

13. Elementary subgroup performance of sample schools under the 2008 Ohio AYP rules 13

14. When the special ed or ELL subgroups “count,” the school almost always fails 14

15. Implications For an accountability system to be effective, educators must believe that it is fair, consistent, and understandable. Unfortunately, the way NCLB rates schools appears to be idiosyncratic—even random—and opaque. The success or failure of a given school under NCLB is driven as much by the way the law is implemented by its home state as its student performance. 15

16. This is the Accountability Illusion. 16

17. 17