1. Robert L. Linn CRESST, University of Colorado at Boulder Paper presented at a symposium sponsored entitled “Accountability: Measurement and Value-Added Models”, at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL, April 12, 2007 Approaches to Educational Accountability

2. Test-based Accountability Popular tool for purposes of educational reform Accountability is one of few tools available to policymakers to leverage changes in instruction In use in many states since the early 1990s Quite a range of approaches to using student test results for accountability systems Central component of NCLB

3. Approaches to Test-Based Accountability Status Approach: compare assessment results for a given year to fixed targets (the NCLB approach) Growth Approach: evaluate growth in achievement (allowed for NCLB pilot program states) “Growth” may be measured by comparing performance of successive cohorts of students Growth may be evaluated by longitudinal tracking of students from year to year

4. Status and Growth Approaches Status approach has many drawbacks when used to identify schools as successes or in need of improvement Does not account for differences in student characteristics, most importantly differences in prior achievement Growth approach has advantage of accounting for differences in prior achievement, but may set different standards for schools that start in different places

5. NCLB Pilot Program Five states have received approval to use growth model approaches to determining AYP Early results suggest that it does not radically alter the proportion of schools failing to make AYP Constraints on growth models are severe, most notably the retention of the requirement that they lead to the completely unrealistic goal of 100% proficiency by 2014

6. Value-Added Models Popular Name suggest that the models provide the basis for causal inferences about school and teacher effectiveness Several variations in analytical approach used in value-added models

7. Validity of Causal Inferences Status approach does not provide a defensible basis for inferring that higher scoring school is more effective than a lower scoring school Making an inference about school quality requires the elimination of many alternate explanations of differences in student achievement other than differences in instructional effectiveness Prior achievement differences Differences in support from home

8. Inferences About Schools Growth models rule out the alternate explanation of differences in prior achievement Nonetheless, causal inferences about school effectiveness are not justified by the growth approach to test-based accountability (Raudenbush, 2004, Rubin, Stuart, & Zanutto, 2004) Many rival explanations to between-school differences in growth besides differences in school quality or effectiveness Results better thought of as descriptive for generating hypotheses about school quality that need to be evaluated

9. School Characteristics and Instructional Practice School differences in achievement and in growth describe outcomes and can be the source of hypotheses about school effectiveness Accountability systems need to be informed by direct information about school characteristics and instructional practices

10. Conclusions Test-based accountability has become a pervasive part of efforts to improve education in the U.S. The features of accountability systems matter Requirement to include nearly all students in test-based accountability has brought needed attention to groups often ignored in the past

11. Conclusions (continued) Performance standards are supposed to define the level of achievement that students should reach, but Using percent proficient or above a primary indicator does not give credit for gains of students at other levels Using percent proficient or above to monitor gaps in achievement is not an adequate approach

12. Conclusions (continued) Status-based approach to accountability does not provide a valid way of distinguishing successful schools from schools that are in need of improvement Growth models have advantages over status models but still are best thought of as providing descriptive information rather than the providing the basis for causal inferences about school quality