Learning from Student Assessment Results: Lessons for New York State Richard J. Murnane and Nancy S. Sharkey Harvard University Graduate School of Education March 2004

Standards-Based Reform Content and performance standards Assessments Instructional materials and professional development Incentives for educators and students

New York State Has strong standards and assessments Pressure to meet standards: –High standards to earn a high school diploma –No Child Left Behind

Lessons for New York State The State should: –Provide student assessment results rapidly –Develop a formative assessment for districts –Continue on plan to develop a statewide student identification system –Compare NYC high stakes test scores to NYS and NYC NAEP scores –Examine effectiveness of BOCES

Uses of Student Assessment Results Instrumental: To make programmatic decisions Conceptual: To enrich dialog Symbolic or Political: To support a predetermined stance or decision

Necessary Conditions for Effective Data Use Assessments aligned to standards and curriculum Technology needs to be user friendly Teachers need to know how to do analyze data and translate findings into instruction Teachers need time to do this work School culture must support this work

New York City’s Challenge More than 1 million students 139,000 receive special education services 126,000 are English Language Learners 147 languages are spoken in the schools In , 59% of elementary and middle school students did not meet state standards in reading; 58% did not meet standards in math

New York City’s Response Assessment system –High stakes citywide exams in grades 3, 5, 6, 7 –Statewide high stakes tests in grades 4 and 8 –Citywide formative assessments 3 times/year –Monthly formative assessments in Region 2 Handling data –The Grow Network –Princeton Review

The Grow Network Summative assessment data reporting Paper reports of how students performed by standard Computer display mirrors paper report, with links to content and pedagogical support Links for parents and students

Strengths and Limitations Strengths –Reports are easy to read –Enable teachers to differentiate instruction –Curriculum neutral approach; geared to standards Limitations –Timing –Limited questions and broad standards make standards- based analysis hard to interpret –Curriculum neutral approach

Princeton Review Formative assessments three times/year Test developed, scored, and analyzed Tests online or on paper Reports online only Online content and pedagogical links Links for parents and students

Strengths and Limitations Strengths –Reports of performance against standards and by individual test item –Helps educators see where students are throughout the year –Aligned with state standards Limitations –Assessments not linked to school curriculum –Technological hurdles –Hard to identify who is using reports (one teacher may print out the report for many other teachers)

The Princeton Review in Region 2 Monthly formative assessments in math in grades 2 through 10 Assessments on paper Math consultants provide in-person professional development, based on needs identified on assessments

Common Concerns Do scores on the assessments accurately reflect mastery of state learning standards? Are teachers teaching to the test or teaching to standards? Are formative assessments being used as low- stakes tests, high-stakes tests, or not at all?

Lessons for New York State The State should: –Provide student assessment results rapidly –Develop a formative assessment for districts –Continue on plan to develop a statewide student identification system –Compare NYC high stakes test scores to NYS and NYC NAEP scores –Examine effectiveness of BOCES