Beyond Peer Review: Developing and Validating 21st-Century Assessment Systems National Conference on Student Assessment New Orleans, LA June 25, 2014 Ellen Forte, edCount, LLC Les Morse, Alaska Department of Education & Early Development Steve Sireci, UMass-Amherst Michael Hock, Vermont Agency of Education Thanos Patelis, Center for Assessment Kristen Huff, New York Regents Research Fund Six 10-minute presentations followed by a 30-minute discussion period

An Overview of Federal Peer Review of States’ Systems of Standards and Assessments Ellen Forte, Ph.D. CEO and Chief Scientist, edCount, LLC Presentation at the National Conference on Student Assessment New Orleans, LA June 25, 2014

History: IASA OESE reached out to members of the CCSSO Title I SCASS in 1998 to draft peer review guidance under the 1994 Improving America’s Schools Act This guidance was used to review states’ standards, assessment, and accountability systems beginning in 1999 Only about 1/3 of states had systems that met basic requirements for assessments in math and reading/language arts in the grade ranges Major problem areas: Alignment Alternate Assessments

History: NCLB OESE convened two small groups of consultants to draft two guidance documents: Standards and Assessments Accountability The Standards and Assessments peer review process took the same form as had been used under IASA: centralized review by panels of 3-4 ‘peers’ For the Accountability peer review process, peers and feds traveled to each state

Reliable and Valid Criteria – the peer review guidance ESEA legislatio n Joint Standard s Structur e

Process OESE vets and hires peers States submit evidence Peers review evidence and produce a report Feds identify findings. States get news.

Concerns The US Department of Education is obligated to ensure that states’ systems meet federal mandates, but who in that agency is qualified to or should make judgments about validity? States’ experiences varied widely depending on the peers and the federal program officer assigned to them What role could or should NCME play?