Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Articulation in the Context of States’ Transition to the Common Core State Standards How did we get here, Where are we now, and How is that working out? National Conference on Student Assessment June 25-27, 2014

Copyright © 2013 by Educational Testing Service. All rights reserved. How did we get here? Vertical Articulation in the Context of States’ Transition to the Common Core State Standards Patricia A. Baron Educational Testing Service National Conference on Student Assessment June 25-27, 2014

Copyright © 2013 by Educational Testing Service. All rights reserved. Where are we? The current standards and testing movement is in transition from the multi-state standards to the Common Core State Standards (45 states and DC)

Copyright © 2013 by Educational Testing Service. All rights reserved. How did we get here? Historical Highlights U.S. Federal Policy and Initiatives Research – ongoing and results since 1995

Copyright © 2013 by Educational Testing Service. All rights reserved. US Education Policy Title I ESEA Launched development of the field of educational evaluation and school accountability NAEP Subject-area National Survey, Grades 4,8, and 12; subject area frameworks developed by NAGB NAEP State Assessments No Child Left Behind Act Annual testing for all subjects in grades 3-8; Annual progress objectives; State Standards in reading and mathematics ED Invites states to include achievement and growth models; vertical scaling not required but promise of greater comparability RTT Created to spur innovation & reforms in state & local districts; points for complying with CCSS

Copyright © 2013 by Educational Testing Service. All rights reserved. Setting the Stage Three Related Areas of Research and Practice 1.Defining proficiency 2.Linking tests. Linking scales. Vertical scales and growth. 3.Standard Setting: Content and Performance Standards

Copyright © 2013 by Educational Testing Service. All rights reserved. Defining proficiency 1968 NAEP: The Nation’s Report Card. Introduced as a data gathering tool and not an accountability tool; 1988 Scale anchors included; NAGB changed from Can Do to … Should Do ; Standard setting process under fire 2001 NCLB –Increased the amount of testing and increased the consequences to the states States defined proficiency according to state content standards, grade- level assessments and proficiency standards 2003 Currently the variability in the stringency of state standards defining proficiency is so great that the concept of proficient achievement lacks meaning (Linn) The Common Core is designed to promote deep learning through rigorous standards aligned with college and career readiness Defining proficiency

Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Scales & Growth Models Vertical scales introduced with ED invitation (2006) to include growth models in state performance reporting. Methods existed for linking different tests, different scales – Linking statewide tests to NAEP, accuracy of combining test results across states (Linn and Kiplinger, Ercikan) – Linking NAEP to international tests, e.g., TIMMS (Johnson)

Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Scales & Growth Models Issues for measuring growth & change scores – Scales measuring latent variables such as proficiency or achievement are not equal interval scales (e.g., Patz, 2007). ____+____+___+__+__+__+ – Valid vertical scales require a set of content standards that provide continuity across the grade span Learning progressions (e.g., Wilson, 2009)

Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Scales & Growth Models Issues for measuring growth & change scores – Linking tests across grades without an anchor (common items) problematic – Standard setting conducted by grade not sufficient

Copyright © 2013 by Educational Testing Service. All rights reserved. Standard Setting: Content and Performance Standards Content Standards: Content standards define the knowledge, concepts, and skills that students should acquire at each grade level. Performance Standards: These standards specify how much understanding of content students need at each level of performance (e.g., basic, proficient, advanced), relative to the content standards.

Copyright © 2013 by Educational Testing Service. All rights reserved. Standard Setting: Content and Performance Standards Best practice in standard setting – appropriately-informed panelists – alignment between test and content standards Cross-grade expectations and learning progressions: – Cohesive content standards, e.g., Grades 3 to 4 to 5, …Grade 11. Options for cross-grade alignment of performance standards – Vertical scaling and vertical moderation

Copyright © 2013 by Educational Testing Service. All rights reserved. Good sources for further reading The Future of Test-Based Educational Accountability, Ed. Ryan & Shepard, (2008) Vertical Scaling in Standards-Based Educational Assessment and Accountability Systems, published by CCSSO, Rich Patz (2007) Vertical Scaling – in Test Equating, Scaling and Linking: Methods and Practices, Kolen and Brennan (2004)

Copyright © 2013 by Educational Testing Service. All rights reserved. Research Highlights 2003 – Vertical equating for state assessments: Issues and solutions in AYP and school accountability (Lissitz and Huynh) 2005 – Vertically Moderated Standards: Special Issue of Applied Measurement in Education 2009 – Impact of vertical scaling decisions on growth interpretations (Briggs and Weeks) 2010 – Post-standard-setting panel considerations for decision- makers (Geisinger and McCormick) 2012 – Growth, Standards and Accountabiilty (Betebenner) in Cizek, Ed., Setting performance standards

Copyright © 2013 by Educational Testing Service. All rights reserved. Today Current Research Priya Kannan, Research Scientist, Educational Testing Service Current Practice Deb Lindsey, Director of Assessment, Wyoming Department of Education Discussion Marianne Perie, Co-Director at Center for Educational Testing and Evaluation, University of Kansas