An Overview of the Race-to-the-Top Assessment Consortia Joe Willhoft, Executive Director SMARTER Balanced Assessment Consortium NATD Annual Meeting April 8, 2011 New Orleans

2 2 Apr. 8, 2011 Council of Chief State School Officers (CCSSO), National Governors Association (NGA), and Achieve, Inc. agree to partner on a common standards project Summer 2009: College and career readiness standards developed in English/Language Arts and Mathematics K-12 learning progressions developed leading to college and career readiness in high school Multiple rounds of feedback from states, teachers, researchers, higher education, and the general public June 2, 2010: Final Common Core State Standards (CCSS) released

3 3 Apr. 8, 2011 Aligned with college and work expectations Focused and coherent Include rigorous content and application of knowledge through high-order skills Build upon strengths and lessons of current state standards Internationally benchmarked so that all students are prepared to succeed in our global economy and society Based on evidence and research State led – coordinated by NGA Center and CCSSO

4 4 Apr. 8, 2011 The US Department of Education’s Race-to-the-Top competition awarded extra points to a state’s application if the state provided evidence of adopting, or moving toward adoption of “a” common core of standards When adopting the standards, states may not remove standards, but may add to them as long as the added state-specific standards comprise no more than 15% of the total

5 5 Apr. 8, 2011 As of April 1, 2011, 43 states including the District of Columbia have adopted the Common Core State Standards The eight states that have not yet adopted the standards: Alaska, Minnesota, Montana, Nebraska, North Dakota, Texas, Virginia, Washington

6 6 Apr. 8, 2011 For more information about the CCSS, go to...

7 7 Apr. 8, 2011 The MOSAIC Formative Assessment Consortium In addition to the points awarded in RTTT applications for adoption of the Common Core, states get points if they are part of an assessment consortium for formative (not summative) assessments. In Oct/Nov 2009, Wisconsin, Nebraska, Missouri, Iowa, and Kansas establish a formative assessment consortium, called MOSAIC. About states sign MOUs to join the MOSAIC consortium

8 8 Apr. 8, 2011 The SMARTER Summative Assessment Consortium In fall of 2009 the US Dept of Ed signals it will divert $360M from RTTT for summative assessments aligned to a common core, and begins planning meetings December 2009: Oregon, Idaho, Washington, Utah, Hawaii confer and agree to collaborate on an online, adaptive summative assessment that would include performance items The “Summative Multi-state Assessment Resources for Teachers and Educational Researchers” (SMARTER); Web site is launched; weekly planning calls begin

9 9 Apr. 8, 2011 SMARTER and MOSAIC Merge About 20 states are engaged in both SMARTER and MOSAIC January 2010: MOSAIC states want their formative and interim tools to align to a common summative scale – MOSAIC elects to merge with SMARTER and adopt the SMARTER name; MOUs are merged

10 Apr. 8, 2011 SMARTER merges with Balanced Assessment System About 20 states (leadership from New England Common Assessment Program and West Virginia) work with Linda Darling-Hammond on a consortium that would include a balance of stand-alone tests and extensive performance events and projects March 2010: The Balanced Assessment Consortium and SMARTER join to work on a common proposal as the “SMARTER Balanced Assessment Consortium”

11 Apr. 8, 2011 PARCC has an overlapping history Fall 2009 conversations among chiefs in Florida, Massachusetts, and nine or ten other states to develop a common assessment with a design similar to that of Florida, with a level of rigor at least at the level of Massachusetts. Early engagement with Achieve, Inc. as a development partner Commitment to a “through course” design, with focus on summative results

12 Apr. 8, 2011 Fall 2009, declares $350M for summative assessments USED meetings, with invited papers and testimony about what a multi-state assessment program should look like: Nov (Boston) Nov (Atlanta) Dec. 1-2 (Denver) NRC Board on Testing and Assessment: 2 conferences on new directions for state assessments: Dec and Apr. 6-7 Through March, no indication on what the assessment grant will look like

13 Apr. 8, 2011 April 7: USED release of “Notice Inviting Applications” (NIA) for the Race to the Top Assessment Program Applications due in 11 weeks No more than two awards up to $150M each for summative ELA/Math assessments in grades 3-8 and high school, with possible supplement in $10M pieces No more than one high school course assessments; $30M Grants for development only; preparing assessments that can be given in

14 Apr. 8, 2011 “Applicant” is a single state acting on behalf of the consortium; consortium must have at least 15 states with at least 5 governing states Two types of membership: Governing state (part of only one consortium, part of decision-making) or advisory (can be in both consortia, less involved in decisions) Member states submit an MOU signed by Governor, Chief School Officer, President of State Bd of Ed, Chief Procurement Officer Membership condition: States must adopt the consortium’s common core by Dec. 31, 2011 or drop Acquire services of a Project Management Partner

15 Apr. 8, 2011 Assess acquisition of and progress toward “college and career readiness (students able to take credit-bearing courses in English/math upon entry to college) Have common, comparable scores across all consortium states Provide achievement and growth information that can be used for teacher and principal evaluation and professional development Assess all students, except those with “significant cognitive disabilities” Be administered online, with timely results Use multiple measures Federal Register /Vol. 75, No. 68 / Friday, April 9, 2010 pp

16 Apr. 8, states – 17 Governing, 14 Advisory WestEd was as interim Project Mgt Partner Substantial foundation support from Sandler, Hewlett, and Gates foundations for WestEd contract and convening of consortium states; CCSSO provided fiscal management Requested $150 million and a $10M supplemental Obtained support from higher ed institutions (IHEs) representing > 75% of 2009 college-enrolled seniors

17 Apr. 8, 2011 Two comprehensive assessment awards to the only two bidders: SMARTER Balanced Assessment Consortium (SBAC) Partnership for the Assessment of Readiness for College and Careers (PARCC) Both awarded their entire request: SBAC ~ $160M PARCC ~$170M No award for high school course exams (one unsuccessful bidder) Supplemental $16M to SBAC and PARCC; to support states’ implementation of Common Core

18 Apr. 8, 2011 Fiscal Agent: Washington State 17 Governing States 12 Advisory States CT, HI, ID, KS, ME, MI, MO, MT, NC, NH, NM, NV, OR, UT, VT, WA, WI, WV AL, CO, DE, IA, KY, ND, NJ, OH, PA, SC, SD, WY Membership a/o

19 Apr 8, States in the SMARTER Balanced Consortium (a/o March 31, 2011)

20 Apr 8, Governing Board State Participating State 25 States in the PARCC Consortium (a/o March 31, 2011)

21 Apr. 8, 2011

Consortia Designs

23 Apr. 8, 2011 An eligible applicant’s application must include... “...(a) An executive summary of the eligible applicant’s proposed project; (b) A theory of action that describes in detail the causal relationships between specific actions or strategies in the eligible applicant’s proposed project and its desired outcomes for the proposed project, including improvements in student achievement and college- and career-readiness; (c) A plan for designing and developing the proposed assessment system; (d) A plan for research and evaluation of the proposed assessment system; (e) A plan for implementing the proposed assessment system; and (f) A project management plan (including a workplan and timeline)...” Federal Register; April 9, 2010 (p )

24 Apr. 8, 2011 Grounded in a thoughtful, standards-based curriculum and represent an integrated system Assessments produce evidence of student performance Teachers involved in development and scoring of assessments System is state-led with transparent governance Assessments are structured to continuously improve teaching and learning Useful information on multiple measures that is educative for all stakeholders Adhering to established professional standards

25 Apr. 8, 2011 State policies and practices that support consortium goals Clear and rigorous content and skill targets for college and career readiness Effective communication of policies and standards Supports and resources for teachers Technology supports for learning and assessment systems Rigorous summative assessments Interim assessments and formative tools providing actionable information Teachers engagement in design, scoring, reporting of assessments Assessment information designed to improve learning

26 Apr. 8, 2011 The SMARTER Balanced Theory of Action All students leave high school college and career ready Summative adaptive assessments are benchmarked to college and career readiness Technology supports innovative and comprehensiv e assessments Technology provides increased access to learning State policies and practices support increased expectations Common Core State Standards specify K-12 expectations for college and career readiness Clear communication of expectations to stakeholders Professiona l capacity- building PD and other supports for teachers to instruct on the CCSS Teachers design and score assessment items and tasks Interim assessments are used as progress checks Teachers use formative tools and practices to improve instruction

27 Apr. 8, 2011 How do we get from here......to here? All students leave high school college and career ready Common Core State Standards specify K-12 expectations for college and career readiness...and what can an assessment system do to help?

28 Apr. 8, 2011 The SMARTER Balanced Theory of Action All students leave high school college and career ready Adaptive summative assessments benchmarked to college & career readiness Common Core State Standards specify K-12 expectations for college and career readiness

29 Apr. 8, 2011 Summative assessments using online computer adaptive technologies Efficiently provide accurate measurement of all students, across the spectrum of knowledge and skills, with shorter tests Incorporate adaptive precision into performance tasks and events Will assess full range of CCSS in English language arts and mathematics; will include a variety of item types Describe both current achievement and growth across time, showing progress toward college- and career-readiness Scores can be reliably used for state-to-state comparability, with standards set against research-based benchmarks The option of giving the summative tests twice a year.

30 Apr. 8, 2011 The SMARTER Balanced Theory of Action All students leave high school college and career ready Adaptive summative assessments benchmarked to college & career readiness Common Core State Standards specify K-12 expectations for college and career readiness Interim assessments that are flexible and open

31 Apr. 8, 2011 Optional interim assessments Are aligned to and reported on the same scale as the summative assessments Help identify specific needs of each student, so teachers can provide appropriate, targeted instructional assistance Incorporate significant involvement of teachers in item and task design and scoring Are non-secure and fully accessible for use in instruction and professional development activities Provide students and teachers with clear examples of the expected performance on common standards.

32 Apr. 8, 2011 The SMARTER Balanced Theory of Action All students leave high school college and career ready Adaptive summative assessments benchmarked to college & career readiness Common Core State Standards specify K-12 expectations for college and career readiness Teachers can access formative tools and practices to improve instruction Interim assessments that are flexible and open

33 Apr. 8, 2011 Web-based formative assessment resources Online resources on assessment literacy, aligning assessments to CCSS, and formative assessment guides Training for local development of item and tasks and design and use of scoring guides Support of best practices through online learning modules Comprehensive information portal, providing: educator access to information about student progress toward college- and career-readiness exchange of student performance history across districts and states

34 Apr. 8, 2011 The SMARTER Balanced Theory of Action All students leave high school college and career ready Adaptive summative assessments benchmarked to college & career readiness Common Core State Standards specify K-12 expectations for college and career readiness Teachers can access formative tools and practices to improve instruction Interim assessments that are flexible and open

Optional Interim assessment system — no stakes Summative assessment for accountability Last 12 weeks of year* DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools. Scope, sequence, number, and timing of interim assessments locally determined 2 PERFORMANCE TASKS Each: Reading/Writing Math COMPUTER ADAPTIVE TESTS w/ Re-take Option The SMARTER Balanced Design * Windows may be adjusted based on results from the research agenda and final implementation decisions. English Language Arts and Mathematics, Grades 3 – 8 and High School Computer Adaptive Tests and Performance Tasks INTERIM ASSESSMENT Computer Adaptive Tests and Performance Tasks INTERIM ASSESSMENT (Source: ETS K-12 Center for Assessment and Performance Management)

Through-course ASSESSMENT4 Speaking Listening 25% Through-course ASSESSMENT 1 ELA Math 50% Through-course ASSESSMENT 2 ELA Math 90% END OF YEAR COMPREHENSIVE ASSESSMENT 75% Through-course ASSESSMENT 3 ELA Math PARTNERSHIP RESOURCE CENTER: Digital library of released items, formative assessments, model curriculum frameworks, curriculum resources, student and educator tutorials and practice tests, scoring training modules, and professional development materials Summative assessment for accountability Required, but not used tor accountability 9-Oct-15 English Language Arts and Mathematics, Grades The PARCC Design

37 Jamal Abedi UC Davis/CRESST Randy Bennett ETS Derek Briggs University of Colorado Greg Cizek University of North Carolina David Conley University of Oregon Linda Darling-Hammond Stanford University Brian Gong The Center for Assessment Ed Haertel Stanford University Joan Herman UCLA/CRESST Jim Pellegrino University of Illinois, Chicago W. James Popham UCLA, Emeritus Joe Ryan Arizona State University Martha Thurlow University of Minnesota/NCEO

38 Henry Braun Boston College Bob Brennan University of Iowa Derek Briggs University of Colorado Wayne Camera College Board Linda Cook Retired, ETS Ronald Hambleton University of Massachusetts Gerunda Hughes Howard University Huynh University of South Carolina Michael Kolen University of Iowa Suzanne Lane University of Pittsburgh Robert Luecht University of North Carolina, Greensboro Jim Pellegrino University of Illinois at Chicago Barbara Plake University of Nebraska, Lincoln Rachel Quenemoen National Center for Educational Outcomes Laurie Wise Human Resources Research Organization (HumRRO)

Challenges

40 Apr. 8, 2011 Multiple testing occasions per year in grades 3-8 and high school Infrastructure readiness Student readiness / Opportunity-to-learn issues Rapidly-changing platform technologies Software interoperability Maintaining interoperability across consortia 2 CCSS consortia; 2 “1%” consortia; 2 ELD consortia; 2 science consortia(?) Standardization vs. Innovation

41 Apr. 8, 2011 Development of growth scales “Progress toward college / career readiness Within grade or out-of-level testing? Comparability across consortia Standard setting for college/career readiness What to use for empirical criteria and database? Does college ready = career ready? Reliable and Feasible Scoring of New Item Types Reliance on AI scoring

42 Apr. 8, 2011 Who does what in 2014? RTTA is for development only; consortia end in 2014 Within grade or out-of-level testing? Comparability across consortia Research needs: Bridge studies (old-to-new) and Comparability studies (across states) Development and Use of Common Protocols Common accommodations Loss of local control?

43 Apr. 8, 2011 Ensuring that clear communications are well understood Across multiple states Down to the school/classroom level Maintaining coherent and effective governance Perils of being too top heavy Risks if too disconnected from leadership

44 Apr. 8, the SMARTER Balanced Assessment Consortium can be found online at