Designing Next Generation Accountability Systems: Big Picture Tony Evers, Wisconsin Superintendent of Public Instruction Marianne Perie and Chris Domaleski, Center for Assessment September 29,

Next Generation Accountability Systems Taskforce Composed of roughly 20 state chiefs and SEA leaders. The Taskforce has participated in robust meetings and discussions to develop a framework for next- generation state accountability systems. In engaging in this work, the Taskforce has relied on its members' experiences in implementing accountability systems over the past two decades along with the knowledge of various experts and the latest research. 2

Outcomes of Taskforce and Moving to State Design Roadmap for Next-Generation State Accountability Systems Statement of Principles and Processes Wisconsin is designing a new differentiated accountability system aligned with the CCSSO principles and roadmap 3

Moving from Waivers to Principles Principles are broader so begin with those Focus first on your goals, then let every decision fall from there – Think of the main points along a logic model (aka Theory of Action) – How will this piece of my accountability plan help ensure more students leave high school ready for college and careers? and/or – How will this piece of my accountability plan help reduce the achievement gap? 4

Basic Decision Points Articulate goals for students, teachers, schools, district, state Determine what measures go into those goals Identify available or attainable data sources Decide how to combine data from sources to create a district/school/teacher “score” Determine annual targets and how to identify focus, priority, reward, and “other” schools Design a diagnostic review system and interventions to help support schools (which ones?) Continually evaluate system 5 All made with consideration of practical constraints and capacity

A Process Necessarily, we will focus on some specific issues and decisions today However, we don’t want to lose sight of the broad and ongoing process at work 6

Improving Accountability - Focus on Implementation Today we will focus on two fundamental attributes of accountability systems to promote improvement in line with CCSSO’s principles and the federal waiver guidelines – Identifying Indicators: What elements are most important to measure in the system? – Design Decisions: How can accountability measures be combined into clear, meaningful outcomes? But let’s start with articulating our goal 7

Consider desired distributions of student scores, accounting for target & variance among students 5 Starting point End A – most/all students on target, variance same as start (could change relative position) End B – most/all students on target, little score variance End C – most/all students on target, more variance College and career ready benchmark

Identifying Indicators Indicators refers to the specific elements or measures that are included in the accountability system. We will focus on refining and/or broadening the set of indicators to provide measures of college and career readiness (CCR), which is featured prominently in CCSSO’s principles and the wavier guidance 9

Types of Performance Indicators Along-the-way Indicators: “Is the student making progress toward readiness standards?” – For example: ‘on-track’ to CCR assessment results or accumulation of course credit Attainment Indicators: “Has the student met readiness expectations?” – For example: achieving readiness benchmarks on assessments, earning a CCR diploma or other credentials such as career endorsements Post Secondary Indicators: “What evidence certifies that the student is achieving post-secondary success?” – For example: enrollment in college or qualifying career path; performance in credit bearing college courses 10

Additional but Familiar Indicators Student attendance High school graduation rates Student participation in assessments Or slightly less familiar AP/IB participation and performance Course-taking, specifically STEM courses 11 Brings focus to Principle of “Timely, actionable, accessible data reported to all stakeholders, including outcome and richer data to drive continuous improvement”

Other Assessment Options Additional subjects – Summative or interim Additional grade levels Normative assessments College-entry assessments – ACT/SAT participation and performance Interim assessments – Pre-post tests to track within year growth 12 Could be included in student outcome measures or other actionable data to support continuous improvement

“Out-of-the-Box” Indicators School environment – Student & staff safety – Student discipline Teacher participation in school – Outside of classroom duties (lunch, bus) – Extra-curricular sponsorship Feedback on teacher performance – Principal observation/evaluation – Expert teacher observation – Peer/parent/student evaluation 13 Information can be used in various ways—it may be that some indicators are used for purposes other than measuring performance outcomes for accountability determinations.

Combining Multiple Components: Assessment Accountability systems that adhere to CCSSO principles and waiver guidelines should move beyond status (e.g. percent on target) alone Such systems will include multiple performance components to reflect: – Growth: Evidence that students are making adequate progress toward meaningful outcomes. – Equity: Evidence of achievement/gains for all groups, especially those that are traditionally low performing. We will discuss specific mechanisms to account for growth and promote equity. 14

Multiple Components, continued In addition, we need to consider how other non- assessment components relate to teacher and school evaluation How do we develop a holistic picture of teacher and school effectiveness without maintaining the current “39 hurdles” approach? Which indicators should go into school accountability versus school improvement or educator effectiveness systems? 15

Setting Targets How can states establish appropriate targets on accountability measures? – Targets must be ambitious but attainable – Provide a credible evidence that ALL students and schools are making progress toward and achieving CCR – Promote equity and excellence – Standards may be customized or differentiated based on relevant factors (e.g. distance to target) 16

Choice for Targets Waiver options – Reduce gap from performance to 100% by 50% over six years – 100% CCR by – Other Avoid “safe harbor” trap by looking longer term Year 1: 60% not proficient Year 2: 54% not proficient Year 3: 60% not proficient Year 4: 54% not proficient Year 5: 60% not proficient Year 6: 54% not proficient 17

Differentiation Different measures? – Performance – Other Different targets? – Status – Growth – Improvement – Reduction in gap Different intervention? – Miss by a little vs miss by a lot (diagnostic review) – CCR vs equity 18

Bringing It All Together How can multiple elements come together into a clear coherent system? – Balance the desire for simplicity and flexibility – Aligned to policy priorities (e.g. the metrics and weights support state values) – Support the need for utility (e.g. results that inform practice) We can discuss examples of design alternatives to illustrate options for specific components in small groups 19

Afternoon Discussions Articulate goals for students, teachers, schools, district, state (Marianne) Determine what measures go into those goals (Marianne and Chris D.) Decide how to combine data from multiple measures to create a school/teacher/district “score” (Chris D.) Determine annual targets and how to identify focus, priority, reward and “other” schools (Marianne) Design a diagnostic review system and interventions to help support schools (Bill and Amy) Waiver policy and process discussion (Chris M. and Scott) 20