Scott Cook CONTENT DIRECTOR Idaho State Department of Education Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT VS Current ISAT The purpose of this slide deck is to briefly highlight the key differences between the construct of the current ISAT test and the new Smarter-Balanced test aligned to the Common Core that features the evidence centered design model. Scott Cook CONTENT DIRECTOR Idaho State Department of Education

SBAC Item Types-Not the same old test Current ISAT is 100% multiple choice, SBAC is……. Computer adaptive, selected response Technology enhanced,[drag and click] Constructed response [paragraph or two] Emphasis on Performance Assessment [@50% of SBAC tests]: Performance tasks, multiple standards inform, use texts, video, speaking, listening, writing Sample items and Content Specifications docs: http://www.smarterbalanced.org/smarter-balanced-assessments/ Item Writing Modules: http://www.smarterbalanced.org/smarter-balanced- assessments/item-writing-and-review/ While the current ISAT test consists solely of multiple choice items, the SBAC assessment will employ a wide range of item types, the depth and complexity of which is best exemplified by the performance tasks for ELA/Literacy and Math. Although the ISAT is a very valid and reliable measure of student achievement, SBAC offers a more 3 dimensional view of student achievement. The emphasis on performance assessment brings an active, integrated, and authentic focus to the new assessment, providing a platform for instruction, many formative uses, and critical implications for instructional design. For a complete description of the SBAC assessment design, including discussion of evidence centered design, assessment targets, claims and reporting categories, see the Content Specifications document at the link above. The item writing modules break down the new assessment design in 25 short presentations which include subjects such as universal design for learning, bias and sensitivity, and the connection of claims to assessment targets and targets to standards. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Objective 1 Goal 2 Reading: Standard 2 Determine author’s purpose for writing various texts. [finer grain-a twig] Acquire strategies and skills for comprehending text [smaller branch] Metaphorically, the current ISAT test is like a tree: each item is written to the single objective level, with no items written with multiple standards or parts of standards in mind. Put another way, each separate piece of the carpet is unraveled and examined separately. This example shows the lineage of a single objective for Reading. Reading: Standard 2 Comprehension/Interpretation [a big branch] Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Test Design Contrast: Current ISAT Design [Reading] Content area: Reading is the trunk/whole tree Standard 2: Comprehension/Interpretation [a big branch] Goal 2: Acquire strategies and skills for comprehending text [smaller branch] strategies and skills for comprehending text [smaller branch] Objective 1: Determine author’s purpose for writing various texts. [finer grain-a twig] ALL ISAT items are written to the Objective level [Same example as previous slide in text only] Metaphorically, the current ISAT test is like a tree: each item is written to the single objective level, with no items written with multiple standards or parts of standards in mind. Put another way, each separate piece of the carpet is unraveled and examined separately. While it is true that items on the current ISAT test can reach the Depth of Knowledge 3 level, extended work to the DOK 4 level is not possible with the all multiple choice construct. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Standard/Big branch, Goal/Small branch, Objective/Twig Notice how many twigs there are to which we could write items ? That is why we select only some objectives to be tested on the current ISAT Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

SBAC-Evidence Centered Design: Wheel=Claim, Spoke=Target, Hub=CCSS [ELA/Literacy] Spokes/Assessment Targets- Informed by one or more standards, provide evidence/support Wheel/Claim-an assertion needing evidence Hub/CCSS- it all starts here, inform targets Metaphorically, The SBAC test design is similar to the supporting structure of a bike wheel: The hub which supports the whole wheel is the CCSS- from this point all things are created and supported in the evidence centered design model, the model SBAC uses. Think of the outer wheel, sitting unsupported, as the claim. What is a claim?-a broad assertion we make that must be supported with evidence. For example, we ‘claim’ that if the CCSS are taught effectively and with fidelity, students will be able to read closely and analytically to comprehend ….a range of text….this is Claim One [Reading] for ELA/Literacy. There are 4 claims total for ELA/Literacy one each for reading, writing, speaking and listening, and research. Each claim is supported by a number of assessment targets. See the Content Specifications at the SBAC site for the complete matrix. But how do we support this broad ‘claim?’ By writing items to certain assessment targets. Targets are formed and informed by one or more standards or parts of standards. This reflects the integrated nature of the CCSS-seeing how multiple skills can be assessed in concert as in the performance tasks. The Targets are like spokes supporting the outer wheel, providing evidence for the claim. In contrast with ISAT, the separate threads are woven together in a tapestry-this is a massive paradigm shift from ISAT’s isolation-by-objective design, with important implications for instructional design and delivery. And all the assessment targets for each claim are supported, as is the whole wheel, by the Hub, the Common Core State Standards. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Evidence Centered Design: Claim/target [ELA/Literacy] Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational text. Target 4: Reasoning and Evaluation-use supporting evidence to justify/explain inference Target 4 informed by Standards:RL-2, RL-3, RL-6 All items are written to the ASSESSMENT TARGETS-targets are informed by one or more standards or parts of standards, targets provide evidence to support the claim In this example, Claim 1, Reading is supported by Target 4, Reasoning and Evaluation. This target provides some of the evidence for the broad claim about reading. This particular target is informed by and anchored in three common core reading standards: RL standards 2,3 and 4. Reading Literature 2: Determine Central Ideas or themes…. Reading Literature 3:Analyze how a story’s plot unfolds…. Reading Literature 4: Determine word meanings including figurative and connotative…and how they shape meaning… RECAP: The strength of wheel is derived from the hub, transferred to the outer wheel by the spokes. Evidence centered design supports the integrated nature of the common core. Examine any performance task and you will see that many standards or parts of standards are assessed via the assessment targets connected to the task. This integrated, holistic way of assessing mirrors the expectations of the CCSS and has important implications for instructional design. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Test Design Contrast: Current ISAT Design [Mathematics] Content area: Mathematics is the trunk/whole tree Standard 1: Number and Operations [a big branch] Goal 2: Understand and use numbers [smaller branch] Objective 3: Count value of collection of bills and coins up to $10.00 [gr.4] [finer grain-a twig] ALL ISAT items are written to the Objective level Metaphorically, the current ISAT test is like a tree: each item is written to the single objective level, with no items written with multiple standards or parts of standards in mind. Put another way, each separate piece of the carpet is unraveled and examined separately. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

SBAC-Evidence Centered Design: Wheel=Claim, Spoke=Target, Hub=CCSS [Math] Spokes/Assess.Targets Claim 1 informed by clusters [1 or> Stand]. Claims 2,3,4 derived from Math Practices --[evidence/support ] Wheel/Claim-an assertion needing evidence Hub/CCSS- it all starts here, inform targets Metaphorically, The SBAC test design is similar to the supporting structure of a bike wheel: The hub which supports the whole wheel is the CCSS- from this point all things are created and supported in the evidence centered design model, the model SBAC uses. For math, Claim one, ‘Students can explain and apply mathematical concepts and carry out mathematical procedures with precision and fluency,’ is supported by assessment targets associated with the clusters [groups of standards]. Claims 2, 3 and 4 assessment targets are derived from the math practices found on pages 6-8 on the CCSS for Mathematics. Think of the outer wheel, sitting unsupported, as the claim. What is a claim?-a broad assertion we make that must be supported with evidence. For example, we ‘claim’ that if the CCSS are taught effectively and with fidelity, students will be able to read closely and analytically to comprehend ….a range of text….this is Claim One [Reading] for ELA/Literacy. There are four claims total for Mathematics in order from one to four: Concepts and Procedures, Problem Solving, Communicating Reasoning, and Modeling and Data Analysis. Each claim is supported by a number of assessment targets. See the Content Specifications at the SBAC site for the complete matrix. But how do we support this broad ‘claim?’ By writing items to certain assessment targets. Targets are formed and informed by one or more standards or parts of standards or mathematical practices. This reflects the integrated nature of the CCSS-seeing how multiple skills can be assessed in concert as in the performance tasks. The Targets are like spokes supporting the outer wheel, providing evidence for the claim. In contrast with ISAT, the separate threads are woven together in a tapestry-this is a massive paradigm shift from ISAT’s isolation-by-objective design. And all the assessment targets for each claim are supported, as is the whole wheel, by the Hub, the Common Core State Standards and the Mathematical Practices. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Evidence Centered Design: Claim/target [Math] Claim 1: Concepts and Procedures: Students can explain and apply math concepts and carry out math procedures with precision and fluency Target 1: Represent and solve problems involving multiplication and division Target 1 informed by Standards 1,2,3, and 4 [grade 3] All items are written to the ASSESSMENT TARGETS-targets for math claim one are informed by clusters which encompass one or more standards or parts of standards, targets provide evidence to support the claim In this example, Claim 1, Math is supported by Target 1. This target provides some of the evidence for the broad claim about math. This particular target is informed by and anchored in four common core math standards. 3.0A.1:Interpret products of whole numbers 3.OA.2: Interpret whole number quotients 3.OA.3 Use multiplication and division within 100 to solve word problems 3.OA.4 Determine unknown whole number in a multi. or division equation relating three whole numbers RECAP: The strength of wheel is derived from the hub, transferred to the outer wheel by the spokes. Evidence centered design supports the integrated nature of the common core. Examine any performance task and you will see that many standards or parts of standards are assessed via the assessment targets connected to the task. This integrated, holistic way of assessing mirrors the expectations of the CCSS and has important implications for instructional design. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Resources/PD Opportunities SDE Common Core Toolbox, deconstructed standards, Hunt Videos, sample units, PD list http://www.sde.idaho.gov/site/common/ Webinars on CCSS subjects and office hours-fall 2012 Continued Spring and Summer Seminars with Strategic Partners; Best Practices Institutes Teacher Newsletter http://www.sde.idaho.gov/site/edsource/ 21st Century UDL Master Teacher Lesson Plan project, sample units across content areas shared statewide on SchooNet The SDE is committed to providing resources and professional development on CCSS implementation both in face to face formats and in workshop-in –a-box modules that replicate this training. In addition, the CCSS Tool Box on the web and resources in SN will act as a repository for resources districts and teachers can access and use asynchronously to further understanding of the core on their own schedule. Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT

Scott Cook, Director of Content scook@sde.idaho.gov Contacts Scott Cook, Director of Content scook@sde.idaho.gov Diann Roberts, Language Arts and Reading Coordinator droberts@sde.idaho.gov Chris Avila, Mathematics Coordinator cavila@sde.idaho.gov Thumbnail Assessment Design Comparison: Smarter-Balanced vs. Current ISAT