San Diego County Office of Education Smarter Balanced Digital Library and Interim Assessments:  Resources to Support Implementation San Diego County Office of Education August 10, 2015 Mary Tribbey Senior Assessment Fellow

Topics Interim Assessments Digital Library Considerations for Implementation

Assessment Cycles by Purpose Figure 8.4. Assessment Cycles by Purpose Take a moment to browse this graphic illustrating the assessment cycles by purpose. Notice the range of assessments operating in different cycles: short, medium, and long, providing different levels of details about student learning over time to be used for various decision-making purposes. All assessment cycles should provide a continuous picture of student learning, yielding data about the extent to which students have met or are on the way to meeting learning goals. Remind participants that the three components of the Smarter Balanced Assessment System provide tools and resources for all three assessment cycles. Interim assessments occupy a middle position between short-cycle formative assessment and long-cycle summative assessment. They may be used “for” instruction to inform next steps, and/or may also be used “of” learning for evaluative purposes. Formative Assessment Interim/Benchmark Large-scale Summative Adapted from Herman and Heritage (2007) California Department of Education (2014) English Language Arts/English Language Development Framework for California Public Schools: Kindergarten Through Grade Twelve © 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Formative Assessments | 07.31.15

Summative assessments benchmarked to college and career readiness (Grades 3-8 and 11) Teachers and schools have information and tools to improve teaching and learning Standards set expectations on path to college and career readiness All students graduate college and career ready Digital Library: Formative assessment tools and practices for teachers to improve instruction Interim assessments Flexible, open, used for actionable feedback

Interim Assessments Support Teaching and Learning Elicit evidence of student learning to inform teaching and learning. Engage in professional learning. Deepen teacher content knowledge. Evaluate grade level instructional progress and plan adjustments to increase student learning. Observe student use of the testing platform and accessibility supports with actual test items. © 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Interim Assessments

Interim Assessments Designed as resources to help teachers determine how students are progressing throughout the school year. Interim assessments are provided free for local educational agency (LEA) use. Available for Grades 3 through 8 and high school; ELA and mathematics Uses are determined by the LEA. Reminder − practice and training tests remain available and there is no limit on their use.

Interim Assessment Types Interim comprehensive assessments (ICAs) Assess the same standards and claims as the summative assessments Can administer up to three times per year Interim assessment blocks (IABs) Assess smaller sets of targets or specific areas Allows flexibility by selecting blocks aligned with curriculum and instruction

Administration of ICAs and IABs Administered online (no paper/pencil administration) Same teacher registration process and test delivery interface as the summative assessment Fixed length test format in 2015-16 Both of the tests are administered on line. Teachers will register one time and use the same login for the summative assessment and the Interim Assessments. Test administrators and students will experience the same interface as they will during the summative assessment. When the tests are administered is up to the LEA, but during this initial rollout phase, it’s important to remember that the item pool from which the IABs and ICAs draw is limited. That means if a district uses both the IABs and the ICAs, it is likely that students will see the same items in both tests and their results might be influenced by having seen the same item multiple times. Districts need to think this through when making agreements with teachers about the purpose and use of the tests locally.

Interim Comprehensive Assessments (ICAs) http://www.cde.ca.gov/ta/tg/sa/sbacinterimassess.asp

Interim Assessment Blocks (IABs) http://www.cde.ca.gov/ta/tg/sa/sbacinterimassess.asp

Access to the Interim Assessments Access to the Interim assessments is restricted to test administrators (TAs), as defined in the California Code of Regulations. Interim assessments are not secure − TAs may view tests and use scoring materials for professional development purposes. Interim assessments are not public − Only viewable or shareable with anyone who is identified as a user in the Test Operations Management System (TOMS).

TOMS Access to TOMS is provided to site coordinators and test administrators (TAs) by the LEA CAASPP coordinator. Access to TOMS requires a current-year CAASPP Test Security Agreement or Affidavit that is signed and on file with the LEA http://caaspp.org./administration/forms/index.html

Student Demographics and Settings All students, grades K through 12, are eligible to take interim assessments Demographic information (e.g., grade, SSID, program participation) in TOMS needs to be reviewed and updated, if needed, for all students. Online test settings need to be configured and reviewed for all students in TOMS to ensure students receive the appropriate accessibility supports. See the Administration of the Online Interim Tests Resource Guide for additional information. http://www.caaspp.org/rsc/pdfs/CAASPP.interim-tests-admin.2015.pdf

Accessibility Supports Who does the work of identifying and entering test settings? Do principals know that schools should be paying attention to these test settings? Teachers can verify whether students are benefitting from the appropriate supports.

Preview the Tests TAs can access and view the interim assessments for professional development and/or training purposes. http://caaspp.org/ta-resources/interim.html

Preview the Tests f

Preview the Tests f

Steps to Preview the Interim Assessments http://www.caaspp.org/ Hover over TA Resources tab Select Interim Assessments Select Interim Assessments Viewing System Select appropriate grade level Choose from available tests Choose form Answer questions to move forward

Questions?

Interim Assessment Scoring Most interim assessment items are scored by the Smarter Balanced test delivery engine. Hand scoring of constructed-response items and performance tasks is a local responsibility. Results for individual students are generated only after the constructed-response item scores and performance task scores are input into the Interim Assessment Hand Scoring System (IAHSS). (Processing time of up to 48 hours.) Scoring is supported by training guides, rubrics, exemplars (i.e., anchor and practice papers).

Training Guides and Exemplars Training Guides and Exemplars for all constructed response items and performance tasks are accessible in TOMS. LEAs can access these materials for professional development and training purposes.

Training Guides and Exemplars Also, archived information and training webinars

Preview the Interim Scoring Materials http://www.caaspp.org/ Select TOMS button (Log in to TOMS) Select the Help button Select the Training tab Select appropriate grade level Choose from available items

Interim Assessment Hand Scoring System

Interim Assessment Hand Scoring Home Page Once you log in to the interim assessment hand scoring system, the home page or the page you land on is called the Response List and it looks like this. Your Response List can be many pages long depending on how many tests have been administered and how many student responses are in the system that require local scoring.

IABs That Do Not Require Local Hand Scoring ELA, all grades Edit/Revise Listen/Interpret Research Mathematics, all grades All blocks, except Performance Task blocks

Questions?

Interim Assessment Reporting System LEA CAASPP coordinators provide users access to the Interim Assessment Reporting System to view student data. Users then have access to student Personally Identifiable Information (PII) and results for all students at a site or LEA.

Interim Assessment Reports of Results Reports for lists of students and individual student reports. No aggregate reports. ICA: Overall scale score (i.e., 2000 to 3000); Achievement Level (Level 1, 2, 3, or 4); and Claim level (Below, At/Near, or Above Standard) IAB: Level of Below, At/Near, or Above Standard for each Block Exportable data file

Please Remember! Scores and achievement levels from the interim assessments cannot and should not be compared with data from STAR Program or CAHSEE. Different standards are assessed. Different assessment methods and score scales are used. Different levels of cognitive rigor are assessed.

Interpreting Interim Assessment Results The extent to which interim assessments were administered under standardized conditions will affect the interpretations. Interim assessment results should only be used in combination with other evidence of student learning; they should not be the sole basis for making any high-stakes decisions.

Interim Assessment Reporting System Requires same logon credentials used for the Digital Library: User Name: LEA e-mail Password: ****** (6 characters) Data Warehouse Receive and view student reports LEAs need Digital Library logins to see the reports.

Reporting Home Page List of Districts Search Once you log in to the Interim Assessment Reporting System, this is the landing or home page. All the LEAs is California are listed on this page. You can use the search box, here to the upper right, to more quickly find your LEA instead of scrolling through the list of districts, which is displayed on the left. However, one can only access data for their own LEA. Notice everything is grayed out and the words “Interim Data Only” or “No Data Available” are visible here. It is only when you go to your own LEA, and select the name of your LEA that you will navigate to a screen where you can see your results.

ICA − List of Students Note the blacked out boxes are the names of students…

ICA − Student Report

IAB − List of Students

IAB − Student Report On this screen, we have a sample of the Individual Student Report for the ELA/literacy block Edit/Revise. In this particular example, the student took this block once. If they took it a second time, you would see the most recent results listed first and then these particular results would move down to the Previous Results part of the report. Remember that the results for the blocks are reported in terms of the three classifications, above standard, at or near standard, or below standard. In this example, for Edit/Revise, this grade four student is above standard for this block. If the student were administered other blocks, they would similarly show up on this report. Each blocks shows up in its own little square of this report.

Interim Assessment Blueprints Interim Comprehensive Assessments (ICAs) Because the ICAs are built to the same blueprints as the summative assessments, use the summative blueprints found at: http://www.cde.ca.gov/ta/tg/sa/sbacsummative.asp. Interim Assessment Blocks (IABs) IAB blueprints found under Additional Resources at: http://www.cde.ca.gov/ta/tg/sa/sbacinterimassess.asp . Connecting the interims to the standards…

ELA IAB Fixed Blueprint

Cognitive Rigor Matrix This matrix from the Smarter Balanced Content Specifications for ELA draws from both Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below. The Cognitive Rigor matrix combines two common taxonomies that categorize levels of cognition and shows how the Smarter Balanced depth of knowledge categories relate to these taxonomies. As shown here, the concept of remembering information only relates to the first depth of knowledge level. In contrast, evaluation does not relate to either of the first two levels of depth of knowledge. This table, which can be found in the Smarter Balanced Content Specifications for ELA, is a useful aid for guiding the development of items at different depth of knowledge levels. Now let’s examine the item specifications.

ELA Claim 1 Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Targets 1–7 correspond with literary texts Targets 8–14 correspond with informational texts The assessment targets incorporate the content clusters from the Common Core State Standards From the content specifications: For Claim 1, there are fourteen assessment targets. {+} Assessment targets 1-7 focus on reading literary texts. Assessment targets 8-14 focus on reading informational texts. Assessment targets are linked to the content clusters within the Common Core State Standards. Now let’s look at a specific target for Claim 1.

ELA IAB Fixed Blueprint Here are the seven assessment targets that the content specifications referred to. Let’s focus on Reasoning and Evidence.

Grade 5 ELA Claim 1 Target 4 4. REASONING & EVIDENCE: Use supporting evidence to justify interpretations (theme, events, conflicts/challenges, setting, character development/ interactions, point of view) Standards: RL-2, RL-3, RL-6 (DOK 3) RL-2 Determine a theme of a story, drama, or poem from details in the text, including how characters in a story or drama respond to challenges or how the speaker in a poem reflects upon a topic; summarize the text. RL-3 Compare and contrast two or more characters, settings, or events in a story or drama, drawing on specific details in the text (e.g., how characters interact). RL-6 Describe how a narrator’s or speaker’s point of view influences how events are described. This grade 5 target focuses on reading literary text. {+} Notice that several standards are associated with this target. There is not a one-to-one correspondence between a given assessment target and a standard. Instead, assessment targets focus on skills and knowledge that cross a cluster of standards. Also note that the depth of knowledge level that must be applied to achieve this assessment target is also specified.

Questions? Ask: What are one or two things that jumped out at you? What one or two wonderings might you have? (Two minutes) Popcorn around the rooms and share before moving on…

Reflection Think about implementation: Who needs to learn about these tests? What are the implications for administrator and teacher training? What resources might your “champions” need for this work? What existing collaboration structures might you use? What obstacles might you face?

Assessment Cycles by Purpose Figure 8.4. Assessment Cycles by Purpose Formative Assessment Interim/Benchmark Large-scale Summative Adapted from Herman and Heritage (2007) California Department of Education (2014) English Language Arts/English Language Development Framework for California Public Schools: Kindergarten Through Grade Twelve © 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Formative Assessments | 07.31.15

Summative assessments benchmarked to college and career readiness (Grades 3-8 and 11) Teachers and schools have information and tools to improve teaching and learning Standards set expectations on path to college and career readiness All students graduate college and career ready Digital Library: Formative assessment tools and practices for teachers to improve instruction Interim assessments Flexible, open, used for actionable feedback

Digital Library On-line collection of resources: ?? COE date On-line collection of resources: Aligned with the intent of the Common Core State Standards Contributed by educators for educators Vetted by a national network of educators against a set of quality criteria Supports implementation of the formative assessment process Incorporates collaboration features Supports diverse groups of learners Senior Assessment Fellow, Mary Tribbey

What the Digital Library Is Not… Rocklin USD September 21, 2014 a bank of assessment items a learning management system a library for the general public a site to freely post resources First, let’s make clear what the Digital Library is not: It is not a bank of items; you cannot use the Digital Library to create a test, although you might find a test in the library that will meet your needs. It is not a learning management system. This is a library of resources that help teachers learn to use or improve the use of assessment as a tool to improve teaching and learning. It is not a library available to the general public. The Digital Library has been licensed by the state of California to support its K-12 teachers in the use of formative assessment and requires that each user have his or her own login and password. It is not a site to freely post resources. Before they are posted, each of the resources must meet strict criteria, and be vetted by the Statewide Network of Educators in order to be accepted. Senior Assessment Fellow, Mary Tribbey

Criteria for Resources Rocklin USD Criteria for Resources September 21, 2014 Aligns with Common Core State Standards Demonstrates high-quality instruction Addresses learner differences Is engaging/user-friendly Incorporates formative assessment practices Each of the resources accepted into the Digital Library have met these, and other, criteria. They must align with the Common Core state standards. They must demonstrate high quality instruction, address learner differences, and be engaging for teachers and students. And, because this is the whole point of the Digital Library, each reasourc MUST incorporate formative assessment practices. So what do we mean by formative assessment and formative assessment practices? Senior Assessment Fellow, Mary Tribbey

Definition of the Formative Assessment Process ?? COE date Definition of the Formative Assessment Process Formative assessment is a deliberate process used by teachers and students during instruction that provides actionable feedback that is used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals. This is the definition that underpins the resources found in the Digital Library. (Note: Read the definition and stress the words in bold.) As we’ve stated, each of the resources offered in the Digital Library are intended to support the use of assessment to guide teaching and learning. This process is also called “assessment FOR learning,” as distinguished from “assessment OF learning.” Senior Assessment Fellow, Mary Tribbey

Four Attributes of the Formative Assessment Process ?? COE date Four Attributes of the Formative Assessment Process Clarify intended learning Elicit evidence Act on Interpret All of resources found in the Digital Library must have or support at least one of the attributes of the formative assessment process, shown here. Clarify Intended Learning: This attribute covers learning goals and success criteria. An example of a resource with this attribute would be one that helped teachers or students to create student friendly language for the skills the students are expected to learn and demonstrate. Eliciting Evidence: This attribute covers the process by which the teacher designs what the students will be doing in order to surface the students’ knowledge and skills. Crafting a question for an exit ticket that will inform the teacher about individual student learning would be an example of this attribute. Interpreting Evidence: This attribute covers the process by which the teacher reflects on each student’s learning and progress toward the intended learning and identifies any gaps Acting on Evidence: This attribute covers the next steps the teacher will take in light of what the students have and have not learned. At this point, the learning goal, or intended learning, might be differentiated for students and the process begins again. Senior Assessment Fellow, Mary Tribbey

The Process of Formative Assessment Feedback lope is pivotal to the process Also pivotal is knowing the crucial steps of the learning progression (points in this graphic where you cannot proceed to the next step without giving feedback, to close the gap, and give differentiated instruction.

How It Works: Filtering, Searching, Favorites and Collaboration Basically a gigantic search engine.

Assessment Literacy Modules Assessment Literacy Module: Understanding the Formative Assessment Process Experts in the field Examples of each attribute Resources Want you to see how to make effective use of the interims as part of a larger system; need to know what that larger systems; want classroom practice to change – want to makes sure teachers are aware of these other pieces and that the interims and summative validate and verify what they are doing in the classroom with kids. Show #8, which has the researchers quotes If time, Show #10, and stop after each attribute to let the audience talk

Digital Library Resources Cognitively Based Assessment of Learning (CBAL) Using a Learning Progression to Formatively Assess the Concept of Slope  The Using a Learning Progression to Formatively Assess the Concept of Slope webinar is intended for teachers of first-year algebra. This webinar provides an example of a CBAL formative digital assessment task in the Linear Functions and Nonlinearity module that incorporates a learning progression to develop students’ deep understanding of the concept of slope. Viewers hear how the learning progression, its application in the task, and ancillary resources support the teacher’s use of the task in the classroom.

Digital Library Connection Using results from either the summative or interim assessments, use the Digital Library to identify resources such as: Using Fluency Stations as Formative Assessment (RF 3.4 and RF 4.4) https://www.smarterbalancedlibrary.org/content/using- fluency-stations-formative-assessment

Other Examples Rocklin USD September 21, 2014 Informational Text Learning and Formative Assessment Strategies Accommodating ELLs: Thinking Strips to Document Reading Comprehension Aligning Formative Assessment with SBAC ELA Claim 1 Assessment Targets The Components of Effective Feedback Senior Assessment Fellow, Mary Tribbey

Questions?

Reflection Think about implementation: Who needs access to the Digital Library? What are the implications for administrator and teacher training? What existing collaboration structures might you use? How might this align with other professional learning? What obstacles might you face?

For Further Information California Department of Education CAASPP Office caaspp@cde.ca.gov 916-445-8765 http://www.cde.ca.gov/ta/tg/sa/sbacinterimassess.asp http://www.cde.ca.gov/ta/tg/sa/diglib.asp Smarter Balanced Web Page http://www.smarterbalanced.org/interim-assessments/ CalTAC http://caaspp.org/ 1-800-955-2954 (for Digital Library: 1-855-631-1510)

IAB Blueprints – Mathematics

California Content Standards

California Content Standards