Presented by Educational Testing Service (ETS) and WestEd Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Session 1: Accessing Scores, Reports, Resources, and Tools Session 2: Analyzing Summative Assessment Results to Inform Teaching and Learning May–June 2017 Presented by Educational Testing Service (ETS) and WestEd

Before We Get Started . . . Name: Passcode: Internet Materials http:/tinyurl.com/lrpqlym Question Cards Housekeeping Facilities Parking Refreshments Evaluation http://tinyurl.com/mv2royk Name: Passcode: 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Agenda Agenda Item Introductions and Purpose Accessing Scores, Reports, Resources, and Tools Analyzing Summative Assessment Results to Inform Teaching and Learning Lunch (Noon−1 p.m.) 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

“California’s testing system makes improving instruction a priority.” The purpose of the in-person workshop is to inform local educational agencies (LEAs) of the various components of the 2016–17 California Assessment of Student Performance and Progress (CAASPP) student results and reports and to discuss how this information might be used to inform teaching and learning. “California’s testing system makes improving instruction a priority.” —Tom Torlakson, State Superintendent of Public Instruction 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Objectives Provide general information about the Smarter Balanced assessments in order to focus on the effective use of the 2016–17 summative assessment results Prepare LEA CAASPP coordinators to access the 2016–17 test results Provide supplementary resources and reference materials to assist LEA CAASPP coordinators with their local training efforts Provide an explanation of achievement levels, claims, and targets and how to interpret them 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Setting the Stage

A Balanced Assessment System 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

The Smarter Balanced Hierarchy of Item Development and Reporting of Scores Overall Claims Content Claims Targets Evidence Statements Items 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Relationships Between Common Core Standards, California Frameworks, and Assessments English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects   English Language Arts/Literacy Content Specifications (Claims) English Language Arts/English Language Development Framework English Language Arts/Literacy Item Specifications (Claims and targets per claim) English Language Arts/Literacy Assessment Blueprint English Language Arts/Literacy Summative Assessment   Mathematics Content Specifications (Claims) Mathematics Framework for California Public Schools Mathematics Item Specifications (Claims and targets per claim) Mathematics Mathematics Assessment Blueprint Mathematics Summative Assessment 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

These tests are connected to what is taught in the classroom These tests are connected to what is taught in the classroom. Talk to your elbow partner: consider how you currently use the summative results to inform teaching and learning. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

The Process of Scoring

Student Demographic Data Data Flow California Longitudinal Pupil Achievement Data System (CALPADS) Test Operations Management System (TOMS): Test Assignments and Settings Student Demographic Data Test Settings and Basic Student Data Test Delivery System Student Responses Online Reporting System (ORS) Scoring System Test Results TOMS (Reporting) 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CALPADS Reminders Confirm that your student demographic data are up-to-date in CALPADS and TOMS prior to the end of your LEA’s selected testing window. Students who will be returning to the same school the following year should not be re- enrolled in the next grade before the LEA’s selected testing window closes. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Objectives of This Section Provide an overview of computer adaptive testing (CAT) and scoring Describe the relative contribution of the performance tasks (PTs) and CAT to the overall scores Describe: Score scale Achievement levels Claims Assessment targets 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How Does a CAT Work? Example: A Student of Average Ability Expanded Very High Ability High Med–High Medium Med–Low Low Expanded Very Low Test Questions 1 2 3 4 5 6 7 8 9 10 Answers (right/wrong) R R W R W W W W R R 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT: Practical Considerations Each student’s test is constrained to ensure coverage of the full range of appropriate grade-level content (e.g., the ELA test cannot consist of only reading informational items). The level of test-question exposure is constrained to maintain test security. Sets of test questions based on a common passage or stimulus constrain the ability to adapt within the set. The test question pools for a particular grade level are designed to include an enhanced pool of test questions that are more or less difficult for that grade but still match the grade’s test blueprint. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAT Scoring As a student progresses through the test, his or her pattern of responses is tracked. Revised estimates of the student’s ability are calculated. Successive test questions are selected to increase the precision of the student’s level-of-achievement estimate, given the current understanding of his or her ability. The adaptive test will select these items based on the student’s performance on prior items. Scores from the CAT portion of the test are based on the specific test questions selected as a result of the student’s responses. Scores are NOT based on the sum of the number of questions answered correctly. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Human-Scored Items in the CAT Some items administered on the Smarter Balanced CAT component require human scoring. Since these items cannot be scored in real time by a human, the student’s performance on these items will not impact subsequent item selection. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Performance Tasks (PTs) In all Smarter Balanced tests, a PT and a set of stimuli on a given topic are administered as well as the CAT. PTs are not adaptive and, therefore, are not targeted to a student’s specific ability level. The items associated with the PTs may be scored by a machine or by human raters. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Score For each student, the responses from the PT and CAT portions are merged for final scoring. The resulting estimates of achievement are based on the specific test questions that a student answered and not the total number of items answered correctly. Higher estimates of achievement are associated with test takers who correctly answer difficult and more discriminating items. Lower estimates of achievement are associated with test takers who correctly answer easier and less discriminating items. It is possible for students to have the same estimate of achievement through different response patterns. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Scoring: Contribution of the CAT and PT Sections Number of Items Defined by Test Blueprints Grade ELA Mathematics CAT PT 3–5 38–41 4–6 31–34 2–6 6–8 37–42 30–34 11 39–41 33–36 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Final Scoring: Contribution of the CAT and PT Sections Relative contribution of the PT and CAT sections on the total score will vary across students since the overall score Is based on pattern scoring Depends on performance on the CAT section Depends on performance on the PT section Depends on the average difficulty of the PT section Ex: A PT might have a significant negative contribution on a students overall score if they did extremely well on the CAT but performed poorly on a very easy PT section Upshot: All parts of the Smarter Balanced Assessments are important 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Available Summative Assessment Scores Scale scores Achievement levels Not met, nearly met, met, exceeded Claim levels Below, near, above Target scores Worse, similar, better Insufficient information 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Properties of the Reporting Scale Scores are on a vertical scale. Expressed on a single continuum for a content area Allows users to describe student growth over time across grade levels Scale score ranges ELA: 2114–2795 Mathematics: 2189–2862 For each grade level and content area, there is a separate scale score range. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Smarter Balanced Scale Score Ranges by Grade Level and Content Area Subject Min Max 3 ELA 2114 2623 Mathematics 2189 2621 4 2131 2663 2204 2659 5 2201 2701 2219 2700 Grade Subject Min Max 6 ELA 2210 2724 Mathematics 2235 2748 7 2258 2745 2250 2778 8 2288 2769 2265 2802 11 2299 2795 2280 2862 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Copyright © 2009 Educational Testing Service.

Smarter Balanced Scale Score Ranges and Achievement Levels for ELA Grade Level 1 Standard Not Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2114–2366 2367–2431 2432–2489 2490–2623 4 2131–2415 2416–2472 2473–2532 2533–2663 5 2201–2441 2442–2501 2502–2581 2582–2701 6 2210–2456 2457–2530 2531–2617 2618–2724 7 2258–2478 2479–2551 2552–2648 2649–2745 8 2288–2486 2487–2566 2567–2667 2668–2769 11 2299–2492 2493–2582 2583–2681 2682–2795 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Threshold Scale Scores by Grade 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Level 2 Standard Nearly Met Level 4 Standard Exceeded Smarter Balanced Scale Score Ranges and Achievement Levels for Mathematics Grade Level 1 Standard Not Met Level 2 Standard Nearly Met Level 3 Standard Met Level 4 Standard Exceeded 3 2189–2380 2381–2435 2436–2500 2501–2621 4 2204–2410 2411–2484 2485–2548 2549–2659 5 2219–2454 2455–2527 2528–2578 2579–2700 6 2235–2472 2473–2551 2552–2609 2610–2748 7 2250–2483 2484–2566 2567–2634 2635–2778 8 2265–2503 2504–2585 2586–2652 2653–2802 11 2280–2542 2543–2627 2628–2717 2718–2862 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Threshold Scale Scores by Grade 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Claim Areas English Language Arts/Literacy Reading Writing Listening Research/Inquiry Mathematics Concepts and Procedures Problem Solving/Modeling and Data Analysis Communicating Reasoning 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Claim Scores Reported at an individual and aggregate level Reported for all claims in ELA and mathematics 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Achievement Levels for Claims Achievement levels for claims are very similar to subscores. They provide supplemental information regarding a student’s strengths or weaknesses. A student must complete all items within a claim to receive an estimate of his or her performance on a claim. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Copyright © 2009 Educational Testing Service.

Why are there only three achievement levels for Claims? Three achievement levels for claims were developed. Achievement levels for claims are based on the distance a student’s performance on the claim is from the Level 3 “standard met” criterion. 2016–17 Post-Test Workshops: Connecting Assessments to Instruction

Assessment Targets Assessment targets connect the content standards to evidence that will be collected from the assessment. Targets map the CCSS onto assessment evidence that is required to support the content categories and claims. Targets are used to guide the development of items and tasks that will measure the CCSS. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Assessment Target Reports Assessment targets are group-level indicators (e.g., for a classroom). Target reports in the ORS indicate group- level strengths or areas for improvement for specific standards that are addressed in each claim in the ELA and Mathematics Concepts and Procedures claim. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning Copyright © 2009 Educational Testing Service.

Assessment Target Reports Example: Mathematics Target Report Assessment Target Report Quick Start Guide available at http://www.caaspp.org/administration/reporting/index.html 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding the Reports

Available Reports  Report LEA School Parent/ Guardian Initial Student Test Results  ORS Partial Aggregate Data Partial Assessment Target Reports Student Score Reports PDF/Paper TOMS†/Paper†† TOMS†/Paper Paper Final Student Data File TOMS † PDFs of the Student Score Reports will be available in TOMS. †† LEAs must provide the copy of the CAASPP Student Score Report to each student’s parent/guardian within 20 working days of its delivery to the LEA. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Example 2016–17 Timeline 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Using the Online Reporting System (ORS)

Partial Results Available Now in the ORS Test results are available three weeks after a student completes both parts—CAT and PT—of a content area. Test results are added nightly. Use Caution: The results available in this reporting system are partial and may not be a good representation of your school’s or LEA’s final aggregate results. These partial results are not appropriate for public release. Final data will be released publicly by the California Department of Education (CDE) in September 2017. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

ORS Summary The ORS is a Web-based system that displays score reports for each student who has taken the following California assessments: Smarter Balanced for ELA and mathematics California Alternate Assessments (CAAs) for ELA and mathematics Standards-based Tests in Spanish (STS) for Reading/Language Arts (RLA) Note: All score report data, except for an individual student’s score reports, can be disaggregated into subgroups for detailed analysis. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Features and Reports in the ORS Home Page Dashboard Subject Detail Report Claim-Level Detail Report Assessment Target Report Student Listing Report Student Detail Report Manage Rosters The ORS Score Reports Quick Start Guide is available at: http://www.caaspp.org/rsc/pdfs/CAASPP.ORS-quick-start-guide.2016.pdf 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Users with Access to the ORS User Role ORS Access Level LEA CAASPP coordinators All students within the LEA CAASPP test site coordinators All students within their school or entity Test administrators (TA) Test examiners (TE) Test administrators/test examiners Students assigned to them via a roster assignment by their CAASPP test site coordinator Educators/Teachers For additional information about user roles available within TOMS, refer to the 2016–17 User Role Guidance document on the CAASPP Portal. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Test Results in the ORS CAASPP Portal: http://www.caaspp.org/ ORS User Guide: http://www.caaspp.org/rsc/pdfs/CAASPP.ORS-guide.2016.pdf 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Test Results in the ORS 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Home Page Dashboard 1 2 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Home Page Dashboard 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Home Page Dashboard: Select Test, Administration, and Enrollment Status 1 Select Administration 2014–15 data are final 2015–16 data are final 2016–17 are partial Enrollment Status View 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Home Page Dashboard: Select Grade and Subject 2 Test results are available for students who have completed both parts—CAT and PT. Students are in the process of completing tests—results are not yet available. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Demographic Subgroups Show/Hide Comparison Data Subject Detail Report Report Name Legend Demographic Subgroups Show/Hide Comparison Data Show/Hide Columns Exploration Menu Time Stamp 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Available Demographic Subgroup Same subgroups as reported to the CDE CAASPP public Web reporting site Based on data received from the California Longitudinal Pupil Achievement Data System (CALPADS) at the end of the selected testing window 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Listing Report Sort 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Detail Report Report Name Legend for Claim Achievement Category Summary Additional details of the student’s performance Average LEA and school scale scores for comparison . Performance on Claims 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Exploration Menu 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Managing Rosters

Manage Rosters Create customized groupings of students within a school Example: School-level users can create a report that lists all of the students within a specific grade or in a particular classroom. Use in combination with user roles (e.g., TA, TE, TA/TE, educators) to provide teachers with access to test results for their students 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Manage Rosters: Print, Modify, and Delete 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

15-Minute Break

Using Reports in TOMS

Features and Reports in TOMS Student Score Reports (PDFs) LEA User List Report Student Demographics Reports LEA Coordinator Report Test Assignment Reports Student Results Report Unlisted Resources Report Percentage Report 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Test Results in TOMS CAASPP Portal: http://www.caaspp.org/ 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

TOMS Dashboard TOMS Downloading Student Score Report PDF Files Quick Start Guide available at: http://www.caaspp.org/administration/reporting/index.html 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Download Student Score Report PDFs Select year Select the school and grade of the reports you want to compile and name the report download file. To select contiguous grades (e.g., grades three–six), hold down the [Shift] key as you select each grade. To select specific noncontiguous grades (e.g., grade three, grade four, grade six), hold down the [CRTL] key as you select each grade. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Score Report PDF Download File LEA CDS Code School CDS Code Grade SSID Date stamp A compressed file (.zip) will be generated. Large files may take time to process. Each PDF contains the Student Score Report for one student. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Student Score Report PDF Download File School Name School Code Grade Last Name First Name Middle Initial State Student Id Subject Code PDF file Demo School 99999989999998 3 LAST01 FIRST01 A 8888888991 ELA,MA ETSCAASPP_2_99999990000000_99999989999998_03_8888888991_20160504.PDF LAST02 FIRST02 B 8888888992 ETSCAASPP_2_99999990000000_99999989999998_03_8888888992_20160504.PDF LAST03 FIRST03 C 8888888993 ETSCAASPP_2_99999990000000_99999989999998_03_8888888993_20160504.PDF LAST04 FIRST04 D 8888888994 ETSCAASPP_2_99999990000000_99999989999998_03_8888888994_20160504.PDF LAST05 FIRST05 E 8888888995 ETSCAASPP_2_99999990000000_99999989999998_03_8888888995_20160504.PDF LAST06 FIRST06 F 8888888996 ETSCAASPP_2_99999990000000_99999989999998_03_8888888996_20160504.PDF Hyperlinks to the individual Student Score Reports 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report

Understanding Your Student Score Report Student Score Report Versions Assessment Grades Smarter Balanced for ELA and mathematics 3, 4, 6, 7, and 11 Smarter Balanced for ELA, mathematics, and science (California Science Test [CAST] pilot) 5, 8, and 11 CAAs for ELA and mathematics CAAs for ELA, mathematics, and science (CAA for Science pilot) CAST or CAA for science (CAST pilot and CAA for Science pilot) 10 and 12 STS for RLA 2–11 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report Student Score Report Types Initial–electronic Final–electronic and printed Additional Student Score Reports The final 10 percent after the original 90 percent are shipped Electronic and printed One report for each LEA Multiple administrations combined as well as all CAA student scores remaining Any re-reported students 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report 1 5 6 2 3 7 8 4 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report Elements of the Student Score Report 1 Overall score and progress for each content area Additional information about the student’s overall score 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Elements of the Student Score Report—Front Page Understanding Your Student Score Report and Grade Eleven Early Assessment Program (EAP) Scores Elements of the Student Score Report—Front Page 1 EAP College Readiness 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Test Results from Multiple Years Elements of the Student Score Report—Front Page 2 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report: Claim Score Information Elements of the Student Score Report—Front Page 3 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report: Claim Score Information Elements of the Student Score Report—Front Page 3 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report: Link to Test Score Guide http://www.testscoreguide.org/ca/ An online resource to help parents/guardians better understand their child’s CAASPP Student Score Report 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report: Student Address Elements of the Student Score Report—Back Page 5 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report: Information About CAASPP Elements of the Student Score Report—Back Page 7 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Other Notable Reports for 2016–17

Understanding Your CAST Student Score Report for High School 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Elements of the Student Score Report—Back Page Understanding Your Student Score Report: California Science Test (CAST) Elements of the Student Score Report—Back Page 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Spanish Student Score Report 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Requesting a Spanish Student Score Report Request through TOMS Select the [Score Status] tab on the Student Profile page Select the checkbox for “Request for Spanish Student Score Report” TOMS Spanish-Language Student Score Report Quick Start Guide available at: http://www.caaspp.org/administration/reporting/index.html 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your Student Score Report Guide Versions Available Electronic versions will be posted by the CDE at http://www.cde.ca.gov/ta/tg/ca/c aasppssreports.asp LEAs may print locally and provide to parents/guardians Language versions: English Spanish Armenian Chinese (simplified) Chinese (traditional) Hmong Korean Pilipino (Tagalog) Vietnamese 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Timeline for Reporting

When Will the Initial Student Score Reports for CAASPP Appear in TOMS? Initial Student Score Reports will be provided as downloadable PDF files in TOMS one week after the student’s last score is reported in the ORS. Initial Student Score Reports are created during the test administration window as the student completes the last assigned test. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Avoiding Scoring Delays for the Smarter Balanced Assessments ETS may place a temporary hold on scoring a student’s results to investigate and/or clarify information provided. Situations that would lead to a hold include, but are not limited to: A Statewide Student Identifier (SSID) has test results for multiple students A student answer document was returned incomplete The student record was not found in TOMS If a temporary hold is placed on a student’s score, it will delay the reporting of results in the ORS as well as the Student Score Reports and data files. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

LEA Dashboard in TOMS 1 2 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Information on the LEA Dashboard 1 Progress of scoring Estimate of when to expect delivery of paper Student Score Reports Time stamp of last update 2 Alert at 80 percent—review and update student demographic data in CALPADS Alert at 90 percent—final Student Score Reports will be processed and shipped 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scoring Progress Dashboard TOMS includes a dashboard displaying the progress of scoring for your LEA’s administration. When 80 percent of student scores are complete for the LEA’s administration, the LEA CAASPP coordinator will receive a notification to carry out the following: Complete testing for students with processed appeals. Verify condition codes and enter any remaining condition codes. Request Spanish Student Score Reports. Select parent/guardian address (if desired). When 90 percent of student scores are complete, the final Student Score Reports will be printed, packaged, and delivered to the LEA. At the end of scoring, all remaining final Student Score Reports will be delivered in a separate shipment. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Future Score Report Resources

Score Report Resources Student Score Report Resources Estimated Availability Location Quick Start Guides Now http://www.caaspp.org/administration/reporting/index.html LEA Student Data File and supporting materials Summer 2017 TOMS, CAASPP Portal, Webcast* CAASPP Post-Test Guide CAASPP Portal, Webcast* Public Web reporting September 2017 CDE Web site, Webcast** * Information about the LEA student data file and the Post-Test Guide will be provided during the same Webcast. ** Information about these resources will be provided in separate Webcasts to be timed to coincide with the release of the resource. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAASPP Portal: http://www.caaspp.org/ 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Introducing Results for the CAAs for ELA and Mathematics

Scores Reported The scores reported include: Scale scores for ELA and mathematics Performance levels for ELA and mathematics No subscores or claim scores are reported. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Scale Scores “Scaling” is the statistical process that transforms scores from one set of numbers to another that allows scores to be more interpretable as well as comparable in some way. A scaled score DOES NOT mean “percent correct.” A scaled score DOES NOT mean “raw score.” The reporting scale for the CAAs for ELA and mathematics is a three-digit scale, where the leading digit corresponds to the grade level and the last two digits range from 00 to 99. Examples: Scale for grade five ELA ranges from 500 to 599 Scale for grade eleven mathematics ranges from 900 to 999 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Performance Levels Scaled scores are mapped into three performance levels Level 1 Alternate—Limited Understanding Level 2 Alternate—Foundational Understanding Level 3 Alternate—Understanding 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Performance Level Descriptors 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAAs for ELA and Mathematics: Scale Score Ranges by Grade Scaled score ranges are the same for both the ELA and mathematics CAA assessments. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding How Scale Scores Are Assigned Students who did not respond or orient to the test will receive a Lowest Obtainable Scaled Score (LOSS) (e.g., 300, 400, 500, and so on). Students who responded to fewer than four questions will receive a “LOSS + 1” (e.g., 301, 401, . . . , 901). Students responding to four or more questions could receive any score above “LOSS + 1” (e.g., 302–399; 402–499;……; 902–999). 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How to Use CAA Results Scaled scores for the CAA can be compared within grade levels and from one year to another. For example: Mean scaled scores for grade five ELA in School A may be compared with grade five ELA in School B Mean scaled scores for grade seven mathematics in LEA A may be compared with grade seven mathematics in California Mean scaled scores for grade three ELA in LEA A in 2017 may be compared with grade three ELA in LEA A in 2016 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How to Use CAA Results Scaled scores for the CAA should not be compared across different content areas and grades. There is no vertical scale for CAA scores. Therefore, you should not compute differences in scaled scores from one year to the next for the same student or group of students that move from one grade to the next. Scaled scores for the CAA should not be compared to different assessments such as: Smarter Balanced assessments California Alternate Performance Assessment (CAPA) 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

How to Use CAA Results Scaled scores for the CAA should not be compared across different content areas and grades. There is no vertical scale for CAA scores. Therefore, you should not compute differences in scaled scores from one year to the next for the same student or group of students that move from one grade to the next. Scaled scores for the CAA should not be compared to different assessments such as: Smarter Balanced assessments California Alternate Performance Assessment (CAPA) 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

CAA Scoring and Reporting Scores will be available in the ORS by mid-July. Student Score Reports for the CAAs will be distributed to LEAs in August. Data files for the CAAs will be available in September. Next year, the CAAs can move from post-equating to pre-equating and, therefore, can report results on the same timeline as the Smarter Balanced assessments. 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your CAA Student Score Report 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Understanding Your CAA for Science Student Score Report for High School 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Help Desk Support The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators! Monday–Friday from 7 a.m.–5 p.m. PT E-mail: caltac@ets.org Phone: 800-955-2954 Web site: http://www.caaspp.org/ 2016–17 CAASPP Post-Test Workshops: Using Assessment Results to Inform Teaching and Learning

Questions and Answers