California Assessment of Student Performance and Progress (CAASPP) Introduction to the 2015 CAASPP Online Reporting System (ORS) 1 – 2 p.m. May 4, 2015

Purpose To provide an introduction to the Online Reporting System (ORS), which is the reporting system local educational agencies (LEA) will use to view preliminary student results on the following CAASPP assessments: Smarter Balanced Summative Assessments in English language arts/literacy (ELA) and mathematics California Standards Test (CST) for Science California Modified Assessment (CMA) for Science California Alternate Performance Assessment (CAPA) for Science Standards-based Tests for Spanish (STS)

Agenda Overview Accessing the ORS Understanding the ORS Interface Viewing Score Reports in the ORS Live Demonstration Questions Updates and Announcements

Overview

What is the Online Reporting System? The ORS is a web-based system that displays preliminary score reports and participation data for each student who takes part in the CAASPP assessments.

Why Online Reporting? Timely data is more useful for affecting learning and instruction. Move to computer-based assessments facilitates faster reporting. Policymakers made timeliness a priority (Education Code 60641(b)).

Important Reminder The data available in the CAASPP ORS represents partial and preliminary results that are not appropriate for public release. As a real-time system, results will change as additional data is received and relevant appeals and rescores are processed. These changes may result in final scores being higher or lower than the preliminary results posted to this system. The California Department of Education (CDE) recommends that data from the ORS only be released publically following the state level release of assessment data that occurs in August.

Process for Release of Preliminary Test Results Hand score Machine score AI Score Score Confirm accuracy by grade and content area Check Preliminary scores available in the ORS Report

Reporting Timeline for Online Smarter Balanced Summative Tests Week 0: Student completes a content area Weeks 1-3: Student responses are scored and merged; preliminary results are checked Week 4: LEA accesses ORS to view preliminary results

Why Preliminary Results? Test administrations may still be in progress Factors that could change results Security breaches Problematic test questions Appeals processed and test rescores

Preliminary Test Results Provides information to support teachers in: Focusing instruction Improving learning Preliminary results include: Student and aggregate results Overall scores reported within one of four achievement levels Achievement Levels LEVEL 4 Exceeded the Standard 3 Met the Standard 2 Nearly Met the Standard 1 Has Not Met the Standard

Estimated Timeline for Release of Preliminary Results Grade and Content Area Estimated Date Smarter Balanced Grade 3 mathematics Smarter Balanced Grade 5 mathematics Smarter Balanced Grade 7 ELA May 3 (today) for students completed testing on or around April 1 Smarter Balanced Grade 6 ELA and mathematics Smarter Balanced Grade 7 mathematics Smarter Balanced Grade 8 ELA and mathematics TBD Smarter Balanced Grade 3 ELA Smarter Balanced Grade 4 ELA Smarter Balanced Grade 5 ELA Smarter Balanced Grade 4 mathematics Smarter Balanced Grade 11 ELA and mathematics CST/CMA/CAPA Science Grades 5, 8 and 10 STS RLA Grades 2-11

Reporting Timeline for Final Results Week 0: Student completes a content area Weeks 1-3: Student responses are scored and merged; preliminary results are checked Week 4: LEA accesses ORS to view preliminary results 4 Weeks After Test Administration Window Closes: LEAs accesses and downloads final student data file from TOMS

Examples of Recommended Uses Use preliminary overall scores and claim information to focus instruction Combine with other academic information about the student to identify learning opportunities

Not Recommended Adding these results to your local student information system Presenting these results as final scores Using these results as the sole basis for determining a student’s learning opportunities Comparing these results to last year’s STAR results

Accessing the ORS

User Roles and Access in the ORS ORS operates in a secure, role-based structure Access to reports and data in ORS depends on your user role in the Test Operations Management System (TOMS) Example: LEA-level users can view data for all schools and students in their LEA; school-level users can view data only for students in their school.

User Roles and Access in the ORS (cont.) Access Level LEA CAASPP Coordinator Test Site Coordinator (SC) Test Administrator (TA) School Listing    Student Listing Preliminary Student Results Report Summary Statistics Retrieve Student Results Plan and Manage Testing Test Completion Rates Search Students

Accessing the ORS Note: To log onto ORS, users must have an authorized TOMS username and password. Select the [Online Reporting System] button on the CAASPP portal at http://caaspp.org. Log onto ORS using your TOMS logon credentials.

Accessing the ORS Upon logging on, the Welcome page will appear.

Accessing the ORS Note: ORS contains students’ personally identifiable information (PII). For security purposes, it is important that you log out to ensure that the reports cannot be accessed by unauthorized users following your session. ORS has a timeout feature that automatically logs you out if you have been inactive for 30 minutes.

Understanding the ORS Interface

Understanding the ORS Interface Two primary features of ORS: Test Management Center (Completion Status System) Score Reports

ORS Welcome Page Takes you to the Completion Status System Takes you preliminary student test scores

Test Management Center Same as the Completion Status System Provides participation data for students taking tests Can be used to determine which students need to complete and/or start testing Can be used to view a summary statistics report and download student data files A detailed overview of accessing and using student participation reports is provided in the Completion Status User Guide

Score Reports Provide score data for each administered test Compare score data between individual students and the school, LEA, or overall state average scores Provide information about performance on claims

ORS Banner Completion Status Reports Preliminary student test results Opens a pop-up window where you can search for a student and access his/her preliminary scores Completion Status Reports Preliminary student test results Displays Retrieve Student Results page, where you can access student performance data files Links to the ORS User Guide (forthcoming) Exports the data displayed on the page in a .xls file

Viewing Score Reports in the ORS

Viewing ORS Reports From the Select drop-down list, select the LEA or school whose reports you want to view. A list will appear only if you are associated with more than one school or LEA Select [Score Reports]. Note: All score reports data, except for individual students’ score reports, can be disaggregated into subgroups for detailed analysis. For example, you can view a Grade 5 Mathematics report for a for an entire school or for a LEA.

Available Preliminary Score Reports in the ORS District Level School Level Student Level Home Page Dashboard Summary of performance (to date) across grades and subjects or courses for the current administration.    Subject Detail Data for a subject within a particular grade or course for the current administration. Claim Level Detail (Smarter Balanced) Claim/Block-level percentage at each performance level for a subject within a particular grade or course for the current administration. Student Listing Roster of all students who belong to a school with their associated subject or course (and claim, if applicable) scores for the current administration. Student Detail Detailed information about a selected student’s performance in a specified subject or course (and claim, if applicable).

Home Page Dashboard Report Displays the overall summary of score data and testing progress for your LEA and/or school Starting point for data analysis Define the students whose aggregated scores you want to view Navigate to more detailed score reports from the Home Page Dashboard Note: The score data you see are dependent on your role

Home Page Dashboard Report (cont.)

Subject Detail Report The aggregation tables on the Home Page Dashboard display score data for students by grade and subject and provide access to Subject Detail Reports.

Subject Detail Report (cont.)

Subject Detail Report (cont.) Each Subject Detail Report consists of the following components: Report descriptor Test name (subject and grade) Administration year Entity (e.g., LEA, school) The title of the score report table Name, number of students, average scale scores, percent in each achievement level All data are based on the total number of students who have taken and completed the test and submitted it for scoring

Claim Level Detail Report 2 1

Claim Level Detail Report (cont.)

Claim Level Detail Report (cont.) The Claim Level Detail Report consists of the following components: Report name [Entity] Performance for Each Claim. What are my [entity’s] strengths and weaknesses in [Subject or Course]? Test name (subject and grade) Administration year Entity (e.g., LEA, school, or roster) The title of the score report table Name, number of students, average scale score, claims, and percentage in each claims achievement level

Student Listing Report 2 1

Student Listing Report (cont.)

Student Listing Report (cont.) Students’ SSIDs are displayed Scores for individual students are not mean scale scores Different procedure for viewing scale score data by demographic subgroup Print on the Student Listing Report prints the current page and also generates a PDF file of individual preliminary student results for all the students in the roster

Student Detail Report 2 1

Student Detail Report (cont.)

Student Detail Report (cont.) Displays the breakdown of the student’s preliminary scale score, achievement level for the selected subject, and performance and claim description for each claim Includes average scale scores for the state and LEA for comparison purposes Note: State-level scale score averages will not be available until formally released by the CDE.

System Enhancements under Consideration Target reports Creating roster groups

Live Demonstration

Questions?

Submitting Feedback on the ORS New systems require patience, we appreciate your understanding and welcome your feedback: https://etsforms.formstack.com/forms/ors_feedback

Updates and Announcements

Updates and Announcements The Online Reporting System User Guide is forthcoming. Next Webcast: 2015 CAASPP Post-Test Workshop Date: Wednesday, May 20, 2015 Time: 1–3:30 p.m.

Resources and Support

Help Desk Support The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators! Monday–Friday from 7 a.m.–5 p.m. PT E-mail: caltac@ets.org Phone: 800-955-2954 Web site: http://caaspp.org