1. Online Reporting System
Training Module
Welcome to the Online Reporting System (ORS) training module. It is designed to help you navigate the Online Reporting System.
This training module includes all features of ORS. However, you should consult your Online Reporting System User Guide located on the WCAP portal for additional information.
Copyright © 2014 American Institutes for Research. All rights reserved.

2. Objectives
After viewing this presentation, you should understand how to:
Navigate the system
View score reports
View participation reports from the Test Management Center
Search for specific students
Manage student rosters
After viewing this presentation, you should understand how to:
Navigate the system
View score reports
View participation reports from the Test Management Center
Search for specific students
Manage student rosters within ORS.

3. Purpose
Provide timely, relevant reports and guide educators to make valid, actionable interpretations of the data
Interactive data
Near real-time reporting (upon completion of scoring for hand-scored responses)
Provide access to data
Downloadable data files for districts, schools, and teachers
Provides results for assessments in one system
Provides participation data
The ORS is designed to provide you with assessment data and tools you need to understand the performance of your students. The system provides interactive data, which allows you to slice, dice, and explore the data to understand the results. The near real-time reporting will allow the speed in distributing the results.
In addition to score reports or performance reports, the ORS also includes participation reports that enable you to determine which students have completed or need to complete testing. Students who are not registered in the Test Information Distribution Engine (TIDE) to take a test are not included in the ORS.
Participation and proficiency rates found in ORS are not used for accountability purposes in Washington state. These figures are provided for local reference and test tracking only. Consult your Online Reporting System User Guide located on the WCAP portal for more detailed information.

4. Things to Remember
The ORS helps educators answer questions regarding the assessment data to improve teaching and learning.
The magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data.
All reports can be printed and exported.
The [Help] (User Guide) button is available on every page.
Printable reports for parents can be generated.
Here are a few things to remember:
the ORS helps educators answer questions regarding the assessment data to improve teaching and learning;
the magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data;
all reports can be printed and exported;
the [Help] (User Guide) button is available on every page; and
Printable reports for parents can be generated.

5. Logging In
The ORS is a secure, role-based system. Your access to reports and data in the system depends upon your user role and your district and school associations. You can access the system via the WCAP portal. To log in to the ORS, you must have an authorized username and password. If you do not have a username and password contact your School or District Test Coordinator as you will need to be added to the Test Information Distribution Engine (TIDE) before you can access the ORS.

6. ORS Interface: Welcome Page
The Welcome page appears when you first log in to the ORS and takes you directly to the type of report you want to view. You can navigate to a different report at any time from any ORS page.
Plan and Manage Testing provides participation data for students participating in an assessment. You can determine which students need to complete testing and who may need to take another opportunity to test, if applicable.
Score Reports provide score data for each of the tests. You can compare score data between individual students and the school or overall state average scores. Information on a specified field’s strengths and weaknesses is also available by reporting category.

7. ORS Interface: Tools
The ORS banner that is displayed on top of every page consists of buttons or tools for accessing the different reports and performing different tasks.
Score Reports takes you to the Score Reports Home Page.
Test Management Center provides access to participation reports and test summary statistics. You can also download large files of student results.
Inbox takes you to the Inbox section of the Retrieve Student Results page that lists the student data files that you have requested. You can also access the inbox by selecting [Retrieve Student Results] from the Test Management Center drop-down list.
Search Students lets you search for student information by using the student’s SSID number or name.
Manage Rosters allows you to create custom rosters of your students for data analysis.
In addition, the banner consists of the following “Global Tools.” All the tools may not be available on every page.
Help takes you to the Online Reporting System User Guide.
Print prints the data displayed on the page.
Export exports the data displayed on the page as an Excel file.
Definitions provides definitions for terms specific to the report you are viewing.

8. Score Report Navigation: Focus of Data
Who
What
When
Click to move to a higher level of a dimension
Click to drill down to details of a dimension
To help you quickly locate the score data you need, navigation in the ORS is designed to explore in any of the three dimensions inherent in the data. These three dimensions parallel the three kinds of questions that the data can help you answer: Who, What, and When.
This format allows you to view data that can be analyzed appropriately by focus. The interface design encourages users to think about the educational questions to which they need answers to and access information from that perspective.
The exploration menu, which appears when you click the magnifying glass icon on any score report, is the main method for moving between reports and deciding which type of data you would like to view. It is designed to help emphasize the hierarchy of the data while allowing you to navigate with a click of the mouse. Using the exploration menu, the magnifying glasses on the right allow you to drill down to details of a dimension, while the magnifying glasses on the left allow you to return to the previous dimension.
While in the ORS, it is recommended to use the exploration menu and other available tools to navigate within the system.

9. Levels of the Three Dimensions
State
District
School
Teacher
Class
Student
Subject
Claims
Targets
“Who” lets you see data for institutions and groups. This dimension includes data for the state, district, school, teacher, class, and student. If you are interested in viewing aggregate testing data or individual testing data for a teacher or student, you can explore data in this category.
“What” lets you explore the data by claims and targets within the selected subject/content area. This dimension includes data by subject, claims, and targets.
“When” explores data by time frame. It allows you to see the data either as a snapshot of performance in the current test window or as a historical trend (for the subject area you are viewing). You can filter data reports by the current test window or by trends in order to analyze changes in performance over time. The trend report will be available in 2016.
Certain data may not be available at all levels depending on the test taken.
Testing Window
Trend

10. Score Report: Home Page Dashboard
The Score Reports Home Page Dashboard appears when you click [Score Reports] on the Welcome Page or from the banner.
This page provides data about your students, the number of students tested, and the percentage proficient for the selected administration. District personnel see district summaries, schools see school summaries, and teachers see summaries of the students they proctored.
Select your test from the [Test] drop-down list. Click any table cell containing a value to begin exploring the data for a particular grade and subject.

11. Home Page Dashboard: Select Test and Administration
You can select the test and administration for which you want to view score data.
From the Home Page Dashboard page, you can select the test and administration for which you want to view score data.
This feature is particularly helpful for school personnel who want to see how students currently assigned to the user’s class roster performed in previous grades, even if students were enrolled in different schools during those previous administrations.
If you currently have a student who did not test in the selected test and administration, no data will be displayed for that student. Examples include students who moved to your school from out of state.

12. Score Data
Scores for students who were mine at the end of the selected administration
Scores for my current students
Scores for students who were mine when they tested during the selected administration
Allows you to see score data for those students who tested in the selected test and administration
Allows you to immediately view score data for those students who are associated to your current rosters, even if they were previously enrolled in a different school or district
Allows you to see score data for those students who were associated with
your school, district, or roster when they were tested in the selected test and
administration
Scores for students who were mine when tested and still mine at the end of the selected administration allows you to see score data for those students who tested in the selected test and administration. This option enables you to see data only for those students who were associated with your school or district at the end of the selected test and administration.
Scores for my current students allows you to immediately view score data for those students who are associated to your current rosters, even if they were previously enrolled in a different school or district.
Scores for students who were mine when tested but may not be mine at the end of the selected administration allows you to see score data for those students who were associated with your school, district, or roster when they were tested in the selected test and administration.

13. Home Page Dashboard: Report Tables
The Home Page Dashboard page displays aggregate data for each subject in a separate table.
Grade displays the grade to which the score data belongs.
Number of Students Tested displays the number of students to date who have completed and submitted their tests for scoring.
Percent Proficient displays the percentage of students to date who have scored proficient on each test.
To access score reports for a particular subject and grade-based test, click a cell for the selected grade and subject. You will only see the tests administered by your district or school. Once a cell is selected, you will be directed to the selected Subject Detail Report.

14. Subject Detail Report
This example shows the information a district user will see on the Subject Detail Report. It shows the list of schools in the district and their aggregate score data.
You can sort the data, show/hide columns, disaggregate data by subgroups and/or test events, and show or hide comparison data.
To disaggregate the score data by a specific demographic subgroup category, from the [Breakdown By] filter, select a group and click [Go]. The report expands to display the data for each subgroup. For example, you can select gender to see the score data for All, Female, and Male students.
To disaggregate the score data by a specific test event, from the [Test Event] filter, select an accommodation and click [Go]. The report expands to display the data for each subgroup.
To drill down to a more detailed level, such as Teacher List, click the magnifying glass next to the school name, then select [View Personnel] on the exploration menu that pops up.
This information is available for the Interim assessments and the Summative assessment.

15. Subject Detail Report: Interim Blocks
The Subject Detail Report for Interim Blocks shows how the state, district, school, teacher, and classes have performed on each block. This report shows the number of students tested by block and the percentage in each block performance level.

16. Student Performance Roster Page
The exploration menu lets you drill down successive levels—district, schools, teachers, classes, and students. This is a sample student roster showing student performance in each achievement level.
Clicking an individual student on this page directs you to students’ individual score reports.
This report is available for the Interim assessments and the Summative assessments.

17. Student Performance Roster Page: Interim Blocks
The exploration menu lets you drill down successive levels—district, schools, teachers, classes, and students. This is a sample student roster for the interim blocks.

18. Individual Student Reports
The students’ individual score report presents a graphic representation of proficiency, along with further descriptions of what it means. It includes more detailed diagnostic information and comparison scores for the class, teacher, school, district, and state. It also includes student performance on each claim along with a description of each claim.
You can batch print individual student reports in a PDF. The PDF reports can be accessed from the [My Inbox] of Test Management Center.
This report is available for the Interim assessments and the Summative assessment.

19. “What” Claims
Using the exploration menu, you can explore the “what” dimension and drill into more content detail at any level (e.g., school, district). This is a sample district claims report.
This report is available for the Interim assessments and the Summative assessments.

20. “What” Targets
For any group of students who takes the interim or summative tests, you can get reports for an individual target—the most detailed level of content.
Information presented compares whether performance was similar, better, or worse than overall performance.
In the future, each target will have a live link to the Smarter Balanced digital library of instructional resources.
This report is available for the Interim assessments and the Summative assessments.

21. “What” Targets Report Benchmark Level
Description
Better than performance on the test as a whole
This target is a relative strength. The group of students performed better on items from this target than they did on the rest of the test as a whole.
Similar to performance on the test as a whole
This target is neither a relative strength nor a relative weakness. The group of students performed about as well on items from this target as they did on the rest of the test as a whole.
Worse than performance on the test as a whole
This target is a relative weakness. The group of students did not perform as well on items from this target as they did on the rest of the test as a whole.
Insufficient Information
Not enough information is available to determine whether this target is a relative strength or weakness.
Better than performance on the test as a whole: This target is a relative strength. The group of students performed better on items from this target than they did on the rest of the test as a whole.
Similar to performance on the test as a whole: This target is neither a relative strength nor a relative weakness. The group of students performed about as well on items from this target as they did on the rest of the test as a whole.
Worse than performance on the test as a whole: This target is a relative weakness. The group of students did not perform as well on items from this target as they did on the rest of the test as a whole.
Insufficient Information: Not enough information is available to determine whether this target is a relative strength or weakness. This could be due to too few items or too few students available to create a report.

22. “When” Trend Reports
Using the exploration menu, you can explore the “when” dimension and explore the performance for the school, teacher, or students over time. This is a sample district trend report.
Select [View Trend] from the exploration menu to view the trend report. Every report lets you toggle between the detailed point-in-time report and a longitudinal graph of the same students over time. You will be able to see how students have performed in both interim and summative assessments.
This report is available for the Interim assessments and the Summative assessments.

23. Trend Report Features Select Administrations to Plot
Choose Who to Graph
View Data by Demographic Subgroup
View Dimensions
Choose What to Graph
Hide Trend Lines
Trend reports are interactive. You can choose the tests, entities, and categories that should be plotted on the graph.
Select Administrations to Plot: The trend report is set to plot Smarter summative and interim administrations on the graph by default. However, you may choose to only plot the administration selected on the Home Page Dashboard screen. To plot only the administration selected on the Home Page Dashboard, select [Selected Test] from the Display drop-down list and click [Go].
Choose Who to Graph: You may select up to five entities to compare at one time by checking the required boxes from the Choose Who to Graph section. A trend line for each selected entity will appear on the graph and will be color coded.
View Data by Demographic Subgroup: You may also choose to plot data points by subgroup, for example, ethnicity.
View Dimensions: You can drill down and view trend data at the teacher, roster, or student levels using the exploration menu.
Choose What to Graph: You may select the overall test subject data (default) or individual reporting categories to plot from the Choose What to Graph section.
Hide Trend Lines: You may temporarily hide a trend line to better view data. Click the trends you want to hide from the trend reports legend box below the graph. The trend you have selected will become grayed out and the line will disappear from the graph. To display the trend line again, click the trend name again.

24. Test Management Center
Summary Statistics
Retrieve Student Results
Plan and Manage Testing
Test Completion Rates
The Test Management Center drop-down menu provides access to the following report options:
Summary Statistics - Count and percentages of students who tested in a selected subject and grade level.
Retrieve Student Results - Access to student performance data files.
Plan and Manage Testing - Generate customized participation reports showing your students’ testing activity. You can determine which students have completed testing and which students have yet to begin or complete a test opportunity.
Test Completion Rates – Generate completion rates by school, grade, subject, and opportunity.

25. Test Management Center: Summary Statistics
The Summary Statistics section displays the statistics for students who have tested for a selected test by administration year and test name (grade and subject or course).
To generate a summary statistics report:
From the Test Management Center drop-down list, select [Summary Statistics].
Select the parameters for your report.
Click [Generate Report].

26. Test Management Center: Retrieve Student Results
From the Retrieve Student Results section, you can download student data for a selected test window by district, school, teacher, or roster. The data will include students’ personal information, including enrolled school and grade level, and the selected test scores and content standard scores (if applicable).
You can download the data file in .xls, .csv, or .txt format.
To generate a data file:
1. From the Test Management Center drop-down list, select [Retrieve Student Results].
2. Select the parameters for your file.
3. Click [Download Roster]. A message appears that your request has been queued and you will be informed via once the file is ready.
4. Once ready, the file is displayed in the My Inbox section. To download the data file, click [Download] in the Status column. Your file is available for 30 days.
PDFs of student score reports generated from the score report pages can also be retrieved from [My Inbox].

27. Test Management Center: Plan and Manage Testing
The Plan and Manage Testing section provides customized participation reports showing students’ testing activity.
To generate a report:
From the Test Management Center drop-down list, select [Plan and Manage Testing].
Select the parameters for your report.
Click [Generate Report]. Alternatively, click [Export Report] to export the data.

28. Test Management Center: Plan and Manage Testing
Which students have not yet tested?
Which students have started but not yet completed their test?
Which students need to finish tests that are going to expire soon?
Which students have paused tests?
You can filter the participation report to include students who match specific criteria. This slide illustrates some common Plan and Manage Testing cases.

29. Test Management Center: Plan and Manage Testing
This slide displays a sample Plan and Manage Testing report filtered on students who have completed the first opportunity in the selected administration and are proficient.

30. Test Management Center: Test Completion Rates
The Test Completion Rates section provides reports of the number of students who have started or completed tests by school or district. District-level users can create a report for a specific school or for the whole district. You can also create a report for all tests or for one specific test.
To generate a report:
From the Test Management Center drop-down list, select [Test Completion Rates].
Select the parameters for your report.
Click [Export Report].
This report is available for the Interim assessments and the Summative assessments.

31. Search Students
You can search for a student’s score reports by the student’s SSID or first name, last name, or full name. This is especially useful if you need to find a student’s score reports but do not know the student’s grade or school. This feature searches for score report results for students, not their test statuses for the current test window.
To search for multiple students at once, you can enter multiple SSID numbers separated by commas in the search field.
The Student Search Results page shows a list of students who matched your search.
To view all the tests taken by a student, click [+].
To view a student’s score report for a particular test, click the test name.

32. Manage Rosters
Rosters are custom groupings of students that you can create for analyzing aggregate score data. For example, school-level users can create a roster of all students in a specific grade who have tested. You can create a custom list of students receiving special instruction or who belong to an extracurricular program to track their performance as a group.
Click the [Manage Rosters] tool to create and manage student rosters.
This report is available for the Interim assessments and Summative assessments.
You can also create and manage rosters from TIDE.

33. Create and Manage Rosters
You can add a new roster or view and edit an existing roster from the Manage Rosters pop-up window that appears when you click the [Manage Rosters] tool.
To create a roster, use the drop-list to select your parameters. From the Roster drop-down list, select [Add New Roster]. Enter the roster details and click [Save] to save your roster.
If you want to view rosters already generated in TIDE by an administrator, select the checkbox for [Show System Defined Rosters]. The available rosters will appear in the drop-down list. However, these system–defined rosters cannot be edited or deleted.
Students can be included in multiple rosters. Rosters may also include students from different grades. However, score reports will only include students who belong to the grade selected on the Home Page Dashboard.

34. Roster Details
You can add, edit, delete, and print roster information.
To edit a roster, modify the necessary roster details. You can edit the roster name, select a different teacher to associate with the roster, and add or remove students from the roster.
To add students to a roster, select the checkboxes for all the grades from which you wish to build the roster. The names of all the available students in the selected grades appear in the Available Students list. Only students in associated grades will be listed in your results. Select the required students and click [Move In].
To print the selected roster, click [Print]. A printer-friendly version of the roster appears in your browser.
To delete the selected roster, click [Delete]. The roster will be deleted from TIDE as well, however, this does not remove your association with the students in the roster.

35. Things to Remember
The ORS helps educators answer questions regarding the assessment data to improve teaching and learning.
The magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data.
All reports can be printed and exported.
The [Help] (User Guide) button is available on every page.
Printable reports for parents can be generated.
Here are a few things to remember:
the ORS helps educators answer questions regarding the assessment data to improve teaching and learning;
the magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data;
all reports can be printed and exported;
the [Help] (User Guide) button is available on every page; and
Printable reports for parents can be generated.

36. Thank You! Further Information Washington Help Desk
American Institutes for Research
Tel
WCAP Portal:
Thank you for taking the time to view this training module. For detailed information, consult your Online Reporting System User Guide located on the WCAP portal or contact the Washington help desk.