1. Online Reporting System
Training Module
Welcome to the Online Reporting System (ORS) training module. It is designed to help you navigate the Online Reporting System.
This training module includes key features of the Online Reporting System. However, you should consult the Online Reporting System User Guide located on your state portal for more detailed information.
Copyright © 2014 American Institutes for Research. All rights reserved.

2. Objectives
After viewing this presentation, you should understand how to:
Navigate the system
View score reports
View participation reports from the Test Management Center
Search for specific students
Manage student rosters
After viewing this presentation, you should understand how to:
Navigate the system
View score reports
View participation reports from the Test Management Center
Search for specific students
Manage student rosters

3. Purpose
Provide timely, relevant reports and guide educators to make valid, actionable interpretations of the data
Interactive data
Near real-time reporting (upon completion of scoring for hand-scored responses)
Provide access to data
Downloadable data files for districts, schools, and teachers
Results for assessments in one system
Participation data
The ORS is designed to provide you with assessment data and tools you need to understand the performance of your students. The system provides interactive data, which allows you to explore the data to understand the results. The near real-time reporting facilitates the speed in distributing the results.
In addition to score reports or performance reports, the ORS also includes participation reports that enable you to determine which students have completed or need to complete testing. Note that students who are not registered in the Test Information Distribution Engine (TIDE) to take a test are not included in the ORS.
Consult the Online Reporting System User Guide for more detailed information.

4. Logging In 1 2 3
To log in to ORS, you must have an authorized address and password, which is the same for TIDE, TDS and ORS. Contact your School Administrator if you do not have a TIDE account.
The ORS is a secure, role-based system. Your access to reports and data in the system depends upon your user role and your district and school associations. Access to the ORS is via the State Assessment Program Portal.
Navigate to the State Assessments Program Portal. Select your user role.
Select ORS. The Login page appears.
Enter your address and password. Click Secure Login.

5. ORS Interface: Welcome Page
The Welcome page appears when you first log in to the ORS and asks you to select the type of report you want to view. Note that you can navigate to a different report at any time within ORS.
Plan and Manage Testing provides participation data for students participating in an assessment. You can determine which students need to complete testing and who may need to take another opportunity of a test, if applicable.
Score Reports provides test score data. You can compare score data between individual students and the school or overall state average scores. Information on strengths and weaknesses in a specified field is also available by reporting category.

6. ORS Interface: Global Tools
The ORS banner that is displayed on top of every page consists of buttons or “Global Tools” for accessing the different reports and performing different tasks.
Score Reports takes you to the Home Page Dashboard that provides overall score data for your district, school, or students.
Test Management Center provides access to participation reports and test summary statistics. You can also download large files of student results.
Inbox takes you to the Inbox section of the Retrieve Student Results page that lists the student data files you have requested. You can also access the inbox from the Test Management Center drop-down list by selecting Retrieve Student Results.
Search Students lets you search for student information by using the student’s name or SSID number.
Manage Rosters allows you to create custom rosters of your students for data analysis.
In addition, the banner consists of the following Global Tools. All the tools may not be available on every page.
Help opens the Online Reporting System User Guide.
Print provides options to print the data displayed on the page.
Export exports the data displayed on the page as an Excel file.
Definitions provides definitions for terms specific to the report you are viewing.

7. Score Report Navigation: General Approach
To help you quickly locate the score data you need, navigation in the ORS is designed to allow you to be able to explore within any of the three dimensions available in the data. These three dimensions parallel the three kinds of questions that the data can help you answer: Who, What, and When. This allows you to view data that can be analyzed appropriately by focus and encourages users to think about the educational questions to which they need answers and access information from that perspective.
The exploration menu, which appears when you click the magnifying glass icon on any score report, is the main method for moving between reports and deciding which type of data you would like to view. To navigate to a report:
From the exploration menu, make appropriate selections under the who, what, and when dimensions.
Click View.
Note that the dimension categories available depend on your user role, the report you are viewing, and the entity level where you have opened the exploration menu. If a drop-down list displays N/A or does not show any options, it means that you cannot navigate any further up or down in that dimension.
While in the ORS, it is recommended to use the exploration menu and other available tools to navigate within the system.

8. Levels of the Three Dimensions
State
District
School
Teacher
Class
Student
Who
Subject
Claims
Targets
What
“Who” lets you see data for institutions and groups. This dimension includes data for the state, district, school, teacher, class, and student. If you are interested in viewing aggregate testing data or individual testing data for a teacher or student, you can explore data in this category.
“What” lets you explore the data by claims and targets within the selected subject/content area. This dimension includes data by subject, claims, and targets.
“When” explores data by time frame. It allows you to see the data either as a snapshot of performance in the current test window or as a historical trend (for the subject area you are viewing). You can filter data reports by the current test window or by trends in order to analyze changes in performance over time.
When
Current Administration
Trend

9. Home Page Dashboard
When you click Score Reports on the Welcome Page or from the banner, the Home Page Dashboard appears.
This page provides data about your students, the number of students tested, and the percentage proficient for the selected administration. District personnel see district summaries, school personnel see school summaries, and teachers see summaries of their students.
Click any table cell containing a value to begin exploring the data for a particular grade and subject.

10. Home Page Dashboard: Select Test and Administration
You can select the test and administration for which you want to view score data.
From the Home Page Dashboard, you can select the test and administration for which you want to view score data. You can also specify the students whose data you wish to view using the available radio buttons.
This feature is particularly helpful for school personnel who want to see how students currently assigned to the user’s class roster performed in previous grades, even if students were enrolled in different schools during those previous administrations.
If you currently have a student who did not test in the selected test and administration, no data will be displayed for that student. An example may be students who moved to your school from out of state.

11. Home Page Dashboard: Student Filters
Scores for students who were mine at the end of the selected administration
Scores for my current students
Scores for students who were mine when they tested during the selected administration
Allows you to see score data for those students who tested in the selected test and administration
Allows you to immediately view score data for those students who are associated to your current rosters, even if they were previously enrolled in a different school or district
Allows you to see score data for those students who were associated with
your school, district, or roster when they were tested in the selected test and
administration
Scores for students who were mine at the end of the selected administration allows you to see score data for those students who tested in the selected test and administration and were associated with your school or district at the end of the selected test and administration.
Scores for my current students allows you to immediately view score data for those students who are associated to your current rosters, even if they were previously enrolled in a different school or district. This view enables you to see your current students’ previous strengths and weaknesses before they begin testing this year.
Scores for students who were mine when they tested during the selected administration allows you to see score data for those students who were associated with your school, district, or roster when they were tested in the selected test and administration.

12. Home Page Dashboard: Report Tables
The Home Page Dashboard displays aggregate data for each subject in a separate table.
Grade displays the grade to which the score data belongs.
Number of Students Tested displays the number of students to date who have completed and submitted their tests for scoring.
Percent Proficient displays the percentage of students to date who have scored proficient on each test.
To access score reports for a particular subject and grade-based test, click a cell for the selected grade and subject. You will only see the tests administered by your district or school. Once a cell is selected, you will be directed to the selected Subject Detail Report.

13. Subject Detail Report
This example shows the information a district user will see on the Subject Detail Report. It shows the list of schools in the district and their aggregate score data.
You can sort the data, show/hide columns, disaggregate data by subgroups and/or test events, and show or hide comparison data.
To disaggregate the score data by a specific demographic subgroup category, from the Breakdown By drop-down list, select a subgroup and click Go. The report expands to display the data for each subgroup. For example, you can select gender to see the score data for All, Female, and Male students.
To disaggregate the score data by a specific test event, from the Test Event drop-down list, select an accommodation and click Go. The report expands to display the data for each subgroup. For students who have multiple accommodations for a test (for example, braille contracted for CAT and braille uncontracted for PT), the report can be disaggregated to include these students into a separate category called “Multiple” or include them in both accommodation values.
To navigate to a more detailed level of the report, such as Teacher List, click the magnifying glass next to the school name, then select Teacher on the exploration menu that pops up, and click View.

14. Student Performance Roster Page
Using the exploration menu, you can view reports at different levels—district, schools, teachers, classes, and students. This is a sample student roster showing student performance in each achievement level.

15. Student Performance Roster Page: Batch Printing ISRs
Using the Print tool, you can batch print student reports in PDF format for all the students listed in the roster. You can either print the reports in a single PDF file where each individual score report is printed on a new page or you can generate individual PDFs for each individual score report that is provided in a zip file.
When batch printing student score reports, you can also choose to include all the opportunities taken by the student, the most recent opportunity, or the opportunity where the student has scored the highest (not currently shown in the above screen).
The PDF reports can be accessed from the My Inbox section of Test Management Center.

16. Individual Student Reports
The students’ individual score report presents a graphic representation of proficiency, along with a parent-friendly description of what it means. It includes more detailed diagnostic information and comparison scores for the class, teacher, school, district, and state. It also includes student performance on each claim along with a description of each claim. The report may also include a trend graph that plots the student’s scale scores at each point in time (across test administrations both within and between school years) and visually depicts his or her performance over time.
If a student has taken multiple opportunities of the test, all the opportunities will be listed on the report and you can toggle between the opportunities by clicking the opportunity number.
Using the Print tool, you can generate a PDF report of the student’s score report. You can choose to include all the opportunities taken by the student, the most recent opportunity, or the opportunity where the student has scored the highest.

17. Subject Detail Report: Interim Blocks
The Subject Detail Report for Interim Blocks shows how the state, district, school, teacher, and classes have performed on each of the blocks. This report shows the number of students tested by block and the percentage in each block performance level.

18. Student Performance Roster Page: Interim Blocks
Using the exploration menu, you can view reports at different levels—district, schools, teachers, classes, and students. This is a sample student roster for the interim blocks.

19. “What” Claims
Using the exploration menu, you can explore the “what” dimension and drill into more content detail at any level (e.g., school, district). This is a sample district claims report.

20. “What” Targets
For any group of students who take the Smarter Summative tests, you can get reports for an individual target—the most detailed level of content.
Information presented compares whether performance was similar, better, or worse than overall performance.

21. “What” Targets Report Legend
Benchmark Level
Description
Better than performance on the test as a whole
This target is a relative strength. The group of students performed better on items from this target than they did on the rest of the test as a whole.
Similar to performance on the test as a whole
This target is neither a relative strength nor a relative weakness. The group of students performed about as well on items from this target as they did on the rest of the test as a whole.
Worse than performance on the test as a whole
This target is a relative weakness. The group of students did not perform as well on items from this target as they did on the rest of the test as a whole.
Insufficient Information
Not enough information is available to determine whether this target is a relative strength or weakness.
Better than performance on the test as a whole: This target is a relative strength. The group of students performed better on items from this target than they did on the rest of the test as a whole.
Similar to performance on the test as a whole: This target is neither a relative strength nor a relative weakness. The group of students performed about as well on items from this target as they did on the rest of the test as a whole.
Worse than performance on the test as a whole: This target is a relative weakness. The group of students did not perform as well on items from this target as they did on the rest of the test as a whole.
Insufficient Information: Not enough information is available to determine whether this target is a relative strength or weakness. This could be due to too few items or too few students available to create a report.

22. “What” Item
The Item Level Report provides summary information for the district’s, school’s, teacher’s, or roster’s performance on each item of a subject for the current window. You can view a listing of all items in a subject sorted by target or block and the performance of the selected entity on those items.
Note: Item Level reports are available for Smarter ICAs and IABs, but not Smarter Summative Assessments.

23. “When” Trend Reports
Using the exploration menu, you can explore the “when” dimension and explore the performance for the school, teacher, or students over time. This is a sample district trend report.
To view a trend report:
On the exploration menu, from the when dimension, select Trend.
Click View.
Every report lets you toggle between the detailed point-in-time report and a longitudinal graph of the same students over time. You will be able to see how students have performed in both the interim and summative assessments.

24. Trend Report Features Select Administrations to Plot
Choose Who to Graph
View Data by Demographic Subgroup
View Dimensions
Choose What to Graph
Hide Trend Lines
Trend reports are interactive. You can choose the tests, entities, and categories that should be plotted on the graph.
Select Administrations to Plot: The trend report is set to plot Smarter summative and interim administrations on the graph by default. However, you may choose to plot only the administration selected on the Home Page Dashboard. To plot only the administration selected on the Home Page Dashboard, from the Display drop-down list, select Selected Test; click Go.
Choose Who to Graph: You may select up to five entities to compare at one time by checking the required boxes from the Choose Who to Graph section. A trend line for each selected entity will appear on the graph and will be color coded.
View Data by Demographic Subgroup: You may also choose to plot data points by subgroup, for example, ethnicity.
View Dimensions: You can drill down and view trend data at the teacher, roster, or student levels using the exploration menu.
Choose What to Graph: You may select the overall test subject data (default) or individual reporting categories to plot from the Choose What to Graph section.
Hide Trend Lines: You may temporarily hide a trend line to better view data. Click the trends you want to hide from the trend reports legend box below the graph. The trend you have selected will become grayed out and the line will disappear from the graph. To display the trend line again, click the trend name again.

25. Test Management Center
Summary Statistics
Retrieve Student Results
Plan and Manage Testing
Test Completion Rates
Test Status Report
State Test Participation Results (for state-level users only)
The Test Management Center drop-down menu provides access to the following report options:
Summary Statistics
Retrieve Student Results
Plan and Manage Testing
Test Completion Rates
Test Status Report
State Test Participation Results (for state-level users only)

26. Test Management Center: Summary Statistics
The Summary Statistics section displays the statistics for students who have tested for a selected test by administration year and test name (grade and subject or course).
To generate a summary statistics report:
From the Test Management Center drop-down list, select Summary Statistics.
Select the parameters for your report.
Click Generate Report.
The report includes the actual number of opportunities taken by students instead of the fixed number of opportunities allowed for a test. For example, if students are allowed to take up to 99 opportunities, but the maximum number of opportunities taken by any student is 20, the Summary Statistics Report will be calculated based on 20 opportunities.

27. Test Management Center: Retrieve Student Results
From the Retrieve Student Results section, you can download:
Student data files for a selected test window by district, school, teacher, or roster. The data contains students’ personal information, including enrolled school and grade level, and the selected test scores and content standard scores (if applicable). You can download the data file in .xls, .csv, or .txt format.
Individual PDFs of each Individual Student Report that is provided in a Zip file for all the students associated with the selected school. A csv file that lists all the PDFs included in the Zip file is also provided.
To generate a report:
From the Test Management Center drop-down list, select Retrieve Student Results.
Select the report from the Report Type drop-down list. The options are Student Data and PDF of Individual Student Reports.
Select the parameters for your file.
Click Download Roster. A message appears that your request has been queued and you will be informed via once the file is ready.
Once ready, the file is displayed in the My Inbox section. To download the report, click Download in the Status column. Your file is available for 30 days.
PDFs of student score reports generated from the score report pages can also be retrieved from My Inbox.

28. Test Management Center: Plan and Manage Testing
The Plan and Manage Testing section provides customized participation reports showing students’ testing activity.
To generate a report:
From the Test Management Center drop-down list, select Plan and Manage Testing.
Select the parameters for your report.
Click Generate Report. Alternatively, click Export Report to export the data.

29. Test Management Center: Plan and Manage Testing
Which students have not yet tested?
Which students have started but not yet completed their test?
Which students need to finish tests that are going to expire soon?
Which students have paused tests?
You can filter the participation report to include students who match specific criteria. This slide illustrates some common Plan and Manage Testing cases.

30. Test Management Center: Plan and Manage Testing
This slide displays a sample Plan and Manage Testing report filtered by students who have completed the first opportunity in the selected administration.

31. Test Management Center: Test Completion Rates
The Test Completion Rates section provides reports of the number of students who have started or completed tests by school or district. District-level users can create a report for a specific school or for the whole district. You can also create a report for all tests or for one specific test.
To generate a report:
From the Test Management Center drop-down list, select Test Completion Rates.
Select the parameters for your report.
Click Export Report.

32. Test Management Center: Test Status Report
The Test Status Report section provides a status report that includes all tests for which a student is eligible. The report includes the student’s information, test name, current test status, and any assigned special code, if applicable. District-level users can create a report for a specific school or for the whole district. School-level users can create a report for their own schools.
To generate a report:
From the Test Management Center drop-down list, select Test Status Report.
Select the parameters for your report.
Click Generate Report or Export Report.

33. Test Management Center: State Test Participation Results
State Participation Counts Report
State Unique Students Counts Report
The State Test Participation Results Report section allows state-level users to generate state-wide participation reports.
The two available reports are:
State Participation Counts – Provides number of students who have started and students who have completed a test (e.g., Summative ELA, Summative Math).
State Unique Students Counts – Provides number of students who have started and students who have completed any test within a test category (e.g., Summative, Interim Comprehensive Assessment).
To generate a report:
From the Test Management Center drop-down list, select State Test Participation Results.
Select the parameters for your report.
Click Export Report.

34. Search Students
You can search for a student’s score reports by the student’s SSID or first name, last name, or full name. This is especially useful if you need to find a student’s score reports but do not know the student’s grade or school. This feature searches for score report results for students, not their test statuses for the current test window.
To search for multiple students at once, you can enter multiple SSID numbers separated by commas in the search field.
The Student Search Results page shows a list of students who matched your search.
To view all the tests taken by a student, click [+].
To view a student’s score report for a particular test, click the test name.

35. Manage Rosters
Rosters are custom groupings of students that you can create to analyze aggregate score data. Based on your user role, you may be able to create and manage rosters, which can be done in ORS and/or TIDE. For some users, rosters need to be created in TIDE before data can be viewed in ORS.
School-level users can create a roster of all students in a specific grade who have tested. You can create a custom list of students receiving special instruction or who belong to an extracurricular program to track their performance as a group.
Click the Manage Rosters tool to create and manage student rosters.

36. Create and Manage Rosters
You can add a new roster or view and edit an existing roster from the Manage Rosters pop-up window that appears when you click Manage Rosters.
To create a roster, select your parameters from the drop-down lists. From the Roster drop-down list, select Add New Roster. Provide the roster details and click Save to save your roster.
If you want to view rosters already generated in TIDE by an administrator, select the checkbox for Show System Defined Rosters. The available rosters will appear in the drop-down list. Note that these system-defined rosters cannot be edited or deleted.
Students can be included in multiple rosters. Rosters may also include students from different grades. However, score reports will only include students who have tested in the grade selected on the Home Page Dashboard.

37. Roster Details
You can add, edit, delete, and print roster information.
To edit a roster, modify the necessary roster details. You can edit the roster name, select a different teacher to associate with the roster, and add or remove students from the roster.
To add students to a roster, select the checkboxes for all the grades from which you wish to build the roster. The names of all the available students in the selected grades appear in the Available Students list. Only students in associated grades will be listed in your results. Select the required students and click Move In.
To print the selected roster, click Print. A printer-friendly version of the roster appears in your browser.
To delete the selected roster, click Delete. The roster will be deleted from TIDE as well. However, deleting a roster does not remove your association with the students in the roster.

38. Things to Remember
The ORS helps educators answer questions regarding the assessment data in order to improve teaching and learning.
The magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data.
All reports can be printed and exported.
The Help (User Guide) button is available on every page.
Printable reports for parents can be generated.
Here are a few things to remember:
The Online Reporting System helps educators answer questions regarding the assessment data in order to improve teaching and learning;
The magnifying glass icon displays the exploration menu, which is used to explore the different dimensions and levels of score data;
All reports can be printed and exported;
The Help button is available on every page and opens a PDF version of the ORS user guide; and
Printable reports for parents can be generated.

39. Thank You! Further Information North Dakota Help Desk
Telephone: Toll-free
Thank you for taking the time to view this training module. For detailed information, consult your Online Reporting System User Guide located on your state portal or contact your help desk.