AIR Ways Reporting Training Module Welcome to the online training module for AIR Ways Reporting. This training module is designed to help you navigate AIR Ways and understand the different types of information available in the system. The AIR Ways Reporting User Guide located on Montana’s portal contains additional information about AIR Ways Reporting features. Please note that Montana’s systems may vary from the images shown in this presentation. Copyright © 2017 American Institutes for Research. All rights reserved.

Objectives After viewing this presentation, you should understand how to: Navigate AIR Ways Reporting View student test results and performance on items Search for reports on specific students Download and print report information Change display settings Customize the appearance of report tables After viewing this presentation, you should understand how to: Navigate AIR Ways Reporting View student test results and performance on items Search for reports on specific students Download and print report information Change display settings Customize the appearance of report tables

Purpose of AIR Ways Reporting Displays student and roster test performance tailored to the needs of classroom teachers: Includes item-level reports for interim assessments Also provides school and district reports for authorized users Allows teachers to: View students’ actual work and responses to each test item Compare individual students’ performance with overall class performance on a test or item Pinpoint areas for improvement throughout the school year AIR Ways Reporting provides access to performance on assessments and individual test items for students and rosters. School- and district-level users can additionally view reports for their school and/or district. AIR Ways Reporting provides insight into how students performed on specific items of interim assessments. Teachers can use the system to view students’ actual work, presenting a clear picture of the topics that students have mastered and where they need to improve. This information can serve as a starting point for meaningful conversations about student progress and help teachers pinpoint areas for improvement throughout the school year.

Key Differences Between ORS and AIR Ways Reporting Online Reporting System AIR Ways Reporting High level reports for all assessments Export score data for multiple grades and tests in one spreadsheet Useful for accountability purposes Used only for interim assessments View actual items and responses Intended primarily for classroom use There are some key differences between the Online Reporting System (ORS) and AIR Ways Reporting. ORS may be used to access high-level score reports for any assessment, including those with fully secure items. ORS allows users to retrieve student results for multiple grades and tests in a single spreadsheet. This makes ORS well-suited for accountability purposes. AIR Ways Reporting produces detailed, item-level reports for interim assessments, allowing users to view individual items and students’ actual responses. AIR Ways Reporting lets teachers see each student’s actual response to each item, which makes this system a valuable tool for teachers to tailor classroom instruction.

Logging in to AIR Ways Reporting To log in to AIR Ways Reporting, you must have an authorized email address and password, which is the same as the one you use for TIDE and the TA Interface. Contact Montana School Administrator if you do not have a TIDE account. AIR Ways Reporting is a secure, role-based system. Your access to reports and data in the system depends upon your user role and your district and school associations. To log in to AIR Ways, go to Montana’s portal. Click the appropriate user role or assessment card. On the next page, click the AIR Ways Reporting system card. Enter your username and password, and then click Secure Login. 5

My Assessments The Dashboard is the first page you will see when you log in. On this page, all users will see the My Assessments table, which provides a list of the assessments completed by your students. Each row in the table displays the assessment’s name, test reason, the number of students who have completed the assessment, the average score earned by those students, a graph showing the percentage of students in each performance level, and the date the assessment was last taken. Test reasons are categories used to classify test opportunities for reporting purposes. If your students will take the same assessment multiple times, assigning different test reasons can help you to distinguish scores for the different test opportunities in AIR Ways Reporting. Test administrators can assign a test reason at the time of testing, when they start a test session in the TA Interface. Otherwise, the test reason will appear as “Unassigned.” We will discuss how to modify test reasons later in this module. To view your students’ performance on a test, click the magnifying glass next to an assessment’s name. We will discuss viewing student test performance in a moment.

My Students If you are logged in as a teacher, you will see a second table on the Dashboard titled My Students. You can get to this table by scrolling down, or access it more quickly by collapsing the My Assessments table. To collapse the My Assessments table, click the blue down arrow next to its title. When the table is collapsed, this arrow will point up instead of down. The My Students table shows a list of your students, their student IDs, their most recent assessment, and the date their most recent assessment was taken. To view the Student Portfolio Report, which includes a list of all assessments taken by a student, click the magnifying glass next to a student’s name. We will discuss Student Portfolio Reports later in this training module.

AIR Ways Reports Reports Available in AIR Ways Reporting: Report Type Used to… My Students’ Performance on Test View how students and rosters in your classes performed on a test School Performance on Test View how students and rosters in your school performed on a test District Performance on Test View how schools in your district performed on a test Student Performance on Test View how a single student performed on a test Student Portfolio View how a single student performed on all tests AIR Ways Reporting organizes performance data into different types of reports. Here, you can see a summary of the reports that are available in the system. Note that not all reports are accessible to all users. For example, only schools can view school performance on a test, and only district-level users can view district performance on a test. For detailed information about report types and user roles, see your AIR Ways User Guide.

Reports for Teachers, Schools, and Districts To view a report showing how your rosters and students performed on an individual assessment, click the magnifying glass next to an assessment on the Dashboard. The sample report shown here is for a teacher. School-level users will see a report for all rosters and students associated with their school, and district-level users will see a report for all schools in their district. In other words, reports for teachers, schools, and districts all contain similar information, but for different populations. Note that AIR Ways Reporting only includes data for scorable test opportunities. A student’s test opportunity cannot be scored when it has a test status of “Expired” or “Invalidated.” For more information on non-scorable test opportunities, consult the AIR Ways Reporting User Guide.

Performance by Roster Some reports can be viewed by roster or by student. This sample report shows performance by roster. You can move between performance by roster and performance by student using the tabs above the report. The report’s first row shows the maximum obtainable points on the test. The second row shows average performance for all students in the report. The other rows show how your rosters performed on the test. To view the performance of individual students within a roster, click the magnifying glass next to the roster’s name.

Informational Popups On some pages in AIR Ways Reporting, you can learn more about the data in a table by clicking the blue information icon next to it. A pop-up will describe the data in more detail.

Expanding and Collapsing Columns Reports contain columns that can be expanded or collapsed. For example, in the report shown here, the Total column is expanded to show the student count, test completion rate, average score, and performance distribution for each roster. You can see additional columns on the right side of the report. These columns are collapsed. Note that column names will vary depending on the test you are viewing. To collapse a column, click the minus sign above its name. Additional data will appear, and the minus will change to a plus. To expand a collapsed column, click the plus sign above its name. As you expand columns, the report may become too large to fit all columns on the screen at once. To view any columns that extend off the edge of the page, click the blue arrow that appears to the side of the report.

Performance by Student To view performance by student, click the Performance by Student tab near the top of the report. When viewing performance by student, each test opportunity for each student is displayed on a separate row. Therefore, students may appear more than once in the table if they have taken the same test multiple times. To view more information about an individual test opportunity, click the magnifying glass next to a student’s name.

Student Performance on Test After clicking the magnifying glass next to a student’s name, a report will appear that shows the student’s performance on each item of the test for the selected opportunity. You can return to a previous report by selecting an item in the breadcrumbs that appear in the gray bar above the report. Item-level data on this page appear as clickable links. Clicking one of these links will pull up an item preview window that shows the student’s actual response.

Item Preview Windows For example, you can click the score that a student received on an item to open the Item Preview Window for that item. The window includes a frequency distribution of student responses, content standards, a scoring rubric, and a view of the item as it appeared on the test, including the student’s response. You can toggle whether to show or hide student test settings in the item by clicking the switch in the upper right corner of the window.

Item Preview Windows Viewing item-level information is what makes AIR Ways Reporting so useful for classroom instruction. For example, teachers can see a student’s wrong answer and then start a conversation with the student about how they approached the problem and where they went off track. You can change the roster or student displayed in the pop-up using the Roster or Student dropdown menu located at the top of the pop-up. To view performance for the previous item on the test, click the item button in the upper left corner of the screen. To view performance for the next item on the test, click the item button in the upper right corner.

Student Portfolio The Student Portfolio shows a list of all tests that the student has completed, including the student’s score and performance level, as well as the average score and performance distribution of all your students who took the test. You can access this report from the Dashboard by clicking a magnifying glass next to a student in the My Students table. You can also jump to a Student Portfolio for any of your students by entering his or her student ID into the search box on the right side of the gray bar above the report. From the Student Portfolio, you can view the student’s performance on a particular test by clicking the magnification glass next to the name of the test.

Printing and Downloading To print or download data on any page in AIR Ways Reporting, click the Print button in the upper right of the report. If you are printing a report that contains multiple tables, you will also need to select which table you wish to print.

Printing and Downloading, continued When you click Print, a print preview screen will appear. Next, examine the options on the left side of the screen. If the report you wish to print provides data about individual test items, you can choose to include or exclude this information from the printed report by selecting Summary Only or Summary and Item Scores. Finally, select whether you wish to print the report, download the report as a PDF file, or download the report as a spreadsheet in CSV format. Note that your available print options will depend on the format you select. When you are ready to download or print, click Confirm.

Using the Inbox Downloaded reports will appear in your Inbox, which is located in the banner. Click the Download button next to the report in order to download it.

My Settings Users wishing to manage test reasons, change display settings, manage rosters, or change their user role can perform these tasks using the My Settings menu. The options in this list will vary, depending on your user role. In the next section of the training module, we will discuss these features in detail.

Manage Test Reasons: Search Test Sessions If you wish to classify one or more test opportunities using test reasons, you can assign test reasons after the fact using the Manage Test Reasons feature. To assign a test reason to a set of test opportunities, select Manage Test Reasons from the My Settings menu. In the pop-up window that appears, search for test opportunities by session ID or by an existing test reason and date. Click Search to display test opportunities.

Manage Test Reasons: Select Test Opportunities Your results will appear in the bottom of the window. Test opportunities are organized hierarchically. Click the plus sign next to a session ID to see all the tests that were in that session. Click the plus sign next to a test name to see all the students who took that test during the session. You can select test opportunities by marking one or more checkboxes. You can select students individually, select all students who took a test, or select all students in a test session. When finished, scroll to the bottom of the pop-up window and click Assign Test Reasons to confirm your selections.

Manage Test Reasons: Assign Opportunities Finally, you will be asked to select a test reason to assign to the selected opportunities. Choose a test reason from the drop-down list, and then click Confirm to assign the test reason.

Select Tests to Display Teachers can change which tests are displayed in AIR Ways Reporting. For example, an ELA teacher can choose to hide all Mathematics tests. By default, all tests are displayed for all teachers. To change displayed tests, click Select Tests to Display in the My Students menu. A pop-up window will appear that lists all available tests that can be displayed in AIR Ways Reporting. You may uncheck a box to hide a single test, or uncheck a box for a subject or grade to hide all tests for that subject or grade. When you have finished selecting tests to be displayed, click Save & Close to save your selections and close the window. AIR Ways Reporting will remember your test display preferences even when you log out. If you cannot see information for a specific test, you can check here to make sure the test is not hidden.

Select Roster Preferences School- and district-level users can choose which rosters display for teachers by selecting Select Roster Preferences in the My Settings menu. In the pop-up window that appears, choose whether to view all rosters, or view rosters according to teacher preferences. If you select “Teacher Preferences,” you will not see data for teachers who have excluded a given test. To continue the example on the previous slide: If an ELA teacher chooses to hide mathematics tests, then you will not see that teacher’s rosters when viewing mathematics tests.

Set Student Settings Display To change whether student test settings are shown by default in item previews, click Set Student Settings Display in the My Students menu. A pop-up window will appear that allows you to choose whether to show or hide test settings by default when you view a test item. You can always toggle this setting on and off for individual items using the switch in the Item Preview Window. If you choose Yes, you will view all test item previews with the student’s test settings applied. If you choose No, you will see all test item previews without the student’s test settings. When finished, click Save to close the pop-up window.

Manage Rosters School- and district-level users can also add, edit, or upload rosters from the My Settings menu. These functions all work exactly the same way they do in TIDE. For details about managing rosters, consult your TIDE User Guide.

Change User Role If your account is associated with multiple user roles, you may change the role you are using by selecting Change Role in the My Settings menu.

Customizing Report Tables You can customize report tables in AIR Ways Reporting in the following ways: Filter Sort Change the number of rows displayed per page Move to the next/previous page Next we will discuss some helpful tools for customizing report tables. You can filter and sort report tables in AIR Ways Reporting. You can also change the number of rows to display on each page and move to a different page. For detailed information about customizing report tables, see your AIR Ways User Guide.

Filtering Reports You may wish to filter a report to show only certain information. For example, you could filter by test reason to show only the opportunities that have “2nd Quarter Interim” as the test reason. You could also filter by the claim, “Modeling and Data Analysis,” to show only the items associated with that claim. To filter a report, select the Filters tool and then select the desired filter criteria in the drop-down lists. The available criteria will depend on the report type and the assessment you are viewing. After selecting your criteria, click Apply to apply the filter.

Sorting Reports You can sort a report by clicking the arrows next to a column header. Click once to sort in descending order, and click a second time to sort in ascending order.

Customizing Table Rows You can change the number of rows displayed on each page of the report by entering a number into the text box labeled Rows per page, which is located under the report table. If your report is two or more pages long, you can also go to the next or previous page using the arrow buttons shown here.

Contact Your Help Desk You can contact Measured Progress Help Desk for assistance with any technical issues you encounter. When contacting the Help Desk, please be ready to provide: Any error messages that are appearing (including codes) Your operating system and browser information Your network configuration information Your contact information for follow-up by phone or email Any other relevant information, such as test names or content areas, student IDs, session IDs, and search criteria For test administration or policy issues, please contact your district test coordinator. You can find the AIR Ways Reporting User Guide and other helpful resources on Montana’s portal. However, if you still require assistance with troubleshooting a technical issue, you may also contact Measured Progress Help Desk. When contacting, please be ready to provide the following information: Any error messages that are appearing (including codes) Your operating system and browser information Your network configuration information Your contact information for follow-up by phone or email Any other relevant information, such as test names or content areas, student IDs, session IDs, and search criteria For questions about test administration or policy issues, please contact your district test coordinator.

Thank You! Further Information Montana Portal: Montana Help Desk http://mt.portal.airast.org/ Montana Help Desk Toll-Free: 1-888-792-2741 montanahelpdesk@measuredprogress.org Thank you for taking the time to view this training module. For detailed information, consult your AIR Ways Reporting User Guide located on Montana’s portal or contact Measured Progress Help Desk