Teacher Hand Scoring System for Interim Assessments Training Module Welcome to the Teacher Hand Scoring System training module. This presentation will introduce you to the Teacher Hand Scoring System (THSS) for the ISAT ELA/Literacy & Math Interim Assessments.The Idaho Assessment Systems Manual – AIR Systems User Guide located on the ISAT portal contains additional information about features of THSS in The Teacher Hand Scoring System chapter. Idaho State Department of Education, ISAT Assessment System Copyright © 2017 American Institutes for Research. All rights reserved.

Teacher Hand Scoring System Overview Test Administered (TDS) Test Submitted Hand Scoring (THSS) Score Reported The ISAT ELA/Literacy & Math Interim Assessments have some items that require human scoring. THSS allows authorized users to score these item responses for ICAs and IABs. When a student takes a test in the Test Delivery System (TDS) and submits it, TDS forwards hand-scored responses to THSS for scoring, where authorized users can view student responses to each item, enter scores, and mark them with condition codes as necessary. After hand scoring is complete, the student’s scores are forwarded to the Online Reporting System (ORS) and AIR Ways Reporting, where they become available for authorized users to view, download, and print.

Scoring Non-Computer Scored Items Interim Assessment Item Counts Shows total number of items Shows number of items to be human-scored Document is in the “Resources” section of the Portal / Interim Assessment Folder AIR has created a document to determine the number of items that require hand scoring for each Interim assessment. This document is titled Interim Assessment Item Counts and can be found in the ELA/Literacy & Math Assessments resource section under the Interim Assessments sub-folder on the ISAT portal.

Teacher Hand Scoring Roles Scorer – District/School-level users and Teachers (DA, DC, SC, and TE roles in TIDE) Score items assigned to them Reassign items assigned to them Score Manager – District- and School-level users (DA, DC and SC roles in TIDE) Reassign any items There are two types of roles that authorized users can have in THSS. These roles are Scorer and Score Manager. The primary difference between a Scorer and a Score Manager has to do with reassigning items for scoring. Scorers may only reassign items that are currently assigned to them. Score Managers may reassign any item in their school or district. District Test Administrator (DA), District Test Coordinator (DC), and School Test Coordinator (SC) roles in the Test Information Distribution Engine (TIDE) are “Score Managers” in THSS. These users can also score student responses that are assigned to them. Teachers (TEs) only have “Scorer” access in THSS and are only able to score student responses that are assigned to them.

Logging in to THSS To log in to THSS, go to the ISAT portal. Click the ELA/Literacy & Math Assessments card. On the next page, click the Teacher Hand Scoring System card. Enter your user name and password, and then click Secure Login to continue. 5

Test Administrator Role Test Administrators (TAs) no longer have access to the THSS. If a TA attempts to access the system, the user will receive an error message indicating they are not an authorized user. New this year, the Test Administrator (TA) role no longer has access to THSS. Because this system is set up to automatically assign student responses to the person who proctored a student’s test, Score Managers may need to monitor THSS to ensure student responses are being reassigned to scorers who have access to the system. As a reminder, Score Managers include district and school level users such as DAs, DCs, and SCs. Scorers are typically TEs. Please be aware if a TE proctors a student’s test session, student’s hand-scored responses will automatically appear for them. Even if they are proctoring a test session with a student from another teacher’s roster. For a complete list of all the tasks performed by the various user roles, refer to the 2017-2018 User Roles and Access to Idaho Assessment Systems document available in the Resources section of the ISAT portal under the ELA/Literacy & Math Assessments, System Information – Systems Manual sub-folder. 6

Home Page Once you have logged in, you will see the Response List page. For Scorers, this page displays the hand-scored responses for all the students in your associated school or district, depending on your user level. Score Managers will see all responses in their jurisdiction.

Response List Table Column Description Name Name of the student. Item Name of the hand-scored item. Session ID ID of the session in which the student completed the test containing this response. Status “Not Scored” indicates the response still needs to be scored. “Tentatively Scored” indicates the response was scored but still needs to be marked as complete. Assigned To* The Scorer to whom the response is currently assigned. Score Click the button in this column to enter a score for the response. If you are a Score Manager, you can only enter scores for responses that are assigned to you. The response list table contains a number of columns: The Name column is the name of the student. The Item is the name of the hand-scored item. The Session column shows the session ID the student was in when the test was submitted. Status indicates the current status of the item, either “not scored” indicating that the item has not yet been scored by a user, or “Tentatively Scored,” indicating that the item has been scored but not yet marked complete. Score Managers will see an extra column, called Assigned To. This column shows the current Scorer to whom the item is assigned. Click on the button in the Score column to enter a score for the selected item. *This column only appears in the Response List table for Score Managers. 8

Filtering and Sorting All users can filter the responses that appear in the response list table using the drop-down lists. Scorers can filer the table by Test, Session, and Items. Score Managers can additionally filter the table by Grades, Subjects, and/or assigned Scorers. When you open a drop-down list above the table, you can either select a response shown in the list, or type in your own keyword to use as a filter. The table will show only the items that match your filter. You may also sort the items in the table by clicking on the header of the column you wish to sort by. For example, click “Status” to sort the items based on whether they have been scored.

Scoring Responses: Rubric Section When you click the score button next to an item, the Score Response page for that item will appear. This page contains several sections. First, in the Rubric section, review the rubric description for the item to be scored. To view the item’s passage and an example of a perfect response, you can click Exemplar. To view instructions for scoring each item response, users can click on Training Guides.

Scoring Responses: Response Section Next, in the Response section of the Score Response page, review the item stem and the student’s response. While scoring, you may find it helpful to expand or collapse sections by clicking the plus or minus sign in the upper-left corner of that section.

Scoring Responses: Scoring Section In the Scoring section, enter a score for the response by either manually entering a numeric score in the Score field or using the arrow button next to the field to enter the score in increments of one. The score cannot exceed the maximum value in the Points column. If the response cannot be scored, you may assign a condition code to the response by selecting the appropriate option from the Condition Code drop-down list. If a response contains information indicating a potentially troubled student or crisis—for example, extreme depression, abuse, neglect, or intent to harm self or others—the Scorer should contact their District or School Test Coordinator immediately. For more information about how to score responses and use condition codes, refer to the resources in TIDE (THSS Scoring Guide and Training Materials in the General Resources section and Interim Assessment Answer Keys in the Secure Test Materials section). All users have access to the General Resources materials, however DAs, DCs, SCs, and TEs will be the only users able to view and download the answer keys.

Completing Item Responses When you have finished scoring responses, be sure to complete and submit them. To submit scored items, mark the checkbox next to each scored response on the Response page, and click the button labeled Mark Selected as Complete, which is located at the bottom of the page. Once an item is marked complete, it is immediately submitted and can no longer be edited or viewed. Once all of a student’s items for a specific test have been scored and marked complete, the student’s score for the entire test will become available in the Online Reporting System and AIR Ways Reporting.

Reassigning Responses A mentioned before, by default, THSS automatically assigns a response to be scored to the user who proctored the session in which the student completed testing. If a TE proctors a student’s test session, student hand scoring responses will automatically appear for them, even if they are proctoring a test session with a student from another teacher’s roster. If a TA proctors a test session, Score Managers will need to monitor THSS to ensure student responses are being reassigned to scorers who can access THSS, such as TEs. Responses can be reassigned to other Scorers in the users’ jurisdiction. Scorers can reassign their items to other users in the same school. Score Managers can reassign any item in their jurisdiction to any other user in their jurisdiction. For example, school-level users can reassign responses in their school to any school personnel, and district-level users can reassign responses in their district to any district personnel. To reassign one or more responses, select the responses you wish to reassign and click the Reassign All Selected button. A pop up window will appear with a drop-down list that contains users in your jurisdiction. Choose a new Scorer and click the Reassign button. The selected responses will be moved from the previous Scorer’s queue to the new Scorer and will no longer appear in the original Scorer’s Response List.

Thank You! Further Information Visit http://idaho.portal.airast.org Call, fax, or email the American Institutes for Research Idaho Help Desk Hours: 8:00 am to 8:00 pm Mountain Time, Monday-Friday (except holidays) Phone: 1-844-560-7365 Fax: 1-877-218-7663  Email: IDHelpDesk@air.org Thank you for taking the time to view this training module. For detailed information, consult The Teacher Hand Scoring System chapter in the Idaho Assessment Systems Manual – AIR Systems User Guide available on the ISAT portal or contact the Idaho Help Desk.