Rater & Verifier Training Presented by TELPAS 2015 Rater & Verifier Training Presented by *This training does not replace your responsibility to read and follow all manuals and supplements*

What is TELPAS? The Texas English Language Proficiency Assessment System assesses all students who are identified as LEP including parent denials in the domains of reading, writing, listening and speaking K-1 2-12 Reading Holistically rated observational assessments Multiple choice online reading test Writing Holistically rated student writing collections Listening Speaking

The Testing Window Writing Online Reading Collect Writing Samples February 16 – March 3 Verifiers Verify Collections March 3-6 Raters Rate Writing Collections and other Domains (Reading for K-1, Listening & Speaking) March 16-25 Online Reading March 16 – April 2 March 16-20 (HS/MS/AC) March 23-27 (Elementary)

Who tests? ALL students K-12 who are identified as LEP including parent denials are required to participate in TELPAS Exceptions: ARDCs in conjunction with the LPAC may determine that a student receiving Special Education services should not be tested in one or more domain of TELPAS. Participation should be considered on a domain by domain basis. Students who enroll on or after March 16 will only take the online reading test (contact the Testing Office for confirmation if test is needed)

CTC Responsibilities Read and adhere to all manuals Attend required coordinator training sessions CTC and campus principal are responsible for test security and confidentiality on the campus Supervise and actively monitor testing Be the campus contact for all questions about testing Report testing irregularities and security violations immediately to the district testing coordinator Compile required documentation and turn in to testing office

CTC Responsibilities Obtain CTC access in Texas Training Center Identify and train a writing collection verifier(s) by February 16 Identify and train Raters by February 16 Identify and train Online Test Administrators by March 6 Train appropriate personnel to enter/verify student information in TAMS (data can be entered by the CTC) Ensure computers have been checked for TestNav capability Ensure that all students are identified and tested Oversee the implementation of validity and reliability

CTC Responsibilities for Training Raters Identify Raters and the type of online training they need to complete Ensure Raters understand the importance of being properly trained on the holistic rating process Ensure Raters have obtained access to Texas Training Center Ensure new and returning Raters know which training to take and for which grade cluster Monitor that all Raters complete their training requirements and receive supplemental training support if needed

Rater Responsibilities Attend training on administrative procedures Obtain access to Texas Training Center Complete holistic rating training requirements (online calibration sets) Print Certificate of Completion and turn in to CTC (page 75 Rater Manual) Assemble and verify student writing collections Rate writing collection and the listening & speaking domains Complete required documentation Maintain security and confidentiality of student writing collections and ratings Report test irregularities Sign all oaths

Rater Credentials Must have student on spring roster during the assessment window (train PE or Fine Arts teacher as a backup rater) Be knowledgeable about the student’s language ability Be trained in the rating process Complete online rating calibration set with a 70% or better (3 attempts) Attend mandatory supplemental training if the calibration set was not passed after the first two attempts

Online Training- Raters Raters need to self register for an account in the Texas Training Center https://texas.pearson.desire2learn.com Site Code: see CTC Returning users that have forgotten their login information click on the “Login Help” link New Users click on the “Self Registration” link Page 57 in Rater Manual

Online Training- Raters Raters will access the online training and calibration activities through Texas Training Center Raters need to view the training in their grade cluster K-1 2 3-5 6-8 9-12 Raters will complete both the Online Basic Training Course & Calibration Activities OR the Calibration Activities

Online Training- Raters Note: If a rater has not calibrated in the last three years they are considered “new” and must complete the Basic Online Training Course AND Calibration Activities

Online Training- Raters Access the Basic Training Course for grade cluster New, new to grade cluster or a rater who has not passed the Calibration Activities in the last 3 years Basic Training Course is open from January 26 - February 7 Click on TELPAS 2015 under the “My Texas Training Center Program” header then: Page 62 in Rater Manual

Online Training- Raters Access the Calibration Activities Calibration Activities are open from February 17-21 ALL Raters must complete and pass the Calibration Activities with at least a 70% Raters have 3 attempts to pass and must attend Supplemental Support Training before the 3rd attempt Teacher collaboration is NOT allowed on Calibration Activities Page 62 in Rater Manual

Online Training- Raters As per the TEA Security Supplement: Test Administrators must independently complete TELPAS calibration activities. Answer sheets or notes that contain responses to TELPAS calibration activities are not allowed to be shared Motes created by raters during practice and calibration activities must be destroyed

Supplemental Support Training If a Rater has not passed the Calibration Set with a 70% or higher after the second attempt, the Rater is required to attend Supplemental Support Training CTCs submit a list of Raters needing Supplemental Support by February 23 (use form provided by Testing Dept) A Rater will not be able to attempt the 3rd Calibration Set until they have attended Supplemental Support Training Supplemental Support Training is scheduled for: February 25, 26, or 27 from 4:00-6:00 at the Johnson Center

Assembling the Writing Collection K-12th teachers will collect writing samples during the window and turn in to the CTC Writing samples are considered to be secure materials Writing Sample Collection window is February 16-March 3 All Writing Sample Collections must be returned to the CTC by March 3 Must follow the guidelines for collecting writing samples

Writing Collection Guidelines At least 5 writing samples (Rater Manual page 19 for examples of writing) At least 1 narrative about a past event At least 2 academic samples from math, science, social studies Samples must include the student name and date Cover sheet must be used and stapled to each writing collection (page 53 of Rater Manual) Raters will initial the Verification Checklist (page 54 of Rater Manual) Samples may be typed but spell and grammar check must be disabled May be a “clean” photocopy from a journal Must only be collected during the window

Writing Collection Guidelines What NOT to include… Papers done with heavy use of dictionaries/thesaurus Papers with teacher, peer, or parent corrections Written primarily in another language other than English Worksheets or Q&A writing assignments Brief, incomplete or rushed papers Writings that have been copied from a textbook, lesson or other sources. Must be in students own words.

Cover Sheet Rater initials each box while collecting writing samples Cover Sheet must be attached to each Writing Collection Verifier initials each box during verification window FRONT BACK

Verifier Responsibilities Attend training on administrative procedures Obtain access to Texas Training Center Complete online training “Assembling and Verifying Writing Collections” Print Certificate of Completion and turn in to CTC (page 75 in Rater Manual) Verify student writing collections in accordance with the checklist Sign Oath Verifier cannot be a TELPAS Rater or the CTC

Online Training- Verifiers Verifiers need self register for an account in the Texas Training Center https://texas.pearson.desire2learn.com Site Code: telpas2015 Returning users that have forgotten their login information click on the “Login Help” link New Users click on the “Self Registration” link Page 57 in Rater Manual

Online Training- Verifiers Verifiers need to self register for an account in the Texas Training Center https://texas.pearson.desire2learn.com Site Code: telpas2015 Click on TELPAS 2015 under the “My Texas TrainingCenter Program” header then:

Verifying the Writing Collection Verifiers ensure that the writing collections meet all of the criteria based on the Verification Checklist by initialing each item (checklist is on the back of the Writing Collection Cover Sheet) page 54 of Rater Manual Verifiers sign the bottom of checklist then return the collection to the CTC Writing Collections are secure materials Verification Window is March 3 - March 6

Rating Students Raters should review pages 24-30 in Rater Manual prior to rating students in each domain (writing, listening, speaking) CTCs will return the Writing Collection to the Raters during the Rating Window March 16 – March 25 Raters must use the district required Rating Roster form to indicate the student’s rating While rating students, Raters must refer to the PLDs (Proficiency Level Descriptors) pages 33-37 in Rater Manual Rating Roster must be signed by Rater, Additional Rater (if needed) Teacher Collaborator and Campus Principal All student ratings and Writing Collections must be returned to the CTC by March 25

Rating Roster An Additional Rater is required if the Rater did not pass the Calibration set with a 70% or higher after the 3rd attempt There must be teacher collaboration on ratings as part of the Validity & Reliability process

Validity and Reliability TEA requires districts to implement one or more rating support and verification procedures for every campus with TELPAS-rated students SAISD has developed a TELPAS validity and reliability plan that meets TEA requirements Campus test coordinators will be expected to submit required documents to the district test coordinator by the requested deadlines in order to comply with our district validity and reliability plan An internal audit will be conducted by the Testing Department

SAISD TELPAS Validity and Reliability Procedures For all language domains – grades K-12: Raters will collaborate with other teachers of the students in determining the student’s rating. One collaborator will initial the TELPAS Rating Roster for each child For Writing– grades 2-12: 5% of all writing collections will be randomly pulled for assembly verification by the campus test coordinator

Enter Ratings CTCs must create Rating Entry Groups in order to enter ratings for students CTCs, Rater or Rater Assistants may enter ratings Must be entered in TAMS Domains Grades Listening K-12 Speaking Reading K-1 Writing Level Rating Beginning B Intermediate I Advanced A Advanced High H

Enter Ratings If a student will not be rated in a domain, one of the following codes must be used: Code What it Means Information E Extenuating Circumstance Documentation of the reason must be maintained on campus X ARD Decision Decision is made student by student and domain by domain *decision for a student not to participate in all 4 domains should be rare

Enter Ratings

Campus Procedures

Input Campus Specific Procedures Here Include any information that is campus specific for collecting, verifying and rating writing samples

Security Training Modules Web-based Texas Test Administrator Online Training Modules www.TexasAssessment.com/TAonlinetraining All three modules must be completed by certified campus personnel involved if they: - are new to the district - took an incident in the 2013-14 - have not completed the Modules in the last 3 years

The following govern test security General Security Information The following govern test security Texas Education Code (TEC) Chapter 39, Subchapter B Texas Administrative Code (TAC) 19 Subchapter 101, Assessment Family Educational Rights and Privacy Act of 1974 (FERPA) Texas Penal Code 37.10 - Tampering

Penalties for Prohibited Conduct Placement of restrictions on the issuance, renewal, or holding of a Texas educator certificate, either indefinitely or for a set term Issuance of an inscribed or non-inscribed reprimand Suspension of a Texas educator certificate for a set term Revocation or cancellation of a Texas educator certificate without opportunity for reapplication for a set term or permanently Release or disclosure of confidential test content is a Class C misdemeanor and could result in criminal prosecution under TEC ξ39.0303, Section 552.352 of the Texas Government Code, and Section 37.10 of the Texas Penal Code.

Security Oaths TEA requires that everyone be trained and sign a security oath before handling secure testing materials or participating in the administration of a state test A person who has several roles during TELPAS testing must sign all oaths (Test Administrator, Rater, Verifier, etc) Paper test require that the test administrator has signed the bottom box of the oath

Security and Confidentiality Account for all secure materials before, during, and after each test administration (including Student Authorizations) Protect the contents of all tests booklets and student answer documents (including writing samples) No person may view, reveal, or discuss the contents of a test or answer document unless specifically instructed to do so by the procedures in the test administrator manuals and the bottom portion of the oath has been signed No collaboration allowed during online calibration sets

Secure TELPAS Information Content of the online reading test is secure and confidentiality must be maintained Student Authorization Letters are to be kept secure All student data in the TELPAS system or entered into the system is confidential Passwords and user IDs are secure and should be kept confidential. DO NOT SHARE!

Testing Irregularities Incidents resulting in a deviation from documented testing procedures are defined as testing irregularities Procedural Examples Serious Examples Improper accounting for secure materials Directly or indirectly assisting students with responses to test questions Eligibility Error Tampering with student responses Monitoring Error Viewing secure content IEP Implementation Issue Discussing test content, student responses or student performance Procedural Error Formally or informally scoring student tests Duplicating or recording test content

What is NOT active monitoring? Monitoring during test administrations is the responsibility of the test administrator, the campus test coordinator and administrators. What is NOT active monitoring? Anything that takes the test administrator’s attention away from the students during testing. Examples include: Working on the computer or checking email Reading a book, magazine, or newspaper Grading papers or working on lesson plans Leaving the room without a trained substitute test administrator in the room Leaving students unattended during meals or breaks

Reduce the risk of having an irregularity! Before the test administration: Emphasize importance of training and reading manuals Account for all test materials Ensure proper storage of materials Verify accuracy of student information Verify testing requirements for students receiving special education services

Reduce the risk of having an irregularity! During test administration: Ensure each student receives the correct assessment and testing material(s) Emphasize and verify active monitoring Ensure all testing personnel understand that they may NOT: provide assistance view the tests without authorization discuss confidential student information check for strategies

Reduce the risk of having an irregularity! After test administration No unauthorized viewing (only TEA may permit) No scoring of student responses Shred Student Authorization forms (ASAP) No discussion of confidential student information No erasing stray marks or darkening response ovals No copying of students answer documents Account for all test materials No opening answer documents to view answers

Pitfalls to avoid: Assigning untrained staff to administer and/or monitor tests (must be certified personnel) Failing to inventory testing materials Having insufficient test booklets and/or test administrator manuals Not accounting for all test booklets and answer documents each day Teacher violates time limit Testing beyond the normal school day

Pitfalls to Avoid: Assigning a student the incorrect assessment or student authorization form Failing to give appropriate accommodation(s) Improper (or nonexistent) monitoring Leaving secure materials unattended (to include Student Authorizations and Writing Samples) Stopping an on-line session prior to removing absent students

Reporting & Documentation If you become aware of any irregularities, contact the DTC immediately with the who, when, where, what and why. If the irregularity is deemed an incident the following documentation is required: Incident report including a plan of action (local form) Signed statement(s) from individual(s) involved (typed, printed and signed name and dated) Statements should not include names of students The local disciplinary referral form used to report disciplinary actions taken against students for cheating, cell phone issues or disruptions

Plan of Action

Questions?