1. 2013-2014 TCAP Administration Training for School Assessment Coordinators Thompson School District Colorado Department of Education

2. Who should be trained?  Everyone involved in TCAP administration must be trained each year. This includes DACs, SACs, Proctors and any other school or district staff involved in any aspect of TCAP administration  DACs/SACs must collect and maintain a record of signed training completion forms as well as signed confidentiality agreements for all who come in contact with test materials  SACs responsible for ensuring that all test proctors are properly trained

3. Overview 1. Colorado Achievement Assessments 2. Personnel / Core Assessment Team 3. Important Dates 4. TCAP Assessment Administration 6. Training 7. Test Accommodations 8. Test Security 9. Data Quality 10. Communication and Resources 3

4. Colorado Achievement Assessments

5. TCAP vs. CoAlt TCAP CoAlt ACCESS for ELLs Timed Whole class Scored only by CTB No breaks during session Untimed One-on-one Partially scored by examiner Students may take breaks One-on-one and grade span groups Partially scored by administrator

6. TCAP Assessment PURPOSE: To measure student achievement relative to the Colorado Model Content Standards. The TCAP replaces the CSAP in order to support schools as they transition to curricula aligned with the Colorado Academic Standards. 26 tests  Reading and Writing - Grades 3-10 (Grade 3 Reading separate)  Mathematics - Grades 3-10 (calculators certain sessions Grades 9 & 10)  Lectura/Escritura (Grades 3 & 4)

7. Personnel and Core Assessment Team

8. Core Assessment Team  District Assessment Coordinator (DAC)  Carmen Williams  Assessment Specialist  Ronda Hanson  Andy Riley  English Language Administrator  Clint Richards  Director of Exceptional Student Services  Charlie Carter  Director of Elementary Education  Dr. Paul Bankes  Director of Secondary Education  Dr. Margaret Crespo

9. District Assessment Coordinator  What is a DAC?  District Assessment Coordinator  Appointed by the district superintendent  Official point of contact with CDE  Responsibilities vary by district, but always include:  Training School Assessment Coordinators (SACs)  Ensuring all students are assessed  Ensuring standardized administration of assessments  Declaring, investigating and reporting misadministrations  Managing and communicating assessment score results

10. Important Dates

11.  All testing, including make-ups, must be completed within our district’s testing window  February 5 through March 14 – CoAlt testing  February 12 – Housekeeping paperwork and training verifications due/ 3 rd grade Reading Teacher Read Direction order forms due  February 19 –Test proctor training needs to be complete for 3 rd grade Reading Proctors  February 20 – Oral Presentation and Teacher Read Direction order forms due  February 20 – TCAP Testing Schedules due  March 6 – Test proctor training completed

12. Important Dates (Continued) All testing, including make-ups, must be completed within our district’s testing window 3 R February/ March 242526272834567 MTWTHFMTW F 12 March 10111213141718192021 MTWTHFMTW F 3 W 12 3 M 1 2 4 R/W 123456 4 M 123 5 R/W 123456 5 M 123 6 R/W 123 4, 56 6 M 123 7 R/W 123 4, 56 7 M 123 8 R/W 123 4, 56 8 M 123 9 R/W1, 2, 3 4, 5, 6 9 M1, 2, 3 10 R/W 1, 2, 3 4, 5, 6 10 M1, 2, 3 Grade Total number of Testing Days 36 49 59 65 75 86 93 104

13. Make-ups / Check-in Schedule 3 R February/ March 242526272834567 MTWTHFMTW F MU √ in MarchApril 14171819202124252627287891011 FMTWTHFMTW FMTW F 3 W Make-UpsCheck-ins 3 M Make-UpsCheck-ins 4 R/W Make-UpsCheck-ins 4 M Make-UpsCheck-ins 5 R/W Make-UpsCheck-ins 5 M Make-UpsCheck-ins 6 R/WMU Make-UpsCheck-in 6 MMU Make-UpsCheck-in 7 R/WMU Make-UpsCheck-in 7 MMU Make-UpsCheck-in 8 R/WMU Make-UpsCheck-in 8 MMU Make-UpsCheck-in 9 R/W Make-UpsCheck-in 9 M Make-UpsCheck-in 10 R/W Make-UpsCheck-in 10 M Make-UpsCheck-in

14. TCAP Administration

15. Essential Understandings  Assessment results are part of the student’s permanent academic record per Colorado law [22-7- 409(1.9)].  Every student enrolled in a public school shall be required to take the (state) assessments (in the content areas and grades administered) [22-7- 409(1.2.a.1.d.I)].

16. “Standardized” Assessment  Standardization - All students will have the same test content, resources, directions, testing conditions, and scoring procedures  No students are provided an unfair advantage  Standardization requires that all proctors administer all test sessions in the same way  A score obtained by one student in one part of Colorado will mean the same as the same score obtained by another student in a different part of Colorado

17. Permitted Practices Due to the complex nature of a standardized assessment process, any practice not specifically permitted should be presumed inappropriate until and unless specifically authorized by CDE. “If it doesn’t say that you can, then you can’t.”

18. Standardization and the Media  Media presence at a school during testing can create a non-standard assessment environment by causing disruptions in the regular flow of the school day or by adding increased stress on the students being assessed  Media are expressly prohibited from taking pictures or video of test materials or students in the process of being tested. CDE notifies the media of this prohibition prior to testing time

19. TCAP School Schedule Deadline  TCAP school schedules are due to Ronda Hanson at the Department of Accountability and Accreditation per the following schedule: Grade 3 Reading – February 12 th, 2012 All other Tests – February 20 th, 2012  Schedules must include start and end times for each grade and content  School TCAP schedule template will be posted on the AA Department wiki space http://tsdaccountabilityaccreditation.wikispaces.com

20. TCAP Schedule  Students in the same grade, at the same school, must take the same session(s) at the same time  Only one grade/content can be checked out to a test proctor at a time  Test sessions must be administered in the order they appear within each test book  Make-ups administered for missed sessions  Session 1 Writing must happen before Session 2 Writing

21.  SAC must keep on file a schedule of exactly when and where each proctor is administering which assessments to which students  Schedule should allow time for proctors to read the directions from the Proctor’s Manual, to distribute materials before and to collect and secure materials after administration of each assessment  Considerations for accommodations should be included in the development of the schedule TCAP Schedule

22. In General, The Test Environment The testing environment must:  Be adequately lit, quiet, free of distractions, and heated or cooled  Provide an adequate writing surface (free of other materials)  Be free of cell phones, music or other distracting devices  “Do Not Disturb” and “No Cell Phones” signs must be placed on the door during test sessions  No food or drinks are allowed on desks or near test materials

23. The Test Environment CANNOT  Must be free of any content related posters or aids that suggest possible answers to students  Word walls  Steps for long division  Multiplication tables  Periodic Table of the Elements  Must be free of all reference materials that a reasonable person might conclude offers students an unfair advantage  ***Name tags – must be writing resistant (laminated or clear plastic tape) and must not include reference materials

24.  Posters that do not include content specific definitions, content related processes or solutions may remain on the wall  A picture of the Earth  The Alphabet  Basic Number Lines  May include study carrels IF they are part of normal classroom environment The Test Environment CAN

25.  Forbidden Devices: No one (including test proctor) is allowed to have cell phones or any other device with the capability to reproduce, transmit or record secure testing materials or information about secure testing materials in the testing environment.  Authorized Personnel Only: Parents, the media, and anyone except authorized school personnel are prohibited from being in the test environment.  Open Desks: Student desks which are open and contain student materials should be organized. All instruments not allowed to be used on TCAP should be made inaccessible. More on the Test Environment

26. Proctoring TCAP  Only one test session at one grade level and in one content area may be given in a testing environment  If a student finishes a test session early, they may read or sit quietly but must not write or do any other activity  All test materials must be removed before a student begins reading  All test proctor’s must meet the qualifications for proctoring as described in the Procedures Manual (pages 27-28)  All test proctors must undergo school or district training before administering the TCAP (proctor training is required annually)  Proctoring is active

27. Active Proctors SHOULD  Ensure that the testing environment complies with TCAP instructions  Ensure they have all necessary materials for each session  Follow all scripts exactly as written  Move throughout the room during testing  Encourage on-task behavior by using proximity

28. Active Proctors Should NOT  Interact with students in any way that would impact student responses (verbal, gestures, or in writing)  Provide feedback  Clarify test questions  Answer content related questions  Interfere with the students’ demonstration of skills  Engage in other tasks during test sessions  Read through test booklets or test items

29. Proctor Situations  Dealing with disruptive students  Disruptive students should be removed from the testing environment. If possible, they may continue the session in a designated area. The student’s test score may be invalidated if the session cannot be completed  Test Proctors should never leave testing students unsupervised while dealing with a disruptive student  Dealing with a concerning student response  On rare occasion, students make concerning responses to test items, such as responses that include information about threats, violence, abuse or other concerns  If discovered before returning test books, these items should be reported to school administration and to the SAC who in turn must inform the DAC

30. Unexpected Situations  If a student becomes ill during a test:  The student’s needs are the primary consideration  The student may make up the test  The student may not change answers in a session that was already begun  The student may only have the amount of time that was left when the test was suspended

31.  If there is an emergency requiring the test to be stopped:  Evaluate and respond to the emergency… safety first!  If it is possible to do so without any risk to students, note the time remaining in the test session  Students will be allowed to use the remainder of the testing time to complete the session at a later time  Notify DAC as soon as possible  Document situation in writing Unexpected Situations

32. Misadministrations  A misadministration is any event that leads to the invalidation of one or more student test scores in one or more given test sessions  DACs investigate, identify and declare misadministrations  Major Misadministrations must be reported to CDE  Also reported to Office of Licensing if it involves a teacher or school (or district) administrator

33. Misadministrations  Any deviation from established standardized testing procedures  A student moves on to a new test session or takes the wrong session  A student receives help from anyone on a test item  A student uses an unauthorized instrument, such as a calculator, cell phone or notes, during a test session  A student is denied appropriate accommodations or given the wrong accommodations

34. Major Misadministrations (three main types)  Misadministrations affecting an entire class or group of students  The wrong test session is administered to a class  Systematic unethical behavior  A teacher, administrator or other person gives students hints, prompts or answers to questions  Students obtain or share secure test materials  Breach of secure test materials  Discussing, reproducing, or transmitting, by any means, secure test materials, or descriptions of secure test materials

35.  All students in the same grade level must be given the same test session at the same time  The school is solely responsible for proper administrations  Prevention: Create schedules and emphasize communication  Only one test book is checked out at one time  Say, “Session 3 Reading”  Do not say, “Reading Session 3”  Verification  Call backs  Classroom checks  Corrective Action  Students must be kept separate  Corrective action must be taken by the end of the school day or a misadministration will be declared Administering the Wrong Session

36. When a Misadministration Occurs  The student must immediately stop working on that session, but should still complete other test sessions as normal  The Test Proctor must immediately inform the SAC, who must inform and consult with the DAC  The appropriate misadministration bubble should be filled in on the student data grid  The student receives a “no score” for the content area

37. District Policies for Misadministrations  Misadministrations are declared to preserve the reliability of test scores and assessment data  The declaration of a misadministration should not be used as a disciplinary measure  A record of misadministrations must be kept by the DAC  Reports or suspicions of major misadministration must be reported to CDE as soon as practicable  CDE reserves the option to investigate and declare misadministrations and apply its own disciplinary sanctions

38. Training

39.  SACs are responsible for training all staff involved in any aspect of TCAP administration  Training must include:  Test Security including Chain of Custody  Test Proctor Requirements  Ethics

40.  Required documentation  Training Participation Agreement  Confidentiality Agreement – all who come in contact with test materials  Certification due by March 6 th Training 47

41. Accommodations

42. Students with a documented need Standard Accommodations Students with an IEP or 504 Nonstandard Accommodations Students identified as English Learners Linguistic Accommodations 49

43. Planning for Reading/Writing Accommodations – 2014 sessions Session1Session 2Session 3Session 4Session 5Session 6 Grade 4 Writing Reading WritingR/W Grade 5 Writing Reading WritingR/W Grade 6 Writing Reading WritingR/W Grade 7 Writing Reading WritingR/W Grade 8 Writing Reading WritingR/W Grade 9 Writing Reading WritingR/W Grade 10 Writing Reading WritingR/W 51

44. Oral Script Small Groups < 15 students Must be requested by February 20

45. Teacher Read Directions  English or native Language  TCAP Reading, Writing, and Mathematics  Follow directions exactly  Separate environment  Individually or small group Grade 3 Reading due February 12 th All other must be requested by February 20

46. Oral or Written Student Response

47. Lectura/Escritura Grades 3 & 4  Must be administered in a separate environment by a Spanish- speaking test proctor  Administration instructions are provided in the English TCAP Test Proctor’s Manual  Scheduling and administration are the same as for the English assessments  Student Data Grids on all test books are identical – both for English and Lectura/Escritura  STOP pages – must be printed and inserted before testing and removed before shipping and scoring  Oral Scripts will be available for Escritura

48. Assistive Technology  Assistive Technology is a Standard Accommodation  iPads are allowed to be used on TCAP tests  All Assistive Technology programs must have grammar checking capability disabled for testing  Assistive Technology devises are considered secure testing materials and must be secured as such  All materials must be printed and transcribed into a scannable test booklet  All student testing materials must be removed from the device immediately following printing  A Verification of Removal of Saved Data is required for each school

49. Extended Time  Extended Time is NOT unlimited time. Extended time permits students 90 minutes in the testing environment  Test proctors can use their professional judgment and allow a brief extension past 90 minutes to allow students to finish their work (Accommodations Manual page 153)  Students in an extended time environment must remain in the test session, even if they complete the test within the standard time allotment  Only students who USE the extended time should have the Extended Time Used accommodation bubbled

50. Mark All Accommodations

51. Security

52. Test Security  Two main goals:  Protect sensitive student information and data  Ensure the validity of the state assessments

53. Materials – Secure – Not Reproducible  Test materials that may never be reproduced:  Test Booklets  Draft Booklets  Student Answers

54. Materials – Secure but Reproducible  The DAC may reproduce the following materials. They must be returned with “Not to be scored” materials.  Oral Scripts  Translated Oral Scripts  Teacher Read Directions  Translated Teacher Read Directions

55. Materials – Non-secure Materials  These materials are non-secure and do not require documentation or chain of custody  Procedures Manual  Accommodations Manual and Accommodations Guide  Test Proctor’s Manual / Test Examiner’s Manual  DAC/SAC Manual  Guide to Test Interpretation  CSAP/TCAP Released Items (Assessment Unit website)  Punch-out Tools (Not secure, may not be reproduced)

56. Chain of Custody  Well documented before, during, and after testing  Materials must be accounted for and stored in a secure and locked location before, during, and after testing  Materials delivered to schools no more than 1 week in advance  SAC responsible for distribution and collection of materials each day of administration  SAC must be able to assure the DAC that every step to ensure security has been taken

57. Chain of Custody Document  Test materials must be stored in a secure and locked location and must only be checked out to authorized individuals at the time of testing  Test materials should be tracked on the Chain of Custody document to ascertain who had what materials at what time (who, what, and when)  Chain of Custody document should be reviewed at end of each testing session to ensure that all test materials have been returned  Risk of major misadministration without proper documentation 65

58. Chain of Custody & Materials  All test books must be kept in a secure, designated area  Test books and other secure test materials must not be kept in classrooms  Proctors must check out materials each day and check them back in at the end of each testing day  Proctors should ensure they have enough test books for the scheduled sessions  When two content areas are tested on the same day, both sets of books must not be checked out to proctors at the same time

59. Basic Principles in Maintaining Security  Test books must be secure while in the TCAP Test Proctor’s possession  No cell phones or other communication or recording devices are allowed in the test environment  Test items should never be discussed – verbally or in writing

60. Questions about Items Proctors should not be studying items Items only accessed during test administration If a test proctor has a question about an item…  Contact the SAC, who will contact the DAC  The DAC contacts the appropriate person at CDE in the Office of Student Assessment by phone  Include in proctor training instructions:  Do Not copy the item  Do Not discuss the item (with other teachers, etc.)  Do Not email the content of the item

61. Housecleaning  Every SAC must sign the School Verification of Training and Housekeeping document:  ensuring that there are no assessments or proprietary materials related to the state assessments stored anywhere on the school premises (test booklets from previous years, etc)  Ensuring that any teacher-created materials (whether distributed or not) are free of actual assessment items  Does not apply to TCAP released items Due by March 6 th

62. Data Quality

63. Use only a standard, wooden, graphite-based, #2 pencil on any part of the test book…. including the front cover! Front Cover of Test Book

64.  Student Data Grid

65. Mark All Accommodations

66. Mark ONE Test Invalidation

67. Transcribing Books  When transcribing is NOT an accommodation  Damaged books  Alternate formats (Large Print and Braille)  Assistive technology used  When transcribing IS an accommodation  When a student responds in a language other than English (Math and Reading)  Must be done in a scannable test booklet

68. Communication and Resources

69. Office of Student Assessment School Assessment Coordinators CTB McGraw-Hill DAC and Core Team Test Proctors, Examiners and Administrators Communication

70. Assessment Resources  2013-2014 Procedures Manual  2013-2014 Accommodations Manual and Guide  2014 DAC/SAC Manual  2014 Test Proctor’s Manuals  Accountability & Accreditation wiki space  Ronda Hanson (613-5024)  Andy Riley (613-5019)

71. Assessment Binder  Signs  Key Points to cover in training  Copy of the letter sent to Superintendents regarding testing  School Verification of Training and Housekeeping  TCAP Security of Testing Materials  CoAlt Security of Testing Materials  TCAP Daily Accounting of Testing Materials  CoAlt Daily Accounting of Testing Materials  Proctor Training Participation Agreement  Confidentiality and Participation Agreement  2014 TCAP/ CoAlt Timeline

72. Assessment Binder (continued)  TCAP SAC Checklist  CoAlt SAC Checklist  TCAP Check-in Checklist  CoAlt Check-in Checklist  Oral Presentation & Teacher Read Directions Request  TCAP Parent Refusal  TCAP Not Tested/ Test Invalidation Form  CoAlt Not Tested/ Test Invalidation Form  Verification of Removal of Saved Data  2013-2014 Procedures Manual

73. Thank You!  Thanks for taking time to participate in this required training  Be sure that you have signed the record of attendance