California High School Exit Examination (CAHSEE) Test Administration Training for District Coordinators June 19, 2009-Sacramento June 26, 2009-Santa Clara June 30, 2009-WebCast July 22, 2009-Los Angeles August 5, 2009-Madera August 6, 2009-Monterey September 29, 2009-San Diego

2 Presenters Mao Vang, CAHSEE Project Manager, Educational Testing Service (ETS) Rebecca Rabin-Reed, Product Administrator for Customer Services, ETS Bonnie Galloway, Consultant, California Department of Education

3 Presentation Objectives What’s New Overview of CAHSEE Maintaining Test Security District Coordinator, Test Site Coordinator, Test Examiner, and Proctor Responsibilities Demographic Data Corrections Resources Questions and Answers

4 What’s New Answer document changes:  Page 1 Box 6 – grid “Yes” if Hispanic or Latino Box 7 – grid all race/ethnicity that are applicable “Decline to state” option is no longer available. New Federal Standards for Collecting and Reporting Race and Ethnicity Categories may be found at  Page 2 Position of some boxes changed, but required data and numbering remains the same.  Page 12 Box 24 – American Indian Education has been deleted.

5 What’s New (Continued) Box 13  Directions revised

6 What’s New (Continued) One set of manuals for entire year (July 2009–May 2010)  District and Test Site Coordinator’s Manual  Directions for Administration  Directions for Administration — Special Test Versions

7 Overview of CAHSEE

8 CAHSEE Purpose Improve student achievement in public high schools and ensure that students who graduate from public high schools can demonstrate competency in reading, writing, and mathematics

9 CAHSEE Content The CAHSEE has two parts:  English-language arts (ELA) State standards up to grade ten  Mathematics State standards in grades six, seven, and Algebra I

10 CAHSEE Requirement All students must satisfy the CAHSEE requirement, as well as all other state and local requirements, to receive a California public high school diploma. The CAHSEE requirement can be satisfied by passing the examination or, for students with disabilities, receiving a local waiver according to California Education Code Section 60851(c).

11 Options to Participate in CAHSEE Administrations July October 6-7 November 3-4 December 5 & 12 February 2-3 March May Grade 10 (Census) Grade 10 (Census or Make up) Grade 10 (Make up) Grade 11 Grade 12 Adult For testing schedules, go to:

12 CAHSEE Testing Opportunities Grade ten: one time (February, March, or May) Grade eleven: *two times Grade twelve: *at least three, but up to five times Adult students: *three times * May participate in successive administrations.

13 Test Day Test Session Approximate Working Time Approximate Total Administration Time Day 1 ELA 12 hours 2 hours 30 minutes 2 1 hour 30 minutes 2 hours Day 2 Math 1 1 hour 30 minutes 2 hours 2 1 hour 30 minutes 2 hours Approximate Testing Times

14 Maintaining Test Security

15 Maintaining Test Security To ensure test security, district coordinators must:  Read, agree, sign, and return the Test Security Agreement (TSA) to ETS  Ensure all test site coordinators read and sign the TSA and all who have access to secure test materials read, sign and agree to the Test Security Affidavit

16 Maintaining Test Security (Continued)

17 Maintaining Test Security (Continued)

18 Maintaining Test Security (Continued)  Keep materials in a secure, locked storage  Establish and maintain procedures that will prevent cheating

19 Maintaining Test Security (Continued)  Handle potential testing irregularities. Call ETS and CDE to report security breaches or test administration incidents  Cooperate with Global Compliance Services when they conduct audits Before-testing During-testing After-testing

20 District Coordinator Responsibilities DC

21 Before Testing Attend training Send signed Test Security Agreement to ETS Read the District and Test Site Coordinator’s Manual (DTSCM) DC

22 Before Testing (Continued) Go to to obtain information and documents regarding the coordination of CAHSEE administrations DC

23 Refer to CAHSEE Project Calendar Before Testing (Continued) DC

24 Before Testing (Continued) November 2009 CAHSEE Project Calendar Sample DC

25 Before Testing (Continued) Manuals are posted for reference. DC

26 Go to the secure CAHSEE Web site by clicking on CAHSEE Online from the ETS CAHSEE Web page or Before Testing (Continued) DC

27 To obtain a username and password for CAHSEE Online, Contact ETS CAHSEE Support at Before Testing (Continued) DC

28 Before Testing (Continued) Use CAHSEE Online at to do the following:  Update your district contact information  Order test materials  Order supplemental shipments  Submit a Pre-ID file  Review/update an order  Make student demographic corrections  Download student results, roster reports, and summary reports DC

29 Before Testing (Continued) Select the appropriate date option. Enter your Username and Password. Select the LOGIN button. DC

30 CAHSEE Online main page Before Testing (Continued) Update District Information and Testing Schedule DC

31 Schedule and set up test administrations Notify parents of district’s testing schedule Before Testing (Continued) DC

32 Before Testing (Continued) Standard Operational Test Books Large Print (Version 998) Audio Books (Version 001) Audio CD Braille (Version 999) LP-Braille (Version 999) LP- BR CD Prepare to order materials CAHSEE Test Books and Special Versions DC

Before Testing (Continued) Prevent excessive materials ordering “Excessive materials” is the difference between the sum of the number of tests scored and 80 percent of the tests ordered by the school district for the entire school year. The difference must be greater than 100 to be “excessive materials.” DC

Excessive Orders Examples Scenario #1 # Ordered (80%)20,00016,000 # Scored12,000 Difference4,000 Cost per test booklet ( ) $0.76 Excessive materials charge $3, Scenario #2 # Ordered (80%)10,0008,000 # Scored7,950 Difference50 No excessive materials charge (Difference is less than 100) Before Testing (Continued) DC

35 Order materials and select option to pre- ID Before Testing (Continued) DC

36 Materials order and option to pre-ID warnings Before Testing (Continued) DC

37 Materials order confirmation Before Testing (Continued) DC

38 Go to “Pre-ID services” tab to obtain file layout Before Testing (Continued) DC

39 Before Testing (Continued) Prepare data file according to pre-ID file layout Upload pre-ID file DC

40 Before Testing (Continued) If submitting Pre-ID file, fill out the List of Demographic Fields Required to be Hand- gridded for Schools Participating in the Pre- ID Service.  Provide the list as a handout for test site coordinators during training A Late pre-ID label service is available for a fee.  You must have submitted a pre-ID file by the deadline to use this additional service. DC

41 Before Testing (Continued) DC

42 Before Testing (Continued) Designate and train test site coordinators Emphasize the following in training:  Maintaining test security  Reading Test Site Coordinator Responsibilities Section of manual  Hand-gridding fields not Pre-IDed  Accommodations/Modifications/EL Variations Signature in Box 13  Checking in/out test materials  Providing handout of important dates DC

43 Before Testing (Continued) Identify and coordinate the testing of students placed in nonpublic schools Go to California School Directory to obtain NPS school code. Enter onto Box 12c. DC

44 Before Testing (Continued) Receive test materials (starting about 2 weeks before test administration)  Shipment 1: nonsecure testing materials  Shipment 2: secure testing materials  Shipment 3A: (optional) Pre-ID answer documents  Shipment 3B: (optional) late Pre-ID labels DC

45 Before Testing (Continued) Upon receipt of shipments, open boxes and conduct inventory DC

46 Before Testing (Continued) DC

47 If more test materials are needed, contact the CAHSEE Support Center or go to CAHSEE Online to place supplemental orders Distribute test materials to sites Before Testing (Continued) DC

48 During Testing Be the liaison between test sites, CDE, and ETS Problem solve Notify both CDE and ETS of potential testing irregularities Complete the Test Security Breach Report Form, if applicable DC

During Testing (Continued) DC

50 After Testing Receive and inventory test materials from sites Account for all test booklets and CDs Complete District Packing List and Security Tracking Ensure all answer documents are batched by grade with School and Grade Identification (SGID) Sheet and packed in scorable boxes DC

51 School and Grade Identification (SGID) Sheet PURPLE Grade 10 only ORANGE Grade 11 only BLUE Grade 12 only GREEN Adult education students only SAMPLE DC

52 Enter district scorable totals Enter district non-scorable totals After Testing (Continued) DC

53 Arrange with courier (FedEx or land carrier) for pickup After Testing (Continued) For Scorable materials For Non- scorable materials DC

54 After Testing (Continued) Confirm that the designated CAHSEE color- coded label is in the lower right-hand corner on the top of each box being returned. Place the appropriate pre-addressed FedEx label to the immediate left of the designated color-coded label. DC

55 After Testing (Continued) Late return of scorable materials may delay results and reports being delivered on time.  “Late” = shipped from district after the Wednesday following the test administration If scorable materials are received late, a late processing fee of $450 is charged to the district. DC

56 Refer to CAHSEE Project Calendar for all pertinent deadlines After Testing (Continued) DC

57 DC After Testing (Continued) Interpreting CAHSEE Scores  Interpretation Guides (Top 11 languages)  Score Conversion Tables

58 After Testing (Continued) Use CAHSEE Online to download student results

59 After Testing (Continued) Student detail files, rosters, and summary report DC

60 After Testing (Continued) Send student and parent reports home DC

61 After Testing (Continued) Send summary reports to schools (ELA Sample) DC

62 After Testing (Continued) Send summary reports to schools (Math sample) DC

63 After Testing (Continued) Send roster reports to schools DC

64 After Testing (Continued) Additional resources and special services form DC

65 Test Site Coordinator Responsibilities TSC

66 Before Testing Attend training Send signed Test Security Agreement and Affidavits to district coordinator Read the responsibilities and procedures in the “test site coordinator” section of the DTSCM TSC

67 Before Testing (Continued) Inform parents, students and teachers about testing schedule Inform teachers about their students’ attendance at the testing session Work with district coordinator to resolve problems TSC

68 Before Testing (Continued) Organize, implement, and supervise all activities necessary to conduct testing at school site Collect signed Test Security Affidavits from all who have access to test materials Control inventory.  Use Test Site Security Tracking Form and Inventory Form-Test Site TSC

69 Before Testing (Continued) TSC

70 Before Testing (Continued) TSC

71 Before Testing (Continued) Receive test materials Conduct an inventory of all materials within 24 hours of receipt Look at sequence numbers written on top of the boxes and make sure all boxes are accounted for  Box 1 of 10, 2 of 10, etc. TSC

72 Before Testing (Continued) Follow the checklist for Inventorying Test Materials at the Test Site Save original boxes for return shipments Contact District Coordinator with additional material requests TSC

73 Before Testing (Continued) Keep test materials secure (Test booklets, Audio CDs, Answer Documents, etc.) in sealed boxes in a locked storage at all times when not in use for test administration Test booklets are shrink-wrapped in sets. Do not remove the shrink-wrap until absolutely necessary. TSC

74 Before Testing (Continued) Train test examiners and proctors Include the following:  Responsibilities of test examiners and proctors  Security of test materials  Testing schedules and locations  Preparation of test locations for test administration  Arrangement for proctors, as required TSC

75 Before Testing (Continued) Train test examiners and proctors (continued)  Information about students who have accommodations/modifications in IEP or 504 plan Directions for Administration-Special Test Versions manual  Verification of student’s identity  Distribution of test materials  Assistance with directions  Administration of the test TSC

76 Before Testing (Continued) Train test examiners and proctors (continued)  Procedures for completing demographic information  Completion of score codes  Collection of and accounting for test materials  Scheduling and procedures for returning test materials to test site coordinator TSC

77 Before Testing (Continued) If district submitted a Pre-ID file, provide the handout from the district coordinator for test examiners during training. TSC

78 Before Testing (Continued) Prepare answer documents  One answer document must be submitted for each student testing  If using Pre-ID service, prepare blank answer documents for recently enrolled students who have not passed one or both sections  VOID documents for students who are no longer enrolled or who were incorrectly included in the Pre-ID file TSC

79 Before Testing (Continued) Prepare and distribute test materials using Inventory Form-Test Site Materials for Each Test Examiner Number to Include Directions for Administration manual1 per examiner Test booklets1 per student + 1 for examiner Answer documents1 per student + 1 blank No. 2 pencils1 per student + a few extras List of Demographic Fields Required to be Hand-Gridded for Schools Participating in the Pre-ID Service 1 copy, if applicable TSC

80 During Testing (Continued) Monitor test administration Visit each testing room to make sure administration scripts are being read word- for-word Ensure students have sufficient time to test Ensure proper monitoring of breaks and extended breaks taken during testing, such as lunch TSC

81 During Testing (Continued) Be prepared for emergency situations, restroom passes, etc. Document testing incidents and potential test security breaches  If applicable, complete Test Administration Incident Report Form and/or Test Security Breach Report Form. Ensure compliance with all examination procedures TSC

82 During Testing (Continued) TSC Administration Reminders  Scratch paper is not allowed.  Do not allow students to use extra paper to answer their ELA writing tasks.  All responses to math and ELA items, including writing task, must be recorded with No. 2 pencil on a single answer document.  Do not use a pre-ID answer document from one administration for another.

83 During Testing (Continued) Administration Reminders (Continued)  Do not use correction fluid on any answer documents.  Do not use paper clips, tape, staples, or glue on the answer documents.  Do not attach removable “self-stick” notes to the answer documents. TSC

84 During Testing (Continued) Administration Reminders (Continued)  Do not disassemble or pull pages out of the multi-page answer document or insert additional pages.  If an answer document is damaged, the answers must be transcribed onto a new one and all student demographic fields must be hand- gridded. TSC

85 During Testing (Continued) Provide accommodations and/or modifications to eligible students if specified in their IEP or 504 plan Provide identified English learners additional test variations if regularly used in the classroom TSC

86 After Testing Collect all examination materials after each test session Verify that all test materials from test examiners are accounted for Put the scorable answer documents in a secure place between test sessions TSC

87 After Testing (Continued) Prepare answer documents for return Inspect for stray marks If a student has a Pre-ID answer document, check to make sure the fields that were not Pre-ID are filled in. If a student is new or does not have a Pre-ID answer document, all demographic data must be hand- gridded. TSC

88 After Testing (Continued) Check answer documents Check that Box 3 Score Code is completed on every scorable answer document. TSC

89 After Testing (Continued) Check Box 13 Accommodations and Modifications  Test examiner must grid Box 13 Accommodations and Modifications codes if any were actually used.  Test site coordinator’s or designee’s signature is required to confirm that accommodations and/or modifications were offered whether used or not. TSC

90 If used, correct code must be hand-gridded. After Testing (Continued) TSC

91 After Testing (Continued) Transcribe student answers as required:  If any student marks answers anywhere other than in the answer document as a condition of his or her IEP or 504 plan, an authorized person must transfer the answers to the answer document exactly as produced by the student.  If a student uses more than one answer document, answers must be transcribed into a single document. Pages from two documents may not be physically combined. TSC

92 After Testing (Continued) VOID an answer document if:  The document has been damaged.  It is a Pre-ID document for a student who is no longer enrolled in the district.  A student has used more than one answer document and the answers have been transcribed onto another that has already been included in the scorable materials. TSC

93 After Testing (Continued) Prepare scorable answer documents for return  Locate the large plastic bag labeled “Test Site Coordinator Packet” that contains all your return instructions and labels.  Separate and count answer documents by grade.  Do not mix scorable and non-scorable materials. TSC

94 After Testing (Continued) Prepare scorable answer documents for return (Continued)  Use appropriate SGID for each grade Hand-grid section B: Grade, Number of Answer Documents, Date of Exam  Place SGID on top of appropriate grade level documents  Wrap each bundle with paper bands provided in groups of 100  Place BLUE Shipping Labels on boxes with scorable answer documents TSC

95 School and Grade Identification (SGID) Sheet PURPLE Grade 10 only ORANGE Grade 11 only BLUE Grade 12 only GREEN Adult education students only SAMPLE TSC

96 After Testing (Continued) Prepare nonscorable materials for return  Test Site Packing List and Security Tracking Forms  Unused SGIDs  Test Booklets, including large type and Braille  Audio CDs  All original copies of transcribed essays Place PINK shipping labels on boxes with non-scorable materials TSC

97 Enter school scorable totals Enter school non-scorable totals After Testing (Continued) TSC

98 After Testing (Continued) Return materials to district coordinator  Follow specific information from district coordinator and return materials accordingly and appropriately to the district office for consolidation of all test sites’ materials. TSC

99 Test Examiner Responsibilities TE

100 Before Testing Attend training Read and sign CAHSEE Security Affidavit Read the Directions for Administration (DFA) TE

101 Before Testing TE

102 Before Testing (Continued) Set up the testing room  Students face the same direction and have at least four feet from center of desk to center of desk Cover or erase any information that could clue answers for students Place a ‘Testing, Do Not Disturb’ sign outside the door TE

103 Before Testing (Continued) Check out test materials from test site coordinator TE

104 During Testing Verify identification of students (only students on the list that you received should be tested) Give pre-identified answer documents to the correct students TE

105 During Testing (Continued) Make sure the students use only No. 2 pencils, including for essay writing. Students should clear their desks.  Students are not allowed to use scratch paper or correction fluid.  Only No. 2 pencils may be used on the answer document.  Students should not have books, papers, cell phones, or any electronic devices on their desks. TE

106 During Testing (Continued) Hand out the test booklets in the order that you receive them. Follow and read the directions exactly as they are written in the DFA. Check that students are responding in the appropriate sections and are working independently. TE

107 During Testing (Continued) Make sure that each student:  Signs name in Box 1  Enters the version number and the barcode number from the back of the test booklet onto the answer document in each portion of the test. TE

108 During Testing (Continued) If Pre-ID service is used, ensure students hand-grid additional fields required, if applicable. TE

109 During Testing (Continued) Approximate Testing Times TE Test Day Test Session Approximate Working Time Approximate Total Administration Time Day 1 ELA 12 hours 2 hours 30 minutes 2 1 hour 30 minutes 2 hours Day 2 Math 1 1 hour 30 minutes 2 hours 2 1 hour 30 minutes 2 hours

During Testing (Continued) If some students need more time and have to go to another testing room, the test examiner should collect the test booklets and answer documents from the students. A proctor or other school staff should escort the students to the designated testing room. No students should leave a testing room with the test booklets and answer documents by themselves. TE

111 During Testing (Continued) Document incidents on the Test Administration Incident Form, if applicable TE

112 After Testing Account for all test booklets and answer documents  Separate the answer documents from the test booklets and make sure that no answer documents are inside test booklets  Separate answer documents by grade You must account for ALL testing materials. TE

113 After Testing (Continued) Code Box 13 & 18, if applicable. TE

114 After Testing (Continued) Code Box 3 (Score Code) for all students. TE

115 After Testing (Continued) Return the test materials following the procedure your test site coordinator has set up. TE

116 Proctor Responsibilities Attend training with the test site coordinator Read, agree to, sign, and return CAHSEE Test Security Affidavit to TSC Assist test examiner with the following:  Distribute materials  Monitor students during testing  Collect materials

117 Demographic Data Corrections

118 After Testing (Continued) Use CAHSEE Online for:  Making student demographic corrections DC

119 Go to the secure CAHSEE Online site at and select the CONTINUE button. DC

120 Select the database in which you are making Demographic Corrections for. Enter your Username and Password and then select the LOGIN button. DC

121 Select the drop down menu under the header “Demographic Corrections”. DC

122 Select the administration in which you need to make demographic corrections to and select the SUBMIT button. DC

123 There are several options to review your demographic data on this page. Select RETRIEVE MY DISTRICT’S PROFILE. DC

124 You can view your data in a summary format. Listed here are critical fields. DC

125 Below the Critical Fields are Optional Fields. Select one of the Optional Fields and you will see a summary of that data on your screen. DC

126 For example, choose Primary Ethnicity and select the SUBMIT button. DC

127 View the breakdown for Primary Ethnicity here. Select the Back Button to get to the Demographic Corrections page. Notice the numbers for African American and American Indian, we will be working with them later. DC

128 Option 1: To edit an individual student record, use any of the fields listed on the right to search for that student record. Then, select the SEARCH button. For example, the last name is used. DC

129 All students with that last name are presented. If you select the last name highlighted in blue on the left, you will be able to access this student’s demographic information. DC

130 You can change any demographic information listed on this page. You are not able to change anything in gray. DC

131 Example: change this student’s Primary Ethnicity (African American to American Indian) and see how that affects your data. Select the SUBMIT button and you will return to the page with the student’s names. DC

132 Select the Back to Search Page button and you will end up on the demographic corrections page. Repeat steps to retrieve your district profile and here is the change. DC

133 African American changed from 83 to 82 and American Indian changed from 10 to 11. DC

134 Option 2: Mass Student Demographic Update This requires a file in either text format or excel format. The layout is exactly the same as your Pre-ID file except there is an added field for Census/Make-up, field 601. DC

135 This PDF on the CAHSEE home page at shows all the fields for a text file as well as what fields can be updated. To create a mass demographic correction file, use your Pre-ID file and add filler fields from positions DC

136 Here is a sample of an excel demographic correction file. Notice only key matching fields are needed. When you are ready to upload your file, remove all the headers from the file. Your file will not upload with headers included. DC

137 Select the BROWSE button and select your demographic correction file from where you have saved it. Select Open. DC

138 Your file has now been selected and you can select the CONTINUE button to update your district’s demographic data. DC

139 Your file is processing. The larger the file the longer the processing time will be. Sometimes the files are too large and will need to be submitted as smaller files. DC

140 You are now able to see where the corrections have been made. You can either CONTINUE with the changes or CANCEL them and start over. DC

141 If you select CONTINUE, you will receive a confirmation page of your edits. At the bottom of the page is a Table of Errors; you will need to select CONTINUE here for the system to accept your edits. DC

142 Once the system accepts your edits, you are taken back to your home page and you are able to see the edits you have made by going back into your District Profile page. DC

143 Now that you have corrected your demographic data, you can see how it affects your District Reporting. Under the District Reporting section, select the CONTINUE button. DC

144 Let’s look at your data prior to making demographic corrections by selecting District Demographic Summary Report for ELA within grade 11. Select the SUBMIT button and you will see your results. DC

145 You can see that there are 5 American Indian students who were tested and 47 African American students tested. Now let’s look at this after demographic corrections are completed. DC

146 You have submitted your Mass Demographic Corrections file and you can now see that there are 6 American Indian and 46 African American students who have tested. DC

147 Final CAHSEE Data Corrections Window Deadline for final data corrections is October 30, 2009 View reports with corrections immediately Make corrections immediately after receiving results – don’t wait Districts are not charged for this service

148 Resources

149 Opportunities to Participate Scoring Opportunities  Item Writers’ Workshop  October 27-28, 2009 Reviewers  Bias & Sensitivity and Content reviews in March and April 2010  Differential Item Functioning (DIF) reviews after each census administration  Contact Usha Altieri or

150 Training Presentation Available Presentation materials posted at :   Look in the “About the Test” tab

151 CAHSEE Regulations California Code of Regulations, Title 5, sections 1200 to 1225 govern the administration of the CAHSEE in concert with state law (California Education Code sections to 60859). The CAHSEE regulations can be found at s/cahseeregs.doc s/cahseeregs.doc

152 Special Education and English Learner Resources Available at  Q & A about Test Variations  Request for Test Variations  Scribe Guidelines  Matrix 1. Matrix of Test Variations, Accommodations, and Modifications for Administration of California Statewide Assessments  Matrix 2. Matrix of Test Variations for Administration of California Statewide Assessments for English Learners

153 Student Testing General Guidance Available at Available at  CDE division updates and program notes  Test administration dates  Guidelines on Academic Preparation for State Assessments  Other guidance for California public schools

154 Contact Information CAHSEE policy and general examination information CDE CAHSEE Office State and federal accountability information CDE Academic Accountability Unit Test material ordering and logistics ETS CAHSEE Support