1. Colorado Measures of Academic Success (CMAS) Training for SAC’s

2. Agenda Pearson Access ◦ Online test management SAC Responsibilities ◦ Communication ◦ Scheduling ◦ Test Security Proctor Training Accommodations

3. Windows, grades, content areas Assessment Window April 14-May 2, 2014 Science grades 5 and 8 Social Studies grades 4 and 7

4. Frameworks The frameworks list ◦ Percent representation and number of score points for each of the reporting categories/standard areas that will appear on the summative assessments ◦ Evidence Outcomes eligible for inclusion  Topics explicitly identified in the Social Studies standards are legitimate content for the assessment in addition to the standards' concept and skills http://www.cde.state.co.us/assessment/newassess- sum http://www.cde.state.co.us/assessment/newassess- sum

5. Changes….. Online Assessment vs Paper Pencil Will be a shift for teachers, proctors, students, parents, administrators- communicate regularly Using Pearson Access to manage your testing Accessibility features vs Accommodations Learning tech readiness for you and your building More rigorous, higher DOK

6. Similarities Test security is important for valid and reliable scores Ethical administration and procedures and ensuring standardized conditions Scheduling will have some hurdles just as when we first scheduled TCAP Active Proctoring is a must

7. Each grade level and content area consists of three sections The assessments are composed of the following item types: ◦ Selected Response ◦ Constructed Response ◦ Computer-enabled/technology-enhanced ◦ Simulations (science) ◦ Performance Events (social studies) Test Structure

8. What are the Responsibilities of School Assessment Coordinators?

9. SAC Responsibilities include Communication Scheduling Test Security and Ethics Technology Pearson Access Proctor Training Online Test prep

10. ◦ Understand all CMAS procedures and requirements ◦ Employ school level security plan ◦ Serve as liaison between DACs and Test Administrators  Read all communications from DAC and distribute as appropriate  NEW for CMAS: during testing, school level personnel will have direct communication with Pearson Key SAC Responsibilities THROUGHOUT Testing

11. ◦ Participate in district trainings and review all manuals ◦ Working with STC, create the testing schedule for your school, including accommodated computer-based and paper-based testing groups and make-up sessions ◦ Determine school specific procedures (make-up testing, communication plan) ◦ Determine school level security plan ◦ Provide and ensure completion of all training by Test Administrators and anyone else you have authorized or expect to come in contact with secure materials or environments ◦ Receive, secure, and distribute all test materials ◦ Manage and return security and training compliance documents Key SAC Responsibilities BEFORE Testing

12. ◦ PearsonAccess  Set up user accounts for all necessary personnel  Identify and maintain a list of students requiring the text-to-speech form and accommodations (paper-based and computer-based administrations).  Verify that PearsonAccess accounts for students needing:  Paper forms of the assessment, including braille and large print  Oral scripts (paper and online)  Request from your DAC additional secure and non-secure materials as needed  Assign all students to a test session  Assign students requiring special forms for the computer-based assessment to the appropriate form  Register/add new students Key SAC Responsibilities BEFORE Testing-PearsonAccess

13. Establish the testing environments PROCTOR CACHE!!!-simply choose the correct proctor caching server from the down menu when creating test sessions in pearsonaccess Prepare students ◦ Meet with students to review school testing policies and expectations, including that test results should be reflective of individual effort. ◦ Ensure all students are familiar with TestNav 8.0— ePats! Key SAC Responsibilities BEFORE Testing

14. Ensure that all students are assessed in accordance with CMAS policies and procedures Request additional secure and non-secure materials from DAC as needed MONITOR TESTING ACTIVITY Distribute and collect paper testing materials daily Report suspected testing irregularities (misadministrations, security breaches, technology glitches). Work with DAC to investigate and declare misadministrations. Account for the assessment of all students and ensure that all students have completed testing. Administer make-up sections as needed. Key SAC Responsibilities DURING Testing

15. ◦ PearsonAccess:  Account for the assessment of all students and ensure that all students have completed testing  Verify all testing sessions have been completed  Verify that PearsonAccess has been updated to reflect the accommodations used during testing, including students taking paper-based assessments ◦ Transcribe student responses from large print and braille test books into scannable test books. ◦ Collect, sort, secure, and return all test materials to DAC by district determined date ◦ Manage, submit and retain compliance documents Key SAC Responsibilities AFTER Testing

16. SAC RESPONSIBILITIES: COMMUNICATION

17. Communication Lead contact for questions in their building Always available to school proctors during all testing sessions in case of tech or test administration question or concern Lead communication with building technology staff Be part of communication with building tech staff and district IT Building communication to DAC

18. Who to contact with questions Test administration ◦ Proctor to SAC ◦ SAC to DAC ◦ DAC to CDE ◦ Tech questions  Building tech to district IT-inform SAC of any tech issues so they are aware

19. Assessment Technology CDE

20. SAC RESPONSIBILITIES- SCHEDULING

21. Definitions PearsonAccess Test Session: Virtual data grouping of students who are grouped together within PearsonAccess for test management purposes. Each PearsonAccess test session consists of one grade level and one content area (three sections). Students must be placed in a scheduled PearsonAccess test session before they can log in to a test. Test Section: The parts of a CMAS: Science and Social Studies assessment. Each grade and content area of the CMAS: Science and Social Studies assessments include three sealed test sections. Physical Testing Group: Students who test within a physical testing environment at the same time.

22. Scheduling Create school wide testing schedule ◦ including accommodations and make ups Proctor scheduling: Suggest proctor and tech savvy person both be in room so if students are stuck with tech issues proctor isn’t stuck with one student

23. Each grade level and content area consists of 3 sections Test sections can only be administered to students in consecutive order One grade level and one content area at a time except for make-ups To the extent possible, all students within a grade should be assessed at the same time. Should that not prove possible, all students should be assessed within the shortest timeframe practicable (i.e., in back-to- back sections, within the same day or consecutive days if more sections are needed than can be administered in a day, etc.) CMAS Scheduling Considerations

24. Scheduling considerations cont… Technology ◦ Number of compatible devices ◦ Bandwidth ◦ Wireless access points ◦ How many concurrent testing sessions can you run?-use the site readiness tool ◦ Extra computers Drag and drop on laptop vs ipad vs desktop computer ◦ Fine motor considerations? ◦ External Mouse available? Vs touch pad

25. Section Timing 90 minutes per section ◦ Includes 10 min. for login, directions, sample items ◦ 80 minutes of actual test time for every student (required) ◦ 55 minutes minimum required 3 sections in Science 3 sections in Social Studies

26. CMAS: Test Structure and Timing CMAS: Science and Social Studies Section Administration Time for Elementary and Middle School Total Section Time Time for Set up and Transition- approximate ly-may take more or less Required Testing Time for All Students Required Additional Testing Time for all students 90 minutes10 minutes55 minutes25 minutes

27. Start up TestNav 8.0 Log in Test transfer (proctor cache…) Give directions Review sample items Time for Set up and Transition Includes….

28. Best Practices-Recommendations 2 extra devices in each testing room Minimum: Two proctors per each testing room No more than 2 test sections per day (this year) Do not start first thing in the morning- make up session Students remain in room for at least 55 minutes, up to 80 minutes if at least one student is still working

29. Schedule first group/first day with more of a time cushion than other groups if you can If you have to do multiple sessions, schedule smallest group first Plan for accommodations Recommendation: Schedule students for only one section per day (exception: make- ups) Remember: This is Colorado Recommendations

30. School CMAS schedules Email to Kim Terry no later than March 31 st Also send security of testing materials form

31. Priority: minimize risk of prior exposure to test content that could result in an invalidation Make-Up Testing

32. Student completes missed test section during make-up testing time in advance of re-joining original physical testing group Make-Up Testing

33. Sample Make-Up Testing Schedule Day 1Day 2Day 3 Regularly Scheduled Make-Up Testing Time (beginning of morning) -Absent Student Takes Section 1 - Physical Testing Group Testing Time (scheduled after initial morning make-up testing time) Physical testing group takes test section 1 Absent Student re-joins physical testing group for test section 2 Physical testing group takes test section 3

34. How do device and facility availability affect scheduling? What other impacts to scheduling do you see? How does make-up testing affect scheduling? ◦ How to schedule for students who miss mutiple days? Scheduling Discussion – Guiding Questions

35. CMAS or CoAlt CSAP-A decision not based on: ◦ Category of disability ◦ Time in special education ◦ Program placement Must be documented by IEP team Contact Barb Rice @ x21931 if you need clarification 35

36. SAC responsibility Test and materials security

37. Test Security Testing environment-screen security and number of computers How far apart to sit students Can students view from front to back the rows of screens in front of them? Student view side to side St authorization tickets Student pre exposure to items

38. Test security No cell phones in testing room ◦ Collect (baggies with names) so students do not have them during testing No student tablets allowed in testing room Ipads are allowed for TEST ADMINISTRATORS only ◦ No recording ◦ No pictures of anything in test environment ever

39. Security of Student Authorization tickets-one approach SAC print the day before the test Put all into an envelope labeled for that test ◦ Maybe with a class list on the outside of it, ◦ All used tickets will be collected by the proctor and put back in that envelope Have another empty envelope labeled “MAKE UP TESTING REQUIRED” ◦ Proctors put any unused authorization tickets

40. Chain of Custody Count in/out and test administrator sign for number of student authorization tickets each time Paper forms Oral scripts Scratch Paper (official CMAS scratch paper, not any other paper) Secure central storage location ◦ Not stored in classrooms

41. Maintaining Security of CMAS All secure test materials must be secured while in the Test Administrator’s possession No duplication of secure CMAS materials is permissible No cell phones or other communication, reproduction or recording devices are allowed in the classroom

42. Unauthorized Visitors and the Media Only students, Test Administrators/Examiners, and authorized school, district, state personnel or state- sanctioned test monitors may be in testing areas during administration Media are not allowed to have access to the tests before, during, or after test administration, or take pictures or video of testing materials or testing students

43. Accessibility Features and Accommodations Overview

44. TestNav 8 ◦ Embedded features: available to all students to increase accessibility ◦ Embedded accessibility features that must be requested ahead of time: available to all students to increase accessibility ◦ Additional embedded and non-embedded accommodations available to students consistent with their IEP, 504, or English learner plans Increased Accessibility

45. Tool Bar Navigation ◦ Back page ◦ Next page ◦ Review ◦ Flag Pointer Highlighter (Caution: in development) Ruler (cm in 5 th and 8 th grade, inch in all grades) ◦ Can also be used as line guide Calculator (4 function for 5 th grade, scientific for 8 th grade) Eliminate answer choice Embedded Features for All Students

46. Drop Down Magnifier Zoom Dependent on device Writing tools for constructed response Bold, italics, and underline Bullet and number Cut, copy, and paste Undo and redo Embedded Features for All Students

47. Selected by form 1.Text-to-speech with volume control  Continue to play  Click to hear – recommend students are taught to default to this 2.Color contrast (accommodation) 3.Text-to-speech with color contrast (accommodation) 4.Oral script (accommodation) Samplers for text-to-speech and text-to-speech with color contrast will be available in the second week of February Embedded Accessibility Features and Accommodations that Must Be Requested Ahead of Time for Computer-based Administration

48. Must be based on an individual need documented in the student’s approved IEP, 504, or English learner plan A master list of all students and their accommodations must be maintained by the SAC. These will need to be recorded in PearsonAccess. Please include students using text-to-speech. Accommodations are available for computer-based and paper-based administration. ◦ Paper-based form is itself intended to be an accommodated form Accommodations

49. Paper Pencil or Online? Paper-based Assessments: A paper-based form of the assessments will only be available for students who: Have a neurological disorder, a condition that causes seizures or another health condition that prevents the student from accessing the computer Need a braille form with tactile graphics IEP or 504 plan requires assistive technology, such as speech-to-text, that may not be compatible with the test engine. www.pearsononlinetesting.com/at www.pearsononlinetesting.com/at Will provide written responses in a language other than English or Spanish

50. Presentation ◦ English oral script used for:  On-site translations into languages other than Spanish*  On-site sign language presentation* ◦ Spanish oral script ◦ Color contrast, including high contrast and reverse contrast ◦ External cover overlays ◦ Word-to-word dictionaries (English/native language) ◦ Assistive technology devices – compatibility will need to be established *Due to the number, length and complexity of sources in the Social Studies assessment, it is recommended that this accommodation be provided individually when practicable. Accommodations for Computer- based Administration

51. Response ◦ Written responses in Spanish ◦ Scribe (must be individually administered)  English  Other languages ◦ Assistive technology devices – compatibility will need to be established Accommodations for Computer- based Administration

52. Setting ◦ Small group ◦ Individual Timing ◦ Extended time greater than time and a half ◦ Time of day ◦ Multiple breaks (requires separate setting) Accommodations for Computer- based Administration 52

53. Devices with demonstrated compatibility with TestNav8: www.pearsononlinetesting.com/at Assistive Technology

54. The paper-based form of the assessment is itself considered an accommodation. Use of the paper-based assessments is expected only to be available to students who: Have a neurological disorder, a condition that causes seizures, or another health condition that prevents the student from accessing the computer* Need a braille form with tactile graphics Have a IEP or 504 plan that requires assistive technology, such as speech-to- text, that may not be compatible with TestNav 8 Will provide written responses in a language other than English or Spanish *Students with disabilities that prevent them from utilizing devices safely may meet this criterion. Accommodations for Paper-based Administration

55. Presentation ◦ Braille with tactile graphics ◦ Large print (Reminder: computer-based has zoom and magnifier) ◦ External cover overlays ◦ English oral script (text-to-speech in computer-based)*  On-site translations into languages other than Spanish  On-site sign language presentation ◦ Spanish oral script (also computer-based)* ◦ Word-to-word dictionary (English/native language) *Due to the number, length and complexity of sources in the Social Studies assessment, it is recommended that this accommodation be provided individually when practicable. Accommodations for Paper-based Administration 55

56. Response ◦ Braille note-taker ◦ Scribe (also for computer-based)  Must be administered individually ◦ Speech-to-text ◦ Assistive technology devices (word prediction) Setting (same as computer-based) ◦ Small group ◦ Individual Timing (same as computer-based) ◦ Extended time greater than time and a half ◦ Time of day ◦ Multiple breaks (requires separate setting) Accommodations for Paper-based Administration

57. SCHOOL TECHNOLOGY RESPONSIBILITIES

58. School tech Java updates shortcut or LAN school set up using login given by IT that locks out other programs, puts test nav screen on every computer Site readiness tool so # of devices to run at one time is known http://systemcheck.co.testnav.com/systemCh eck.jsp?acc=co_chk http://systemcheck.co.testnav.com/systemCh eck.jsp?acc=co_chk Refer to district IT tech training for more information

59. System check http://systemcheck.co.testnav.com/ If you have the specifications for which browsers and versions work with which operating systems, etc you may not need this tool

60. Results from system check-Failed

61. System check-Passed

62. Without proctor caching- 26 students

63. With proctor caching

64. With proctor caching- 466 students

65. TEST ADMINISTRATOR TRAINING Administering the test

66. SAC Responsibilities Administration Training Read Procedures Manual in its entirety after today’s training and prior to training your building A face to face proctor training is required by you to your building proctors prior to the start of the testing window ◦ Use the Proctor Training ppt. provided, it includes ethics and test administration Keep all signed confidentiality agreements onsite for all of your trained CMAS proctors 66

67. Individual at school level responsible for directly administering the CMAS: Science and Social Studies assessments Must be employed by the school/district Be trained and have read the Test Administrator’s Manual prior to any assessment administration Be trained in any specific accommodations they are providing Must sign security agreement before assessment administration begins Read the directions to students exactly as they appear in the Test Administrator’s Manual Test session management (school decision) Administer sections actively CMAS Test Administrator

68. Active Administration Active Administrators: Administrators May Not: Provide feedback Clarify test questions Answer content related questions Interfere with the students’ demonstration of skills Interact with students in any way that would impact student responses Engage in other tasks during test sections Read sources, items, or student responses Use any electronics during testing (other than to monitor in pearson access if sac deems it necessary) Ensure they have all necessary materials for each session Ensure a standardized testing environment Follow all scripts exactly as written Move throughout the room during testing Read directions to students when asked Use proximity to keep students on task

69. Prepare the test environment Pick up student authorization tickets/sign out from SAC Make sure all students from list are present or marked absent on envelope with absent students tickets put into the absent envelope Start session in PearsonAccess-or SAC can do- school decision Follow directions and read script in the Test Administrator’s Manual Hand out student authorization tickets and scratch paper Assist students in logging on to TestNav Put Seal Code on white board or screen Administration Steps

70. Administration Steps-cont… Complete directions to students and practice items if there are practice items Actively administer the test section Collect student authorization tickets and scratch paper Erase Seal Code from board completely Close session in PearsonAccess after third section-SAC can do at end of test window because if any student absent/not marked complete will need to either take all sections or have the SAC “mark complete” in PearsonAccess

71. Test Administrators-helpful hints from field test Administrators should not hand out authorization tickets until students are seated in the lab Have students log in in waves Once students are logged in, proctors should direct them to put hands off of keyboard while directions are read Directions should be that students do not touch keyboard until proctor says so

72. Secure Materials- Test Administrators Collect back student authorization tickets immediately after students login Put student authorization tickets into correct envelopes Return envelopes to SAC’s immediately after testing Sign in the number of student authorization tickets being returned

73. Test Environment No food or drink on desks No content posted that would give students an advantage Put up “Do Not Disturb” sign No music Same rules as TCAP Bathroom-don’t log students out or pause their tests just put something (blank) in front of screen so no one can see it

74. Contact Pearson directly If a technology issue arises during an actual live test session test administrators should contact Pearson directly-after informing SAC of the issue This is the only time test administrators need to/should contact Pearson directly Tell them you are in a live testing situation and they will bump you to the front of the line ◦ Pearson: 1-888-687-4759

75. Testing Irregularities Fill out form online if any testing irregularity ◦ Misadministrations ◦ Security breaches ◦ Technology glitches ◦ Emergencies/unforeseen circumstances

76. Misadministrations Students moving onto next section ◦ Really only able to do if in a make up section and they have access to the seal code to the next section Student receives help from anyone on a test item Students uses a cell phone, calculator, electronic device Student is not given accommodations they require SAC: call Kim Terry if any misadministration has occurred

77. Breach of secure test materials Discussing, reproducing, transmitting by any means any secure test materials Student obtaining or sharing secure test materials Secure test materials include: ◦ Student authorization tickets ◦ Official CMAS test scratch paper ◦ Seal codes

78. Emergencies Student becomes ill ◦ Log that student out-directions on next screen ◦ Record amount of time left in section ◦ Record onto the absent student list that the student left and the amount of testing time left

79. To allow ill student to finish later

80. Emergencies Emergency requiring the test to be stopped: ◦ Safety first, evaluate and respond to the emergency ◦ If possible without risk to students, note time left in session ◦ Students can have time left in session

81. When students are done Students all remain in room until all students are done -recommended Students must be allowed to have 55 minutes minimum-required ◦ If all students are done before 55 minutes, all students must remain in room for 55 minutes and then can all be dismissed-required ◦ All students are allowed 80 minutes, so if any student is still working after 55 minutes, ALL students must remain in testing room until ALL students are done or 80 min. has passed- recommended

82. How students submit their tests They should choose that they are finished and want to submit their final answers This means they cannot go back into that section They can practice this step in the epats also

83. Technology issues

84. E PATS Practice Tests

85. Test preparation for online assessments ePATs ePat guides for proctors are so helpful Familiarize students with tools, give opportunity to use Practice with scrolling, drag and drop items, simulations

86. ePats: Practice Tests www.pearsonaccess.com/co

87. Receiving and Returning Test Materials

88. Online and Paper ◦ Test Administrator’s Manual ◦ Scratch Paper ◦ CMAS Secure Return Envelope ◦ School Security Checklist Online Only ◦ Student Authorization Tickets ◦ English and Spanish Oral Scripts for Online Form ◦ Onsite Translations into Languages other than English/Spanish Paper Only ◦ Test Books, including Braille and Large Print ◦ Source Book (Social Studies Only) ◦ English and Spanish Oral Scripts for Paper Form ◦ Onsite Translations into Languages other than English/Spanish Test Materials

89. Receiving Test Materials Materials will arrive via district warehouse-date tbd ◦ Purple labels on all boxes ◦ Green labels on CMAS boxes ◦ Pink labels on CoAlt boxes

90. Receiving Test Materials ◦SAC to verify materials received with the School Packing List

91. SACs Collect all secure materials and group as follows:  Secure return envelopes  Scorable test books  Unused test books  Accommodated materials  Electronic materials must be removed from any devices after testing and verification sent to the DAC Prepare materials for packaging  Check and deliver CMAS scorable test books to the DAC  Check and deliver CMAS non-scorable materials to the DAC  Return materials with original boxes Returning Test Materials- SACs

92. Questions