1. Test Monitor Training – What’s Working in the Districts? MDE Assessment Conference August 2010 Presented by Mary Roden Mary.roden@moundsviewschools.org

2. Who do we represent? Brand New DAC/SAC? DAC/SAC with 1-3 experience? DAC/SAC with 4+ years of experience? Other representatives?

3. Answering the 4 W’s Why provide Test Security/Test Monitor Training? Who are monitors, Who should be trained, & Who conducts the training? When should training take place? What should be included in the Test Security/Test Monitor training?

4. Purpose of Test Monitor Training To provide an overview of test security and administration procedures required by Minnesota Department of Education in order to reduce the number of security and administration problems during testing.

5. Test Security: Why Is It Needed? Maintain the integrity of the: –Tests –Test results Support fair comparisons among districts Protect the state’s financial investment –Test items are expensive

6. Test Security: Typical Problems Test administration –Failure to follow the Test Monitor Directions script (e.g., providing assistance to students). –Failure to follow test manual and district guidelines for taking restroom or water breaks during the testing session. –Inappropriate materials posted on walls. –Failure to monitor student progress which results in students moving on to a segment not to be tested that day. Missing materials –Unused secure materials not returned to PEM. –A test book removed from the testing session by staff or a student.

7. Who can be a test monitor? Test monitors should be selected from the highest possible ranking from this list. If a school has exhausted the availability of persons who match “a,” it should select staff who match “b,” and so on: A. Licensed teachers or administrators who work in the school. B. Licensed teachers or administrators who work in the district. C. Paraprofessionals who work in the school. D. School district personnel employed by the school district. E. Licensed substitute teachers who are employed by the district for the purpose of proctoring the test. Test monitors who are not employees of the school district cannot be the sole test monitor in a classroom or small group setting.

8. Who should be trained? All test monitors should be trained. –Recommendation: Train staff that could be a test monitor be in the event a test monitor is absent –Have staff sign roster to indicate they have attended training –Training may look different depending on the test being administered- MCA-II Reading and Math, MTELL, TEAE, MCA-II Science District Assessment Coordinator School Assessment Coordinator Who should train test monitors?

9. When should Test Monitors be trained Train Test monitors annually Train Test monitors as close to the test administration dates as possible

10. Test Monitor Responsibilities The Test Monitor is responsible for the security of the assigned testing materials until they are returned to the School Assessment Coordinator. The Test Monitor also ensures that students follow the directions during the test administration.

11. Minnesota Assessments Monitoring Checklist Beginning of 2010 Presentation This spring, MDE will again visit schools to monitor test administration. These site visits are part of an overall strategy to improve the administration of Minnesota’s assessments. What MDE learns from these visits will help districts construct uniform testing conditions across the state. An MDE staff person will arrive unannounced at the school office around 7:00 a.m. and request to see the School Assessment Coordinator. They will stay on site until testing is over for the day and all test materials have been collected from the test monitors. The MDE staff person is there only to observe, not to participate in the administration of the test. Note the checklist MDE staff will use at a site visit

12. Possible Consequences of Violations MDE may: Send a copy of its decision to the Minnesota Department of Administration – Data Privacy Violations. Invalidate a test score – student, class, school or district. Withhold state educational funding.

13. Possible Consequences of Violations MDE may require district to: Develop or amend its test administration/security plan. Clarify roles and responsibilities of staff; establish better tracking mechanisms. Order training of staff or send memo to staff. Refer the matter to the Board of Teaching for possible teacher licensure sanctions.

14. Objectives for Test Monitor Training Test Monitors will… Read, understand, ask questions to clarify, then sign the Non-Disclosure Agreement. Know the testing schedule and where to pick up/return testing materials on the day of the test. Review the Test Monitor Directions. Know how to contact their School Assessment Coordinator during testing. Know testing procedures for common occurrences

15. Why Is Test Security Needed? Maintain the integrity of: –Tests –Test results Protect the state’s financial investment –Test items are expensive to create and maintain

16. Possible Test Security Issues Testing Personnel –Inadequate training of Test Monitors or School Assessment Coordinators. –Failure to read/understand/sign the Non- Disclosure Agreement. –Inadequate staffing of testing rooms or computer labs (student-Test Monitor ratio should be, at most, 30:1, if possible).

17. Possible Test Security Issues Test administration –Failure to follow the Test Monitor Directions or script instructions. –Failure to follow district guidelines for taking breaks during the testing session. –Cheating by a staff member or student. –Inappropriate materials posted on walls.

18. Possible Test Security Issues Missing materials –Test material left unattended. –Secure materials not returned. Especially accommodated materials, such as scripts and CDs (proportionally, the test material most likely to be missing).

19. Non-Disclosure Agreement Anyone involved with the packaging, security, or administration of statewide tests must complete a Non- Disclosure Agreement. Only authorized persons are permitted to view or listen to the test items. –An adult facilitating accommodations for students Reading the Math Script, Scribing answers for students, Transferring answers for students, Monitoring students listening to Math CD Reproducing any part of a test is a breach of security and a violation of copyright law, except as otherwise permitted.

20. Testing Calendar Tuesday, April 24 –Reading Segment 1 & 2 Wednesday, April 25 –Reading Segment 3 & 4 Tuesday, May 1 –Mathematics Segment 1 & 2 Wednesday, May 2 –Mathematics Segment 3 & 4

21. Most districts tested reading first Most districts tested in a 2 x 2 pattern By Middle School 1/5 of districts gave the reading test to their students on a single day.

22. Where to Pick Up Testing Materials Day of Test Test Monitors will need to pick up and return materials each test day Location: Time: – At no time are testing materials left unattended ATTENTION!

23. Test Monitor Directions-Materials Allowed Only materials authorized for use during assessments are on students’ desks.

24. Test Monitor Directions-Check with School Assessment Coordinator Know school schedule –Day 1: Segments 1 &2 –Day 2: Segments 3 &4 Teachers to return test to Office Conference Room at end of testing each day.

25. Test Monitor Directions-Test Book Assignment

26. Another option for a Test Security Checklist

28. Comments/Information on Security Checklist Document students who are were not tested, have not finished testing, had administration problems Notify School Assessment Coordinator of any testing irregularities as soon as possible

29. Test Monitor Directions- Reading to Students

30. Test Monitor Directions- Distribute Testing Materials Distribute Test/Answer Books in one: 3 rd grade only Distribute Answer Documents: They will contain pre-printed student information or have student information handwritten in. Distribute Test Books: Check security number to make sure each student received the materials as assigned on the Materials Security Checklist

31. Test Monitor Directions- Verify Students Data

32. Adhere a sticky note to any answer document that has inaccurate student demographic information. Write the error on the sticky note. Important

33. Test Monitor Directions- Student Directions No Constructed Response in 2011

34. Test Monitor Directions- Print Name & Form Number- Gr6-8 Note to monitors who are reading a math script or having a student listen to a math test on CD: The student MUST have the correct form: 1, 91, 92, 93 that matches accommodation

35. Test Monitor Directions- Students Test Booklet-Gr. 7 Reading Directions Specific for students taking the MCA-Modified Field Test

36. Test Monitor Directions- Students Test Booklet-Gr.8 Mathematics Encourage use of book as scratch paper and use of the grid paper. Directions Specific for students taking the MCA-Modified Field Test

37. Formula Sheets: Tear off p.3

38. Test Monitor Directions- Calculator Use Calculator Segments on Mathematics Assessments Table 18 shows that the segments where calculators are not permitted typically come early on the Mathematics MCA-II. Be very careful to follow this schedule. A student who does not follow directions and uses a calculator during a non-calculator segment will have their results invalidated. The MTELL online assessment also has sections where a calculator is not permitted. The sections that allow calculators will have an online calculator tool available, or the student can use a regular calculator.

39. Test Monitor Directions- Students Test Booklet- Gr.11 Mathematics Students may use a four-function calculator, scientific calculator or graphing calculator (e.g., the TI 83 and TI 84 are acceptable). Encourage use of book as scratch paper and use of the grid paper.

40. Formula Sheets: Tear off p.3-4

41. Test Monitor Directions- Multiple Segments Remember: Day 1: Segments 1 & 2 Day 2: Segments 3 & 4

42. What Can a Monitor Say if… What can a monitor say if she/he notices that a student has overlooked a few questions in the test book? In order to have standard testing conditions, the only directions teachers can give students are those in the Test Monitor Directions (TMD). For example, these two sentences appear in the TMD for the Grade 10 Reading MCA-II: “Be sure to check your answers when you finish this segment of the test” and “Try to answer all of the questions.” Information from a DAC Update

43. Test Monitor Directions- Page Numbers Grade 6 Reading Grade 8 Math Calculators used for Segments 2-4

44. Grade 10-11:Test Monitor Directions- Monitor Information

45. Test Monitor Directions- Student Breaks All students must take the break. (Some students cannot continue to work, while others take the break.) Students are not allowed to visit with each other.

46. Test Monitor Directions- Ending Times Students must be allowed to work at own pace... allowed to finish each part of the assessment without being pressured to finish Know the building procedure for students who will need to continue working on the test when most students are finished.

47. Test Monitor Directions-Ending Testing/Security

48. Test Monitor: Contacting School Coordinator In the event of an EMERGENCY, QUESTIONS, or IRREGULARITIES contact: –The Main Office

49. Test Monitor: Procedures Know building/district procedures for – Restroom breaks during testing: Only 1 student at a time, must have escort, test books and answer books collected from student and redistributed to student upon return, no test materials are taken from the room – Disruption due to inclement weather and/or fire drills: Students should be instructed to turn their test books and answer books face down and exit appropriately. Students taking a computer- delivered test should leave the room and the Test Monitor should lock the door. Students must not discuss test items and answers during their absence from the testing location

50. Test Monitor: Procedures Know building/district procedures for – Student illness: Any students who must temporarily leave a testing session must be accompanied by student personal. If a student is unable to return the segment may not be completed at a later date. The segment must be sealed at the end of the testing day. – Remember: Students who leave room for extended time (lunch hour, remainder of day due to illness, etc.) not allowed to resume segment or section

51. Test Monitor: Procedures Know building/district procedures for  Student discovers a printing error in a test/answer booklet If an answer book is defective, get another one, make sure that the defect is not also in it, and then give the new one to the student. The Test Monitor should complete the “Student Information” section on the new answer book and the student should transfer any responses to the new answer book. Write “Defective” on the defective test material and return it to the District or School Assessment Coordinator for return to Pearson. (Test Monitors should not throw out any test books, answer books, scripts or other secure test material.) If a test book is found to be defective, get another test book with the same form number, make sure that the defect is not also in that one, and give the new one to the student.19 Direct the student to write their name on the new test book. The student may continue working with the same answer book. Write the security serial number of the defective and new test books on the Security Checklist. Print “Defective Material” on the cover of the test book and include it with the other non-scorable secure materials being returned to Pearson.

52. Test Monitor: Procedures Know building/district procedures for Student Points out a test item that appears to be in error:  the Test Monitor should instruct the student(s) to continue the test, and note the grade, subject, form number and item number. Following the testing session the Test Monitor should notify the School Assessment Coordinator who will notify the District Assessment Coordinator. The District Assessment Coordinator will contact MDE and give the grade, subject, form number and item number in question. The content of the item itself should never be referenced, copied or emailed.

53. Test Monitor: Procedures Know building/district procedures for If a Test Monitor discovers that a student has moved on to another segment without proper instructions If a Test Monitor discovers that a student has looked at or responded to four or fewer questions in the next segment, the Test Monitor must immediately stop the student and take the test book from the student. The student will be allowed to finish the remaining portion of the partially completed segment when all other students are scheduled to complete that segment, but the student may not change the answers to questions already attempted. If a Test Monitor discovers that a student has responded to more than four questions in that segment, the Test Monitor must immediately stop the student and take the test book from the student. The student may not make up that segment but may work on other segments that have not been attempted when the Test Monitor introduces them to the rest of the class.

54. Test Monitor: Posters in Classrooms Any reference material that a “naïve observer” would likely think gives student taking the test in that classroom an unfair advantage over students taking the test in other classrooms needs to be removed or covered during test administration. Ask yourself, “What would I remove if I were giving a unit test in these subject areas?” Posters that Could Remain: Mathematics: Posters that state the steps that are followed in the problem solving process: e.g., read and understand the problem, plan what to do, answer the question in a complete sentence, be sure to answer all parts of the question. Reading: Posters that state the steps that are followed in answering an open- ended question e.g., read and understand the question, re-read the parts of the story or text that you need to answer the questions, answer the question in a complete sentence, be sure to answer all parts of the question. Writing: A poster with the following process for writing good compositions would not be considered a reference: a good written composition may be composed of five paragraphs with the first being an introductory paragraph and the last a concluding paragraph, with each paragraph containing multiple sentences that have subjects and verbs. Posters that Could Not: Reference materials such as wall charts that contain multiplication tables, fraction equivalents, or place values need to be removed or covered during test administration.

55. 3 Main Parts of an Essay BEGINNING: Topic Sentence Main Idea Answer the Prompt MIDDLE: Specific details Logical sequence Use transitions END: Conclusion Wrap up Restate

56. a student’s precode label (sticker) is incorrect? or a student does not have a label? Complete a new answer form by bubbling in information with a No. 2 pencil. DO NOT throw away incorrect Answer Document – return form to BTC a student doesn’t understand a test question? Test monitors may not help students with test questions that further inform the student (ie. You may define a word from the directions, but may not define a word in one of the questions) a student writes an answer in the incorrect place? He/she should be instructed to correct such situations themselves. If there is too much to be erased and corrected in the Test/Answer Book, the student can rewrite answers on a new scannable Test/Answer Book. Return BOTH copies to the Building Test Coordinator. a test/Answer Book is damaged? Assign the student a new test/Answer Book. Return BOTH Test/Answer Books to the Building Test Coordinator. a student is caught cheating? The test MUST be invalidated by the Building Test Coordinator a student leaves during a testing session and is unable to return? He/she may NOT complete the testing session at a later date a student temporarily leaves a testing session (e.g. emergency)? He/she must be supervised while in absence. Testing materials and answer documents are not to be taken from the room a student has an injury that prohibits him/her from completing the test by hand? A scribe may be assigned. Scribe must write exactly what the students says (ie. Cat – c-a-t). a student finishes early? All students are to remain seated at the testing site until EVERYONE is finished. Talking, playing cards, passing notes, …. are NOT allowed. a fire alarm sounds? Maintain the integrity and security of the tests to the best of your ability – close test booklets, lock classroom door, students should not be talking about test. RECORD ANY TESTING INCIDENTS on the TEST ADMINISTRATION REPORT

57. Activities Prohibited During Testing Posting visual aids that directly relate to test content. Revealing test items (from sections of the test not yet taken). Reading test items to students (unless using a math or science script or administering the MTAS) Copying or emailing test items. Helping a student answer test items. Changing a student’s answers. Defining terms used in test items.

58. Bottom Line When testing is over, a Test Monitor should be able to say “Yes” to: –Were the test materials secure at all times, and did I return all the secure materials? –Did I carefully read and follow all of the Test Monitor Directions? –Did my students arrive at their answers independently?

59. Make Up Testing Make Up testing continues until May 1 or until all students have been tested. New students to the school will need to take the tests if they did not take them in their previous school.

60. Good Idea: Test Bin or Bag (Optional-color coordinated) Include all materials for administering the test - i.e. Materials Security Checklist, Test Books, Answer Books, Test Monitor Directions, Red Seals, #2 pencils with erasers, Do Not Disturb signs, Sticky Notes, Calculators, Include a short list of Administration & Checklist Reminders - “What If’s?” handout - Return test materials in alphabetical order to the conference room. Please separate the answer documents from the test booklets. - Verify materials against the Security checklist. - Please sign the Materials Security Checklist/Test Administration Report

61. Test of Emerging Academic English (TEAE) Purpose: designed to assess the progressing acquisition of academic English by limited English proficient (LEP) students. Test results will aid teachers in determining whether or not students have a sufficient level of academic English to participate successfully in mainstream instructional settings. LEP-identified students Grades K-2 –Reading and Writing Checklist –Standards-based teacher rating of student skills LEP-identified students Grades 3-12 –Reading and Writing Language Proficiency assessment –Grade Clusters: 3-4, 5-6, 7-12 –Timed tests (must have a clock, timer, or watch) Results used to for Annual Measurable Achievement Objectives Approximately 5 minutes for pre writing

62. TEAE Test Monitor Directions Directions are secure materials Directions may not be opened prior to testing –They contain pictures of pages in the Test Book Directions must be followed accurately. Scripts must be read as written. –Standardization of testing Directions may not be copied –Contact DAC for additional materials

63. Sample Tests Use Sample Test Manual from MDE website if unfamiliar with Procedures and Structure http://education.state.mn.us/MDE/Account ability_Programs/Assessment_and_Testi ng/Assessments/ELL_Tests/ELL_Manua ls_Directions/index.html

64. Materials Needed Test Administration Manual Number 2 Pencil (no mechanical pencils) Highlighters One Test booklet for each student and one for each Test Monitor One Answer booklet for each student and one for each Test Monitor Clock, watch, or timer File folders to act as barriers so students cannot observe each other’s work Red Dot Seals

65. Monitoring Procedures Students are not allowed to use dictionaries, computers, or similar aids Monitor should walk around the room during the test and remain attentive to the students testing Monitors may answer questions with respect to –marking answers, –clarifying directions, and –finding the right place in the test booklet If students have problems understanding directions, the parts they do not understand may be explained in English after the test begins It is NOT appropriate for a monitor to give clues or explain rationales that will help students identify correct answers It is NOT appropriate for a monitor to look at the test questions

66. Online Assessments require Collaboration Teacher/ Para as Test Monitor –Knowledgeable in classroom discipline –Has high expectations for each student –Understands the functionality of the test engine Technical Support –Knowledgeable in the use of the technology Test Coordinator/ ELL Coordinator as –Coordinates schedule that encourages the best student results –Assigns students to sessions Student is familiar with –How the test engine functions –Types of questions they will be asked to answer Parents/Guardians have been informed –Specifics of the Testing: Who, What, Why, When, How, Where

67. District Access To Item Samplers Necessary for monitors to view tutorials and Item Samplers in order to be effective as a monitor

68. Accommodations for MTELL and Science Test Large print (18 and 24 pt) and Braille Script (for an adult to read aloud test to students) For both these assessments, Audio has now been built in that allows ALL students to hear the scenario, the question, and the responses. Audio Accommodation for MTELL & Science (describes the charts and graphs)

69. Test Monitor Directions

72. Do not Click Stop!

73. Test Monitor Directions Power Point –Used to project and read aloud to students before they take the assessment –Found at this web page: http://education.state.mn.us/MDE/Accountability_Pr ograms/Assessment_and_Testing/Assessments/EL L_Tests/ELL_Manuals_Directions/index.html http://education.state.mn.us/MDE/Accountability_Pr ograms/Assessment_and_Testing/Assessments/EL L_Tests/ELL_Manuals_Directions/index.html –Can access online or download to computer connected to projector

74. Test Monitor Directions

76. Questions? Presenter Contact Information Mary Roden Coordinator of Assessment/Evaluation Mounds View Public School Mary.roden@moundsviewschools.org 651-621-6042