Office of Accountability Research, Evaluation & Assessment School Assessment Coordinator Meeting February 9, 10 and 11, 2015 The Minnesota Comprehensive Assessments (MCA) and Minnesota Test of Academic Skills (MTAS) March 9 – May 15, 2015

Office of Accountability Research, Evaluation & Assessment Pearson Online Assessment System Spring 2015 Assessments Test Administration Roles and Responsibilities for Testing Preparing for MCAs Test Results Test Security Sections: 4 1

Office of Accountability Research, Evaluation & Assessment Testing Windows Purpose for Minnesota Assessments Reading, Mathematics and Science MCAs MTAS Reading, Mathematics and Science Title 1 MCA Accountability Windows Dates for PearsonAccess Tasks 3 1 Spring 2015 Assessments

Office of Accountability Research, Evaluation & Assessment MCA Reading – online 3/9 – 5/8/2015 MCA Math – online 3/9 – 5/8/2015 MCA Science – online 3/9 – 5/15/2015 MTAS Reading – paper pencil 3/9 – 5/8/2015 MTAS Math – paper pencil 3/9 – 5/8/2015 MTAS Science – paper pencil 3/9 – 5/8/2015 MCA Modified tests have been discontinued. 4 Testing Windows

Office of Accountability Research, Evaluation & Assessment 5 Purpose for Minnesota Assessments The Minnesota Assessments program has three purposes: 1.To measure achievement towards meeting the Minnesota Academic Standards and to measure progress towards meeting Minnesota’s standards for English language development 2.To provide information for Minnesota graduates related to career and college readiness 3.To measure the academic progress of students over time MDE Procedures Manual 2014 – 2015

Office of Accountability Research, Evaluation & Assessment 6 Reading MCA Grades 3–8 and 10 Reading MCA Available only in the online administration mode Paper accommodated test materials are available for eligible students On-line entry system via Pearson, Data Entry form in Data Entry Session ‘DES’

Office of Accountability Research, Evaluation & Assessment 7 Mathematics MCA Grades 3–8 Mathematics MCA Grades 3–8 Mathematics MCAs are adaptive tests Available only in the online administration mode Paper accommodated test materials are available for eligible students On-line entry system via Pearson, Data Entry form in Data Entry Session ‘DES’ Access to an online calculator when allowed Formula sheets are available for students in grades 5–8 o REA web site: rea.spps.org>minnesota_comprehensive_assessment o PearsonAccess:

Office of Accountability Research, Evaluation & Assessment 8 Mathematics MCA Grades 11 Mathematics MCA Grades 11 Mathematics MCA is a fixed-form test Available only in the online administration mode Paper accommodated test materials are available for eligible students On-line entry system via Pearson, Data Entry form in Data Entry Session ‘DES’ Access to an online calculator or handheld calculator for the entire test as well as an online formula sheet o REA web site: rea.spps.org>minnesota_comprehensive_assessment o PearsonAccess:

Office of Accountability Research, Evaluation & Assessment 9 Science MCA Grades 5, 8 and High School Science MCA Available only in the online administration mode Paper accommodated test materials are available for eligible students On-line entry system via Pearson, Data Entry form in Data Entry Session ‘DES’ High School Science MCA Page 10 in your 2014 –2015 MCA and MTAS Assessment Manual Only for students taking Biology

Office of Accountability Research, Evaluation & Assessment 10 MTAS Minnesota Test of Academic Skills MTAS Reading, Math and Science Alternate assessment based on alternate achievement standards for students with the most significant cognitive disabilities Special Ed. teachers administer, required training via Special Ed. On-line entry system via Pearson, Data Entry form in by Special Ed. Data Entry Session ‘DES’ SAC receives and inventories secure materials, hand-off to Special Ed. SAC receives materials back from Special Ed., secure return to Pearson

Office of Accountability Research, Evaluation & Assessment 11 Title 1 MCA Accountability Windows Accountability windows are different than testing windows AYP Participation windows: Reading and Mathematics April 14 – April 25 Science MCA and Science MTAS March 17 – May 9 AYP Proficiency windows: Students enrolled on October 1 and continues to be enrolled during the accountability window

Office of Accountability Research, Evaluation & Assessment 12 Dates for PearsonAccess Tasks MCA Reading and Mathematics ˗ Testing Dates: March 9–May 8 ˗ Create Test Sessions: TBD ˗ Proctor Cache Tests: TBD MCA Science ˗ Testing Dates: March 9–May 15 ˗ Create Test Sessions: TBD ˗ Proctor Cache Tests: TBD

Office of Accountability Research, Evaluation & Assessment Preparing Schools for Testing Available Resources Test Administration Times for Reading, Mathematics and Science Test Administration with Paper Accommodated Test Materials MCA Valid Score Rules New to Country EL 13 2 Test Administration

Office of Accountability Research, Evaluation & Assessment 14 Choice of Testing Rooms The best location is typically a computer lab. A regular classroom with laptop computers will also work, MUST BE WIRED. Computer monitors or laptop screens should be adequately spaced. A computer mouse must be provided. Headphones (mathematics and science only) must be provided. Because of the need for security, only school owned computers can be used for online tests; student-owned computers are not permitted. All instructional posters and graphics in the computer lab or where students will be taking the MCAs must be covered or removed. Freedom from Distractions Only people involved in taking or administering the test are allowed in the room. The location must be a closed room that is being used only for testing, not a public venue. Take appropriate actions to reduce noise such as limiting the use of alarms (not fire alarms), bells and announcements. Talk to your principal. The doors of each testing room are to be closed during the test. Post signs on the doors reading “Testing—Do Not Disturb.” Do not allow visitors to enter the room while testing is in progress. Seating Arrangements Separate monitors as much as possible. Spread students around the computer lab to the extent possible. Consider using commercial privacy screens (or something similar created at the district) or tape construction paper to the sides of monitors (extending out from the of the screen) to serve as blinders. Preparing Schools for Testing

Office of Accountability Research, Evaluation & Assessment 15 Available Resources on PearsonAccess Select the Go to Resources link on any of the landing pages. Items that are or will be available include:  Student Resources o Student Tutorials (4min 3sec) o Formula Sheets o Calculators  Manuals o Procedures Manual o All assessment manuals  Directions o Student Login Information Instructions and Template o Test Monitor and Student Directions o Student Directions Presentations  User Guides & Technology o PearsonAccess o TestNav o SystemCheck o Proctor Caching  MDE Trainings o Training Module I o Test Security Training o MTAS Trainings  Pearson Trainings o Online Test Monitor Certification Course o All recorded trainings and PowerPoints

Office of Accountability Research, Evaluation & Assessment 16 Test Administration Times Test Administration Times for MCA Reading Total Test Administration includes: 1.Time needed to review the Test Monitor and Student Directions 2.Students logging in to the test 3.Answer student questions 4.Student work time  The estimated times do not include time for completion of item samplers or viewing of the online student tutorial. Estimated Test Administration Times for Reading MCA

Office of Accountability Research, Evaluation & Assessment 17 Test Administration Times Test Administration Times for MCA Mathematics Total Test Administration includes: 1.Time needed to review the Test Monitor and Student Directions 2.Students logging in to the test 3.Answer student questions 4.Student work time  The estimated times do not include time for completion of item samplers or viewing of the online student tutorial. Estimated Test Administration Times for Mathematics MCA

Office of Accountability Research, Evaluation & Assessment 18 Test Administration Times Test Administration Times for MCA Science Total Test Administration includes: 1.Time needed to review the Test Monitor and Student Directions 2.Students logging in to the test 3.Answer student questions 4.Student work time  The estimated times do not include time for completion of item samplers or viewing of the online student tutorial. Estimated Test Administration Times for Science MCA

Office of Accountability Research, Evaluation & Assessment 19 MCA Valid Score Rules For all grades and all subjects: Valid Score (VS) ‒ Respond to 90 percent of the total numbers of items in the assessment to receive a VS code. Not Complete (NC) ‒ Respond to six or more items but fewer than 90% receive a NC score code. Not Attempted (NA) ‒ Respond to fewer than six items receive a NA score code.  Rules for determining how many responses are required to generate a valid score that can be reported for a student vary with the test.

Office of Accountability Research, Evaluation & Assessment 20 New to Country EL English Learners Who Are New to U.S. Schools (New to Country) A new-to-country English learner is a student who is identified as EL in MARSS and first enrolled in a U.S. school within 12 months of the first day of the Title I accountability window. New-to- country status will be verified using the student’s enrollment date in MARSS as the beginning date and the first day of the accountability window as the end date to calculate less than 12 months. For example: A student who entered a U.S. school on April 19, 2014, is considered new to country through April 18, A student who entered a U.S. school on April 14, 2014 would be considered new to country until April 13, 2015; this student could not be indicated as “New-to-Country” for the Title I assessments since the accountability window starts on April 14, 2015.

Office of Accountability Research, Evaluation & Assessment 21 New to Country EL, cont. For Title I assessments, new-to-country students can be removed from AYP proficiency and MMR calculations but they are included in participation calculations. For reading, new-to-country students do not need to take a Title I reading assessment if they have met one of the following conditions. Valid score (VS) or score code for the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test Invalid (INV) score for the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test Student did not have the opportunity to take the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test because the student enrolled after the end of the testing window For mathematics and science, new-to-country students must take the Title I assessments. Districts can indicate that these students are new-to-country, and their scores will be removed from the AYP proficiency and MMR calculations for mathematics (there are no proficiency calculations for science). However, these students will be included in the AYP participation calculation for mathematics and science.

Office of Accountability Research, Evaluation & Assessment SAC Responsibilities for Paper/Pencil Testing SAC Responsibilities for Online Testing Schedule MCA Segments Selecting Test Monitors and Test Administrators Test Monitor Guidelines MTAS Reading, Mathematics and Science 22 3 Roles and Responsibilities for Testing

Office of Accountability Research, Evaluation & Assessment 23 SAC Responsibilities for Paper/Pencil Testing - Accommodations Manage Paper/Pencil Testing ‒ Inventory test materials immediately upon receipt and returns all test materials to Pearson. ‒ Consolidates and request additional materials needed for testing. ›Request additional materials from or ‒ Organize and place accommodated test materials in a secure, locked location until they are distributed for testing ‒ Test administrations are performed by trained test proctors ‒ Distributes Non-secure test materials, such as the Test Monitor and Student Directions for MCA Paper Accommodations, to Test Monitors upon receipt to allow them sufficient time to prepare for the test administration ‒ Schedules paper/pencil test sessions ‒ Completes and monitors the Accommodated Materials Security Sign-out Forms ‒ Distributes test materials on each day of the test, including Red Seals ‒ Reminds test proctors that all test materials are to be kept secure and confidential at all times – Knowing which students have the Math Accommodated Text-to-Speech (ATTS) - Audio CD or Script – Having head-phones available for Text-to-Speech/Audio CD – Establishes who will enter student responses in the Data Entry form in TestNav/DES.

Office of Accountability Research, Evaluation & Assessment 24 SAC Responsibilities for Online Testing Manage online testing – Schedule MCA Test Administrations – Proctor Cache Test Sessions – Create and Start Test Sessions – Monitor Student Progress – Know which students have the Math Accommodated Text-to-Speech (ATTS) – Have head-phones available for Text-to-Speech

Office of Accountability Research, Evaluation & Assessment 25 Scheduling MCA Segments Elements of MCA by Subject − Allows students to exit their test at different points and to return to complete it later − Students responses are saved in the system − Students are brought back to the item last viewed before exiting MCA Reading ‒ Divided into sections by passage and test items ‒ Number and placement of passages will vary by form ‒ Each section begins with a title page ‒ Students will know they have completed all items for the passage when they reach the Review Screen for the section ‒ Before exiting › students must complete a section › review items › select Next Section to go to the next section › students must exit the test on the title page for the next passage  A dialog box will ask students if they want to close the section. Once the section is closed it cannot be reopened.

Office of Accountability Research, Evaluation & Assessment 26 Scheduling MCA Segments, cont. Grades 3–8 MCA Mathematics ‒ Is divided into four sections ‒ Before exiting ›students must complete a section ›review items ›select Next on the Review screen to go to the next section ›students must exit the test on the first item in the next section Grades 11 MCA Mathematics ›students must complete a section ›review items ›select Next on the Review screen to go to the next section ›students must exit the test on the first item in the next section MCA Science ‒ Is divided into two sections containing multiple scenarios and each scenario begins with a title page ‒ Before exiting ›students must complete the first section ›review items ›select Next Section to go the second section ›students must exit the test on the title page of the first scenario on the second section  A dialog box will ask students if they want to close the section. Once the section is closed it cannot be reopened.

Office of Accountability Research, Evaluation & Assessment 27 Selecting Test Monitors and Test Administrators Test Monitors and Test Administrators for Title I Assessments Test Monitors should be selected from the highest possible ranking from this list. If a school has exhausted the availability of persons in category “1,” it should select staff in category “2” and so on. There should be at least one Test Monitor for every 30 students, if possible. 1.Licensed teachers or administrators who work in the school 2.Licensed teachers or administrators who work in the district 3.Paraprofessionals who work in the school 4.School district personnel employed by the school district 5.Licensed substitute teachers who are employed by the district for the purpose of proctoring the test (Not Recommended) For the MTAS, Test Administrators may be any district employee who has completed the MTAS Training for Test Administrators for that year. However, the Test Administrator should be someone who is familiar with the student’s response mode and with whom the student feels comfortable.

Office of Accountability Research, Evaluation & Assessment 28 Materials Allowed when taking The MCAs The following materials are to provided for students for the administration: Student login information Scratch paper (students must write their name on it) A pen or pencil Head phones for Text-to-Speech (mathematics and science only) Hard-copy mathematics formula sheets for grades 5-8 and 11; located on the Student Resources page of PearsonAccess Handheld calculators (grade 11 mathematics only) Materials Not Allowed when taking The MCAs Dictionaries, thesauruses and other reference materials are not permitted (see word-to-word dual-language dictionary in Table 12 for the exception for English learners). Calculator manuals are also not permitted. Handheld calculators are not allowed for the Science MCA. Handheld calculators are not allowed for the grades 3–8 for the Mathematics MCA, unless specified in IEP. Students who require a handheld calculator as an accommodation must use a paper accommodated test book. Test Monitors must actively monitor the student(s) so their handheld calculators are not used on non-calculator test items. Cell phones and other electronic devices must be turned off and put out of reach before the start of testing and throughout test administration and breaks. All instructional posters and graphics in the classroom or on a student’s desk that are related to the subject being tested must be covered or removed during test administration. There is not a list of materials that can remain visible to students during testing. If the materials are related in any way to the subjects being tested, they must be covered or removed. If you have any questions or concerns about materials in the classroom, cover or remove them. Test Monitor Guidelines

Office of Accountability Research, Evaluation & Assessment 29 What Help Can Test Monitors Give to Students? Students must be familiar with how to answer different types of items and Test Monitors cannot provide assistance. Repeat the Test Monitor and Student Directions as written. Scripted directions that Test Monitors can use during online test administrations are included to the Test Monitor Directions. o Test Monitors may not paraphrase Test Monitor and Student Directions, define or pronounce a word within a test item, discuss a test item or give an opinion regarding a test item. The only directions Test Monitors can give students are those that are scripted in the Test Monitor and Student Directions. Students must be familiar with the functionality of calculators or online tools they are using prior to the start of testing; Test Monitors cannot provide assistance with calculators or online tools. If a student points out a test item that appears to have an error, the Test Monitor should instruct the student(s) to continue the test and note the grade, subject, form (if applicable), test session ID (online assessment only), student MARSS number (online assessments only), and item number. o Following the testing session, the Test Monitor should notify the School Assessment Coordinator, who will notify the District Assessment Coordinator. o The District Assessment Coordinator will contact MDE and give the information collected by the Test Monitor. o The content of the item itself should never be referenced, copied or ed within the school or district or in a communication to MDE. If a student points out an issue that appears to be a display error or a technical issue, the student should pause the test and the Test Monitor should immediately contact the District or School Assessment Coordinator or the Pearson Help Desk as directed by the district. Test Monitor Guidelines, cont.

Office of Accountability Research, Evaluation & Assessment 30 Accessibility Panel Tools Accommodated text-to-speech is available for mathematics and science as an accommodation only. Please see more information about accommodated text-to-speech in the next section. The Accessibility Panel tools within the test may be adjusted by students to modify the online testing experience for reading, mathematics, and science. See Table 1 on page 21 in your 2014–2015 MCA and MTAS Assessment Manual. Script For the Mathematics MCA, the script accommodation can only be used in conjunction with the paper accommodated test book. The online Mathematics MCA test will not match the script. For the Science MCA, the script accommodation can be used in conjunction with the online Science MCA or a large print/braille test book and must be administered in a small group or individual setting. When using the science script in conjunction with the online test, the Test Monitor reads from the script while the student sites at the computer and enters responses directly into the online test. Test Monitor Guidelines, cont.

Office of Accountability Research, Evaluation & Assessment 31 Standard Text-to-Speech Standard text-to-speech: ‒ is available for all students taking the Mathematics and Science MCA ‒ is student-initiated ‒ reads everything except the charts and graphs ‒ is available in the Accessibility Panel for all students taking Mathematics and Science MCA Accommodated Text-to-Speech Accommodated Text-to-Speech: ‒ only available for students with an IEP or 504 plan or who are EL-identified in MARSS ‒ is available for mathematics and science as an accommodation only ‒ all graphics and tables will have text-to-speech Encourage the EL teacher to look at the script for the item sampler. Practice the item sampler with the script and then do the online item sampler with the text-to- speech to see what is the best fit for the student.  Detailed information about how the Accessibility Panel works is found in Appendix A in the back of your 2014–2015 MCA and MTAS Assessment Manual. Test Monitor Guidelines, cont.

Office of Accountability Research, Evaluation & Assessment 32 Test Monitor Guidelines, cont. on REA web site rea.spps.org>Test Coordination>Minnesota Comprehensive Assessment MCA

Office of Accountability Research, Evaluation & Assessment 33 4 Preparing for MCAs Item Samplers Training Center

Office of Accountability Research, Evaluation & Assessment 34 Grades 3–8 and 11 Math MCA Online Item Samplers Grades 3–8 and 10 Reading MCA Online Item Samplers Grades 5, 8, HS Online Science Item Samplers MTAS Reading, Math, Science Paper Item Samplers Paper Item Samplers are available via PDF for students requiring accommodations. Access Item Samplers from ANY of the tabs on Use Item Samplers to familiarize students with the format of the test. Item samplers are not practice tests and cannot be used to predict how well students will perform on the assessments. Two ways to access item samplers:  Select the Go to Item Samplers link on any of the landing pages. o From the Item Samplers page in PearsonAccess without logging in. See Accessing Online Item Samplers on PearsonAccess on the General tab on the Item Samplers page of PearsonAccess. o PDFs will also be available for download for any applicable paper accommodations.  Sign in to the Training Center. o After creating sample students and a test session, students can log in to TestNav to take an online item sampler. o Use this to provide both Test Monitors and students familiarity with PearsonAccess functionality and the online testing format. Item Samplers and Training Center On PearsonAccess

Office of Accountability Research, Evaluation & Assessment 35 Test Results Viewing Test results in PearsonAccess On-Demand and Longitudinal Reports Available Reports 5

Office of Accountability Research, Evaluation & Assessment Viewing Test Results in Pearson Access 36 Three components: On-Demand Reports – Preliminary results available during the testing window Published Reports – PDF versions of final paper reports delivered to districts, available following the Posttest Edit Window Longitudinal Reporting System – Results following the Posttest Edit Window loaded to the system for districts to analyze trends and patterns over time

Office of Accountability Research, Evaluation & Assessment 37 On-Demand and Longitudinal Reports On-Demand Reports – access to preliminary online rosters and ISRs – Users can drill down to an individual student to see preliminary student-level results. Longitudinal Reports - analyze results from a specific administration, multiple administrations within a year, or assessment results from year to year – Drill-down, filtering, and sorting capabilities – Aggregate and disaggregate data all the way down to individual student-level results – District, school, and teacher access to student results

Office of Accountability Research, Evaluation & Assessment 38 Which Reports are Available for Which Tests? Test Administration On-Demand Reports Published Reports Longitudinal Report MCA Math, Reading, Science  MTAS Math, Reading, Science  OLPA Math and Reading  GRAD Math and Reading  GRAD Written Composition 

Office of Accountability Research, Evaluation & Assessment 39 When will Results Be Available? Test Administration On-Demand Reports Published ReportsLongitudinal Report Historical Data--10/14/14 OLPA Math and ReadingWithin 1 hour- Loaded weekly beginning10/24/14 GRAD Math and ReadingWithin 1 hour 4 weeks after test window - GRAD Written Composition- 12/15/14 5/22/15 - MCA Math, Reading, ScienceWithin 1 hour7/29/15 MTAS Math, Reading, Science-7/29/15

Office of Accountability Research, Evaluation & Assessment 40 Test Security Test Monitor Certification Test Security Policies Test Security and Training Proctors Topics to Cover during the SAC Training New Security Measures for

Office of Accountability Research, Evaluation & Assessment 41 Test Monitor Certification Test Monitor Certification Training Course – Required for all Test Monitors and SACs – Required once a year – Learn to create, start and monitor test sessions – PearsonAccess ‘Resources’, Pearson Training’  Online Test Monitor Certification Course ( ) Online Test Monitor Certification Course ( )

Office of Accountability Research, Evaluation & Assessment 42 Test Security Policies Nondisclosure Agreement – Everyone with secure access to Pearson must complete the Nondisclosure Agreement. – Test Monitors & data entry personnel must accept the agreement in PearsonAccess or they will not be able to monitor tests or enter student responses. – Everyone with access to the test items must sign the Nondisclosure Agreement. – Anyone who may come in contact with or see test items, can be asked by a student about an item or handle test materials must sign a Nondisclosure Agreement. – Paper Nondisclosure Agreement available here on REA website:

Office of Accountability Research, Evaluation & Assessment 43 Test Security & Training Proctors School SAC Meeting You must ˗ hold a training meeting for your monitors, ˗ have them sign-in to prove they attended the meeting, ˗ allow time for monitors to read the non-disclosure agreement, ˗ have them sign one Non-Disclosure Agreement per school year collect the Non-Disclosure Agreements and file, and ˗ decide what to do when students finish their test, remain seated or leave the classroom. Topics to Cover during your Training Session Proctors must be impartial observers during the test administration. Reproducing or paraphrasing an item in an or in writing is a breach in security. Cell phones, PDAs or translation devices may not be used. Dictionaries may not be used. Scratch paper must be collected from students before leaving the testing room. All instructional posters and graphics in the computer lab or where students will be taking the MCAs must be covered or removed.

Office of Accountability Research, Evaluation & Assessment 44 Test Security & Training Proctors, cont. Test Administration Report (TAR) The TAR is used to record test misadministrations and reasons for invalidations. The SAC completes the TAR and forwards a copy to Robin or Michael in the testing office. The TAR must be kept on file for 12 months. The TAR is available here on REA website: Examples of misadministrations A student uses a calculator during a non calculator segment. A student engages in inappropriate behavior or action that results in a test being invalidated. A student refuses to take an assessment or cheats on an assessment, requiring the assessment to be invalidated. A student’s responses are transferred by staff from an answer document to the Data Entry Interface incorrectly.

Office of Accountability Research, Evaluation & Assessment 45 Test Security Policies, cont. Paperwork to REA – Send paperwork through district school mail to the REA Testing Office, 360 Colborne – Copy of Monitors Training Sign-in Sheet – Nondisclosure Agreement; one for every person having access to MCAs – Classroom Security Checklist for Accommodated Test Materials – Copy of Test Administration Report (TAR)

Office of Accountability Research, Evaluation & Assessment 46 Test Administration Report (TAR)

Office of Accountability Research, Evaluation & Assessment 47 Security Measures for Tip Line MDE has launched a Test Security Tip Line. Educators, parents, students and others can report suspected incidents of cheating, improper or unethical behavior on any Statewide assessments. Links to the Test Security Tip Line are available in two places on the MDE web site. ˗ View the Test Administration page (MDE>School Support>Test Administration) ˗ View the Testing Information page (MDE>Just for Parents> Testing Information)

Office of Accountability Research, Evaluation & Assessment 48 PEARSON ickLink/PearsonAccess SAC’s site for all assessment tasks Help Desk Phone: Monday – Friday 7:00 am – 6:00 pm (CST) MDE REA Test Coordinator Site Contact Information Robin Lane Office; Cell: Michael Wirtz Office: Cell: