11 MME Update: Materials Shipment, Handling, and Return MME Update: Materials Shipment, Handling, and Return Presented by: Michigan Department of Education ACT, Inc. Measurement, Inc.

22 Welcome and Introductions MDE Office of Educational Assessment and Accountability Emily Taylor ACT, Inc. (MME Day 1 and Day 2) Diane Weyer Measurement, Inc. (MME Day 3) Tom Jenkins Kathy Palmer

3 Questions during the Webcast ● ● ● ● Fax:

44 Design of the MME ● ACT Plus Writing – Day 1-ACT, Inc. ● WorkKeys – Day 2- ACT, Inc. ● Michigan Components – Day 3 – Measurement Inc.

55 MME Testing Schedule ● Initial Testing Spring 2010 ● March 9: Day 1 ACT Plus Writing ● March 10: Day 2 WorkKeys ● March 11: Day 3 Michigan Components

66 MME Testing Schedule ● Makeup Testing Spring 2010 ● March 23: Day 1 ACT Plus Writing ● March 24: Day 2 WorkKeys ● March 25: Day 3 Michigan Components

77 MME Testing Schedule ● Accommodations Testing Spring 2010 ● Two-week window ● Day 1 - March 9 – March 23 ● Day 2 - March 10 – March 24 ● Day 3 - March 11 – March 25

88 Testing Contractor – ACT, Inc. ● ACT, Inc. – Day 1 and Day 2 ● Two distinct assessments – ACT and WorkKeys ● Separate administrations ● Separate packaging ● Separate answer folder scoring ● Iowa City, IA

99 Testing Contractor – Measurement, Inc. ● Measurement, Inc. – Day 3 ● Michigan components ● Separate administration ● Separate packaging ● Separate answer document scoring ● Durham, NC

10 MME Day 1: ACT Plus Writing & MME Day 2: WorkKeys 10

11 Non-Secure Materials Day 1 and Day 2 ● Arrive in schools by February 9 ● Separate shipments ● Sent to Test Supervisors ● Packing Lists ● Manuals (standard time only) ● Answer Folders (blank) ● Barcode Labels (pre-printed and blank)

12 Non-Secure Materials Day 1 and Day 2 ● Answer Folder Supplements ● Information not in manuals ● Unique to Michigan ● Day 1 and Day 2 12

13 Non-Secure Materials ● Conduct Pre-test Sessions with Students ● Conduct Training Session for Testing Staff ● Room Supervisors and Proctors ● Anyone involved in MME test administration

14 Day 1 and Day 2 Additional Orders ● Call ACT to place additional orders for Day 1 and Day 2 ● ACT State Testing: (800) ext 2800

15 Pre-Test Sessions ● In-school sessions, supervised ● Required before test day ● Complete Non-cognitive and Demographic Information ● Barcode Labels affixed---Day 1 and Day 2 Answer Folders

16 Barcode Labels ● Labels will be white in color ● Affix to the designated space on the Answer Folder ● All students must grid identifying information and address—for Day 1 ● Make certain label corresponds to correct student

17 Day 1 Answer Folder ● Sections to be completed in pre- test session – about minutes ● Benefit to students ● Barcode Label

18 Day 2 Answer Folder ● Sections to be completed in pre-test session – about 15 minutes ● Required items in Day 2 Answer Folder Supplement ● Not all sections are required to grid ● Barcode Labels

19 Collect and Organize ● Completed Answer Folders following Pre-Test ● Collect folders (keep secure) ● Student reminders ● Organize and Plan redistribution ● After the pre-test session, give Answer Folders for all Accommodated students to TAC

20 Collect and Organize (cont.) ● Prepare Rosters for standard time test rooms ● Create Rosters BEFORE Test Day ● Name of students SCHEDULED to test ● Prepare for Day 1; copy for Day 2 if same

21 Staff Training Session ● BEFORE Test Day ● Staff Training Checklists/Outline ● Administration Manuals ● Verbal Instructions verbatim ● Directions to Room Supervisors ● 1 copy of each applicable manual ● MME webpage

22 Day 1 and Day 2 Standard Time Secure Materials ● Ship week of March 1 ● To Test Supervisor ● TEST BOOKLETS—Standard Time Only ● Supervisor’s Report Form ● Return Packing instructions ● Return Envelopes/Polymailer Bags ● Spring Break Conflict ● KEEP BOX(es) for return

23 Day 1 and Day 2 Accommodations Secure Materials ● Ship week of March 1 ● To Test Accommodation Coordinator ● TEST MATERIALS ● Day 1 in Kits by Student ● Day 2 not in Kits ● Return Packing instructions ● KEEP BOX(es) for return

24 Security of Test Materials ● Check-in immediately ● Use enclosed packing list to inventory secure materials for each shipment ● Problems: Day 1 or Day 2 ● call ACT x 2800 ● or x 1788 (Accommodated Testing)

25 Security of materials ● “Two Lock Rule” ● Treat as you would a large sum of $ MONEY $ ● Documented “Chain of Custody ”

26 Test Day Preparation/Planning ● Review rooms ● Organize answer folders ● Prepare rosters ● Assign staff—staff training ● Prepare test day materials ● Practice verbal instructions ● Student reminders

27 Test Day Briefing Session ● Required each test day ● Room Supervisors and Proctors arrival ● At least 30 minutes prior to student check-in ● Review procedures

28 Test Day Briefing Session ● Last-minute details ● Test Supervisor hand-delivery of test materials to Room Supervisor(s) ● Document with Test Booklet Count Form ● Required Forms

29 Responsibilities After Testing 29

30 Room Supervisor Responsibilities ● Account for all test booklets and answer documents before dismissing students ● Verify that you have an answer document for each student who tested ● Validate correct student barcode label applied to each answer document

31 Room Supervisor Responsibilities ● Verify each answer document has ● Form number ● Complete school use only sections ● Highlight absent students on Testing Roster – need Makeup test ● Return all materials and completed testing documentation forms to Test Supervisor

32 School Use Only Sections – Day 1 Answer Folder ● Completed by Testing Staff, not examinee ● Critical for Accommodations— Questions 3-9 ● Use Day 1 Answer Folder Supplement ● School Use ONLY questions, Section V

33 School Use Only Sections Day 2 Answer Folder ● Completed by Testing Staff, not examinee ● Use Day 2 WorkKeys Answer Folder Supplement ● School Use Only sections: ● Block 17—Report Codes ● Block 26—Local Items

34 Test Supervisor Responsibilities ● Account for all materials returned by each Room Supervisor: ● Test materials ● Answer documents to be scored ● Documentation forms ● Verify each answer document has: ● Correct Barcode label ● Form number ● Completed school use only sections

35 Test Supervisor Responsibilities ● Account for all standard time materials ● Make copies of completed test documentation for your school files

36 Post-Test Requirements - TAC ● Answer Folder Supplements ● Complete Section V on Day 1 Answer Folders ● Complete Admin Codes; Local Items; and School Use only on Day 2 Answer Folders

37 Post-Test Requirements - TAC ● Supervisor’s Manual ACT Plus Writing State Special Testing ● Day 2 Instructions in WorkKeys Administration Manual State Special Testing

38 Day 1 and Day 2 Standard Time Materials Return ● Thursday, March 11 ● Scheduled FedEx pick up for Initial Test Materials for Day 1 and Day 2 for return to ACT in Iowa City, Iowa

39 Scheduled Pickup ● Pick up Time: 8:00 am – 5:00 pm ET ● Place boxes in FedEx pick up space ● Keep record of tracking number(s) ● Do not call FedEx ● Keep materials separate

40 Day 1 Standard Time Materials Return ● RED Ink Envelope ● Administrative Forms ● GREEN Ink Envelope ● Supervisor’s Report Form ● Used Answer Folders for processing ● Polymailer ● Test Booklets - ALL

41 Day 1 Standard Time Materials Return ACT Answer Documents and Admin Forms ACT Test Booklets and Manuals To ACT

42 Day 2 Standard Time Materials Return ● ORANGE Ink Envelope ● Administrative Forms ● BLACK Ink Envelope ● WorkKeys Supervisor’s Report Form ● Used Answer Folders for processing ● Polymailer ● Test Booklets – ALL

43 Day 2 Standard Time Materials Return To ACT WorkKeys Test Booklets and Manuals WorkKeys Answer Documents and Admin Forms

44 Order Makeup Materials ● Order materials on OEAA Secure Site ● Standard time only ● Day 1 and Day 2 – Order March 11 by 11:59 p.m. ● Log into the OEAA Secure Site: ●

45 Day 1 and Day 2 Makeup Materials ● Scheduled to arrive in schools by March 18

46 Day 1 and Day 2 Standard Time Materials Return - Makeup ● Thursday, March 25 ● Fed Ex picks up Day 1 and Day 2 Makeup materials for return to ACT in Iowa City, IA

47 Day 1 and Day 2 Standard Time Materials Return - Makeup ● Keep materials separate ● Your boxes must include: ● Used and unused test booklets ● All testing materials – used and unused ● Reverse the flaps on the boxes

48 Returning Accommodation Materials – Day 1 ● KEEP SEPARATE ● BLUE Ink envelope for ACT-Approved answer folders/forms ● BROWN Ink envelope for State-Allowed answer folders/forms ● NO Polymailer Bag

49 Returning Accommodation Materials – Day 1 (cont.) ● Supervisor’s Report Form ● Accommodations Rosters ● Administration Reports ● Other forms ● Return to ACT in Iowa City, Iowa

50 Returning Accommodation Materials – Day 1 ACT-Approved To ACT ACT Test Booklets, All Formats, and Manuals ACT Answer Documents and Admin Forms

51 Returning Accommodation Materials – Day 1 State-Allowed ACT Test Booklets, All Formats, and Manuals To ACT ACT Answer Documents and Admin Forms

52 Returning Accommodation Materials – Day 2 ● Your scorable boxes must include: ● Purple-Ink Envelope ● Supervisor’s Report Form ● Day 2 Answer Documents ● Accommodations Rosters ● Administration Reports ● Polymailer

53 Returning Accommodation Materials – Day 2 To ACT WorkKeys Test Booklets, All Formats and Manuals WorkKeys Answer Documents and Admin Forms

54 Returning Accommodation Materials Day 1 and Day 2 ● Return Day 1 and Day 2 materials to ACT, Inc. in Iowa City, IA ● Package Day 1 separately from Day 2

55 Avoiding the Pitfalls – ● ACT-Approved separate return from State- Allowed ● Standard Time materials return separate from Accommodations for Day 1 and Day 2

56 Scheduled FedEx Pickups ● March 11 – Day 1 and Day 2 to ACT ● March 25 – Day 1 and Day 2 to ACT

57 ACT Contact Information For ACT and WorkKeys specific questions – Day 1 or Day 2 ACT State Testing ● (800) ext 2800 ●

58 ACT Contact Information For ACT and WorkKeys accommodations questions – Day 1 or Day 2 ACT State Special Testing ● ext 1788 ●

59 Questions during the Webcast ● ● – ● ● FAX:

60 DAY 3: MICHIGAN COMPONENTS

61 Receipt of Non-Secure Materials Shipment

62 Day 3 Non-Secure Materials ● ● Arrive in schools by February 9 ● ● Sent to Test Supervisors ● ● Packing list and Box lists ● ● Barcode labels (pre-printed and blank) ● ● Day 3 answer documents (packs of 10) ● ● Day 3 admin manuals ● ● For students testing with accommodations (1 per 10 extended-time only students) ● ● Standard (1 per 15 students)

63 ● Packing List- contents of entire shipment ● Box List – contents of individual box 63 Day 3 Non-Secure Materials

64 Day 3 Non-Secure Materials ● ● TS and TAC ● ● Work together to inventory materials ● ● Ensure that there is a barcode label for each examinee ● ● Determine if additional non-secure materials are required ● ● Class/Group ID sheets: additional order

65 Day 3 Additional Orders: Non-Secure Materials ● Answer documents: 2/09 – 3/15 ● Admin manuals: 2/09 – 3/15 ● Class/Group ID Sheet: 2/09 – 3/19 65

66 Day 3 Additional Orders: Non-Secure Materials ● Order using OEAA Secure Site: DO NOT CALL MEASUREMENT INC.

67 Day 3 Pre-Test Session

68 Day 3 Pre-Test Session ● Session takes about 15 minutes ● Verbal instructions in the MME Day 3 Administration Manuals ● Pre-Test sections to be completed are on front page of answer document

69 Day 3 Pre-Test Session ● Optional ● Research Codes ● Class Group Codes ● Collect answer documents from students ● Staff affix barcode label in Box 1 on front page

70 Day 3 Pre-Test Session ● TS receives and stores completed answer documents ● After the pre-test session, TS gives answer documents for all accommodated students to TAC

71 Receipt of Day 3 Secure Materials Shipment

72 Day 3 Secure Materials ● Arrive in schools by March 2 ● Sent to Test Supervisors ● Day 3 Standard Time Test Booklets ● Day 3 Accommodated Test Booklets - Extended Time Only ● Accommodations Kits ● Return Materials Kit ● KEEP BOX(es) for return

73 Day 3 Secure Accommodations Testing Materials ● ● Day 3 Form 12 - Standard Print ● ● All students testing with extended time only ● ● Accommodated Format Kits ● ● Include manual ● ● Include Form 12 test booklet (except LP and Braille)

74 ● Packing List- contents of entire shipment ● Box List – contents of individual box ● Security List –each individual secure item in shipment 74 Day 3 Secure Materials

75 Box List ● ● Use to “check in” secure materials received ● ● Use later to “check out” secure materials to RS and TAC.

76 Day 3 Security Barcode Numbers ● ● Each secure item has a 10-digit barcode number /SP ● ● Each shrink-wrap pack has a 10-digit barcode number preceded by “SP.” ● ● Within a shrink-wrap pack, test book barcodes are in numerical sequence.

77 Day 3 Security Barcode Numbers: Initial Forms / SP / SP / SP / SP / SP / SP / SP / SP / SP / SP

/ SP / SP / SP / SP / SP / SP / SP / SP / SP / SP Day 3 Security Barcode Numbers: Form 12

79 Day 3 Barcode Numbers: Accommodated Formats ● ● Each accommodated item has a 10- digit barcode number. ● ● Each accommodated kit has a 10- digit package barcode preceded by “SP”.

80 Day 3 Barcode Numbers: Accommodated Formats ● ● Each kit cover sheet lists contents of package: ● ● Kit: SP – 12 – Form 12 test booklet: – Session 1 DVD: – 12 – Session 2 DVD: – 12 – Session 3 DVD: – 12 –

81 Security of Test Materials ● Problems: Day 3 ● Call Measurement, Inc. ● ● Additional Orders: Day 3 ● Use OEAA Secure Site ● Do not call Measurement, Inc.

82 Day 3 Additional Orders: Secure Materials ● Initial form test books: 2/25 – 3/4 ● Form 12 test books: 2/25 – 3/15 ● Accomm Formats: 2/25 – 3/15 82

83 Day 3 Initial Test Day

84 Distribution of Test Books Day 3 ● After students are seated ● Only when prompted by verbal instructions ● In spiraled form order ● Only to those present ● Individually, one-by- one Form 1 X Form 2 Form 5 Form 4 Form 3 X Form 7 Start End Form 6

85 Answer Document: School Use Only Sections ● ● Completed by Testing Staff ● ● School Use Only sections: ● ● Page 4 of Answer Document—all boxes ● ● Blocks 13 & 14 all students as applicable

86 Day 3 Timing Codes – Accommodated Testing ● ● Completed by TAC or Room Supervisor, not examinee ● ● Completed for each subject ● ● Timing codes in Day 3 Manual ● ● Blocks Accommodated students only

87 Test Supervisor Responsibilities ● ● Receive all materials from RS and TAC ● Account for all secure materials using: ● Test Booklet count form ● Box list from secure shipment ● Security list

88 Test Supervisor Responsibilities ● ● Verify each answer document has: ● ● Correct Barcode label ● ● Form number ● ● Completed school use only sections ● ● Make copies of completed test documentation for your school files 88

89 Return of Initial Test Day Materials

90 Day 3 Scorable Materials Return School Header on top 2. Class/Group ID Sheets (if used) 3. Answer Documents Use a separate school header for each form type. Secure bundle with paper band.

91 Day 3 Scorable Materials Return ● ● Test Administration Forms ● ● GREEN ENVELOPE ● ● Irregularity Reports and supporting documentation ● ● YELLOW ENVELOPE Day 3 Admin Forms Day 3 Irregularity Reports

92 Day 3 Scorable Materials Return ● ● Place yellow and green envelopes in bottom of box. ● ● Place bundle of answer documents into box. ● ● Seal box and place “scorable” label and FedEx label on box

93 Day 3 Non-Scorable Materials Return ● ● RETURN: ● ● Initial Form Test Books ● ● Accommodated formats, if accommodated testing is complete ● ● RETAIN: ● ● Answer docs for makeup ● ● Admin manuals ● ● All accommodated materials still in use ● ● Unused materials return items (labels, envelopes, etc.)

94 ● ● Pack all standard print test books. ● ● If returning accommodated formats also: ● ● Pack LP or Braille next. ● ● Place DVDs and cassettes in plastic bag. ● ● Pack this plastic bag last. Day 3 Non-Scorable Materials Return

95 FedEx Pickup Friday, March 12 ● ● Pick up Time: 8:00 am – 5:00 pm

96 FedEx Pickup ● ● Place boxes in FedEx pick up area ● ● Keep record of tracking number(s) ● ● Do not call FedEx

97 FedEx Pickup ● ● By 5:00 pm ET, if FedEx has not made the scheduled pickup at your school, call: ● ● Measurement, Inc. ● ● 866/

98 Order Makeup Materials ● ● Order materials on OEAA Secure Site ● ● Day 3 – order by 11:59 pm Eastern Time on March 15

99 Day 3 Makeup Materials Return

100 Day 3 Makeup & Accommodated Materials Return ● ● Scorable ● ● Makeup Test Day answer documents ● ● Accommodated testers’ answer documents ● ● Non-Scorable ● ● All test booklets ● ● All accommodated materials ● ● All manuals, unused docs, etc

101 Test Supervisor & TAC ● Test Supervisor receives all accomm materials from TAC ● “Chain of Custody” ● Test Supervisor responsible for return materials 101

102 Day 3 Scorable Materials Return ● ● School Header Sheet ● ● Use separate school header for each form type ● ● Class/Group ID Sheets (if used) ● ● Day 3 Answer Documents ● ● Secure with paper band ● ● Admin Forms & Irregularity envelopes

103 Non-Scorable Materials Return - Makeup

104 Scheduled Pickup Date ● ● March 26 – Day 3 to Measurement, Inc. ● ● Between 8:00 AM and 5:00 PM ● ● Makeup and Accoms in same shipment

105 Measurement, Inc. Contact Information ● ● For Measurement, Inc. – Day 3 ● ● ● ●

106 Conclusion

107 Avoiding the Pitfalls ● Administer the Day 1 writing test ● READ THE MANUALS ● Test materials are NOT to be shared or transferred between school ● Keep Day 1, Day 2, Day 3 materials SEPARATE and SECURE

108 Avoiding the Pitfalls ● Late return—answer document not scored ● Improper return packaging—no guarantee of scoring ● No barcode label on student answer document--$50 processing fee per answer document ● No form number on student answer document – answer document not scored

109 Contact OEAA Or call

110 QuestionsandAnswers

111 Questions during the Webcast ● ● – ● ● FAX: