Proof of Concept Study: Grade 5 Mathematics Grade 6 ELA/Reading Nancy Carolan, Hope Lung NC Department of Public Instruction August 18, 2015

Proof of Concept Study Purposes and Design 2

Purposes–Grades 3-8 Valid and reliable measures of content standards Growth data for educator effectiveness reports Assessments throughout the year that inform and guide instruction and that predict performance on future assessments A stand-alone summative assessment at the end of the year that may be connected to the interim assessments 3

Grades 3-8 Concept Design Students take 3 interim assessments during the school year  The interim assessments are not included in accountability calculations Students take a shortened end-of-year test  End-of-grade test without embedded field test items  The modified end-of-year test is included in accountability calculations 4

Interim 1 Interim 2Interim 3 End-of-Year (Assess entire year) Grades 3-8 Concept Design 5 Note: Design could be altered based on outcome of study

Timeline Based on Current Design YearAdministrationGrade LevelsPurpose 2015–16 Proof of Concept (sample population) Grade 5: Math Grade 6: ELA/Reading Determine feasibility of concept Summer 2016: Decision-point of how to proceed 2016–17 Field Test (sample population) Or Release RFP for assessment Grades 3–8: Math Grades 4–8: ELA/Reading Test items for inclusion in operational test forms 2017–18Pilot Operational Year (statewide) Grades 3–8: Math Grades 4–8: ELA/Reading Full Implementation 6

Proof of Concept Study Research Questions 7

Research Questions Determine the feasibility of implementing the concept across grades 3–8  data analysis to affirm reliability  input/feedback on the integration of the assessments in instruction  logistical/operational (delivery mode, scoring)  professional development requirements  possibility of different item types 8

Research Questions Is it valid and reliable to combine results on the interim assessments for proficiency and growth reporting; thereby, eliminating an end-of-year summative assessment? Do interim results provide teachers and students with useful information to inform and improve the delivery of instruction? Will interim assessments predict performance on the end-of-year test? 9

Research Questions Are the interim assessments required of all students or are some of the interim assessments optional? Does the structure of the content standards require adjusting design of the through-grade model? Is it feasible to incorporate constructed- response items or writing prompts on the interim assessments? 10

Research Questions Is the performance affected by the delivery of through-grade assessment versus a traditional summative assessment model? What information will be available for student-level and teacher-level reports and how is such information best delivered and presented? Is the professional development adequate? 11

Research Questions Is it feasible to deliver the assessments both online and paper/pencil? Does the through-grade model provide parents with useful information and do they view the model as an effective way to assess students? 12

Proof of Concept Study Reporting 13

Information for Teachers 14 Student-Level Reports  Number of items correct by content standard  Number of overall items correct  Item-level reporting Aligned content standard Student response Correct answer Item type Selection type Calculator active/inactive

Proof of Concept Study Sample and Specifications 15

Sample―Grades 5 and 6 Grade 5 Math — 45 schools, 4,021 students Grade 6 ELA/Reading ― 35 schools, 4,859 students Generalizable to the entire state Stratified random sample pulled based on variables Demographic variables: region, ethnicity, gender, economically disadvantaged School-level achievement variable: mean scale score 16

Interim―Test Specifications Test specification development is step 2 in the test development process Separate meetings held late June and early July  Invitees selected by NCDPI Curriculum and Instruction Mathematics and ELA  Active grade 5 Math and grade 6 ELA classroom teachers, instructional coaches, and LEA curriculum and instruction leaders  First half of meeting―Training by NCDPI Curriculum and Instruction  Second half of meeting—NCDPI Test Development collected feedback and recommendations Test Development team discusses teacher recommendations with NCDPI Curriculum and Instruction to finalize test specifications Feedback will be collected from sampled schools throughout the year 17

Interim Design—Grade 5 Math Calculator active and inactive sections 25 Items 21 multiple-choice items (8 calc. inactive, 13 calc. active) 4 gridded-response items (calc. inactive) Interim 1: 5.NBT.2, 5.NBT.5, 5.MD.5.b, 5.MD.5.c DRAFT Interim 2: 5.NF.1, 5.NF.2, 5.NF.3, 5.NBT.6, 5.NBT.7 DRAFT Interim 3: 5.NF.2, 5.NF.4 a & b, 5.NF.6, 5.NF.7 a, b & c, 5.NBT.7 Assessment Specification documents Answer sheets scanned and reports distributed by the LEA Test Coordinator 18

Interim 1 Design—Grade 6 ELA Interim 1 Design 20 multiple-choice items Selections: Poetry (1), Informational (1), Literature (1) Answer sheets scanned and reports distributed immediately by the LEA Test Coordinator Standards assessed on each ELA interim assessment RL.1, RL.2, RL.3, RL.4, RL.5, L.4a, L.5.a RI.1, RI.2, RI.3, RI.4, RI.5, RI.6, RI.8 19

Interims 2 & 3 Design—Grade 6 ELA DRAFT Interims 2 and 3 19 multiple-choice items, 1 constructed response (CR) item CR item will be short-answer and can typically be answered in a paragraph or less. Students must write on lines provided on answer sheet. Interim 2 selections: Informational (1), Literature (2) Interim 3 selections: Informational (2), Literature (1) Answer sheets returned to the LEA Test Coordinator to be shipped for central scoring, reports returned within 8 days Standards assessed on each ELA interim assessment RL.1, RL.2, RL.3, RL.4, RL.5, L.4a, L.5.a RI.1, RI.2, RI.3, RI.4, RI.5, RI.6, RI.8 20

Interim Design—Math and ELA Paper/pencil only to ensure participation Up to 90 minutes maximum test administration time Items pulled from the EOG item pool (if available) One form for each interim assessment Assessment Specification documents  May need to create a new NC Education account to access 21

Proof of Concept Study Policy and Administration 22

Interim Assessment Policy Testing Windows (3)  Interim 1 October 1–30, 2015  Interim 2 December 8, 2015–January 22, 2016  Interim 3 March 3–31, 2016 Districts/charters can determine the testing days within the designated windows Interims are not required to be administered to all students on same day, but should be administered within the same week Make-up administrations are optional but strongly recommended Should be administered by the classroom teacher Proctors are not required Administration does not require the removal of displays 23

Interim Assessment Policy Eligibility Mathematics Grade 5  All students enrolled in grade 5 at sampled schools who participate in the EOG mathematics assessment English Language Arts/Reading Grade 6  All students enrolled in grade 6 at sampled schools who participate in the EOG English Language Arts/ reading assessment Both  Transfer students−−Take the interim(s) and the modified end-of- year (EOY) test  No opt out 24

Interim Assessment Policy Students Not Eligible to Participate Students with disabilities whose IEP documents participation in the NCEXTEND1 alternate assessment LEP students who scored below Level 4.0 Expanding on the W-APT and are in their first year in U.S. schools are not eligible to participate in the grade 6 ELA study, but they are eligible to participate in the grade 5 math study Students who are granted a medical exception from the Division of Accountability Services for the EOG assessments 25

Interim Assessment Policy Accommodations IEP, 504, and/or LEP teams do not have to reconvene and document the accommodations for the Proof of Concept Studies.  Students may use the accommodations that are specified on their current IEPs, Section 504 Plans, or LEP documentation for the EOG for the modified end-of-year test.  Instructional accommodations may be used for the interims except for the Read Aloud and Signing/Cueing accommodations for grade 6 ELA/Reading. Reading aloud or signing/cueing the selections, questions, or answer choices invalidates results because the interims measure reading skills. 26

Interim Assessment Policy Special Print Versions Braille, Large Print (LP), One Item Per Page (OIPP), and Large Print One Item Per Page Editions (LP/OIPP) can be ordered from TNN  Accommodation Notification Request Forms for special print versions do not need to be sent in for interim assessments  Submit orders at least 30 working days before the actual administration date 27

Interim Assessment Policy Materials  Assessment Guide –guide for interims (separate guides: 1 for ELA/Reading; 1 for Math) –guide for end-of year (separate guides: 1 for ELA/Reading; 1 for Math)  Answer sheets  Test books (separate test books for the subjects) 28

Interim Assessment Policy Required Supplemental Materials English Language Arts/Reading  Blank paper Mathematics  Blank paper  Graph paper (auto shipped for interims and end-of-year)  Calculators –Any four-function calculator with memory key 29

Interim Assessment Policy Security Assessment guides are not secure test materials  Store at the school until all interims have been administered, then securely destroy Following the administration, interim assessment booklets are to be kept at the schools for 4 weeks, then securely destroyed  Booklets must remain in the school  Booklets should be accounted for at all times  Local decision where booklets are stored at the school (not accessible to students)  Teachers may (should) use the booklets with students in reviews  Parents may view the booklets only within the school setting. The teacher may share with parents the student’s scores on the items through customary communication (i.e., individual parent/teacher conferences at the school)  Interim assessments (booklets, items, content) cannot be shared with other schools 30

Modified End-of-Year Assessments for Concept Study Participants 31

End-of-Year (EOY) Test Specification— Math Grade 5 32 Number of Multiple- Choice Items Number of Gridded Items Total Number of Items Modified EOY Math G5 (Concept Study participants only) End-of-Grade Grade 5 Math 46854

End-of-Year (EOY) Test Specification— ELA/Reading Grade 6 33 Number of Multiple- Choice Items Number of Constructed Response Items Total Number of Items Total Number of Selections Modified EOY ELA/Reading G6 (Concept Study participants only) End-of-Grade Grade 6 ELA/Reading 560 7

EOY Test Policy—Concept Study End-of-Year assessments follow the same guidelines as the End-of- Grade Assessments –Multiple forms –Test window –Test security –Supplemental materials –Paper/pencil administrations only –Submit accommodation notification forms –Scanned locally –Reporting –Scores included in accountability calculations 34

Proof of Concept Study Resources 35

NC Education Page Use the following link to access presentations, memos and resources 36

Proof of Concept Study Curriculum & Instruction Professional Development 37

38 ELA Professional Development Webinars Part One: Standards Study − includes resources for understanding, expectations for students, progression, integrating the strands. Date: August 19 3:00 ‒ 4:00 p.m. Part Two: Text Complexity − includes how it should be considered within instruction, scaffolding, and determining the complexity of a text. Date: August 20 3:00–4:00 p.m.

39 If you were unable to attend math professional development… All materials—PowerPoint and handouts will be available on the DPI Mathematics Wiki: See “Proof of Concept” Page on wiki for registration links as well.