December 5 and 6, 2012 Formative Assessment Items and Tasks – in OAS 1 Phase I of Formative Item Bank

Background on RT3 Assessment Initiatives Purpose – To provide assessment resources that reflect the rigor of the CCGPS – To balance the use of formative and summative assessments in the classroom – To promote student learning 2

RT3 Assessment Initiatives Development of a three-prong toolkit to support teachers and districts and to promote student learning – An assessment literacy professional learning opportunity that focuses on implementation of formative instruction – Initial Pilot January 2013 – A set of interim benchmarks in ELA, Math, and selected grades for Science and Social Studies plus additional items – Initial Pilot Spring 2013 – An expansive bank of formative instructional assessment items/tasks based on CCGPS in ELA and Mathematics as a teacher resource - Phase I Release Fall 2012; Phase II release Fall

Formative Instructional Practices 4 Formative Assessment/ Diagnostic Formative Assessment Summative Assessment Design Teach Re-Design Teach Re-Design Teach Re-Design Teach Common Core State Standards

Formative Instructional Practices Conducted during instruction (lesson, unit, etc.) Identifies student strengths and weaknesses Helps teacher determine next steps – Review – Differentiation – Continuation Supplies information to provide students with detailed feedback Assessment for the purpose of improving achievement LOW stakes 5

Purpose of the Formative Item Bank The purpose of the Formative Item Bank is to provide items and tasks for teachers to use in order to assess students’ knowledge while they are learning the curriculum. To provide an opportunity for students to show what they know and show teachers what students do not understand To provide teachers good examples of tasks/items that assess the rigor of the Common Core. 6

Formative Instructional Assessment Items Assess students’ conceptual and skill-level understanding Tasks require students to – Apply the content they know to real world problems – Express logical reasoning by writing, showing their work and/or explaining their answer 7

Items/Tasks for Formative Instructional Use Aligned to CCGPS for Grades 3 - High School ELA & Math Format similar to PARCC’s Common Assessments prototypes - – In other words, items/tasks are more rigorous and require students to show what they know and how they know it Item Development Process (same as used for state tests) – Written to align with curriculum – Piloted in RT3 classrooms in spring 2012 – Reviewed with student data and approved by Georgia Educators 8

Formative Item Bank Item types – About 1700 Formative Classroom Assessment Items About 750 Phase I items Phase I available in OAS as of Nov. 30, 2012 – Some multiple choice items – Most are open-ended or constructed-response format Holistic Rubric Exemplar Paper (i.e.,“ideal response”) Student Exemplars/Responses – DOK targets DOK 2; 19% DOK 3; 41% DOK 4; 39% 9 Anchors used in scoring

The Essence of the Common Core State Standards 10 The Common Core State Standards focus on students learning how to think, not just doing by rote memorization or quick scan-reading, and sets higher expectations on student comprehension and their ability to articulate their knowledge.

Capturing the Essence of ELA Common Core ELA CCSS require students to use evidence from what lies within the four corners of the text and make valid claims that can be proven with the text. Students must be more adept at explaining that evidence orally and in writing. 11

Capturing the Essence of ELA Common Core In order to be college-, career-, and life-ready, students need to be familiar and comfortable with a broad range of genres and formats and much greater emphasis on informational text. – In grades 3- 5: there is a call for more scientific, technical, and historic texts. – In grades 6-8: more literary nonfiction including essays, speeches, opinion pieces, literary essays, biographies, memoirs, journalism, and historical, scientific, technical, and economic accounts. 12

Capturing the Essence of Mathematics Common Core Math CC standards require students to understand why they are doing the things they are doing. Students should – be able to explain why they're doing what they're doing and – justify why they are doing it, not just explain what they are doing 13

Pilot Conducted in spring 2012 Students assigned ELA forms could respond within OAS Students assigned Mathematics multiple- choice forms could respond within OAS. Students assigned Mathematics constructed response items responded on a paper-pencil answer document. 14

Extended Response Items Performance-based tasks May address multiple standards, multiple domains, and/or multiple areas of the curriculum May allow for multiple correct responses and/or varying methods of arriving at a correct answer Scored through use of a rubric and associated student exemplars 15

Grade 3 ELA: Reading Passage (Literary) 16

17 Grade 3 ELA: Reading Passage (Informational) and Item 1

Grade 3 ELA: Rubric for Item 1 18

Grade 3 ELA: Exemplar for Item 1 19

Grade 3 ELA: Sample Student Response for Item 1 20 Student Response Teacher Feedback Rating based upon rubric

Grade 3 ELA: Sample Student Response for Item 1 21

Grade 3: Sample Student Response for Item 1 22

Grade 3: Sample Student Response for ELA Item 1 23

Grade 3 ELA: Performance Data for Item 1 Item Type: Extended Response DOK: 3 Number of Students in Pilot: 90 Response Data – 4’s: 1 for 1.11% – 3’s: 6 for 6.67% – 2’s: 27 for 30% – 1’s: 53 for 58.89% – 0’s: 3 for 3.33% % proficient

Grade 6 Math: Item 25

Grade 6 Math: Exemplar & Sample Student Response 26

Grade 6 Math: Sample Student Response 27

Grade 6 Math: Performance Data Item Type: Extended Response DOK: 3 Number of Students in Pilot: 87 Response Data – 4’s: 0 for 0% – 3’s: 5 for 5.75% – 2’s: 34 for 39.08% – 1’s: 39 for 44.83% – 0’s: 9 for 10.34% % proficient

Scaffolded Items Include a sequence of items or tasks Designed to demonstrate deeper understanding May be multi-standard and multi-domain May guide a student to mapping out a response to a more extended task Scored through use of a rubric and associated student exemplars 29

Grade 3 Math: Scaffolded Item 30

Grade 3 Math: Performance Data Item Type: Scaffolded DOK: 3 Number of Students in Pilot: 44 Response Data – 4’s: 5 for 11.36% – 3’s: 3 for 6.82% – 2’s: 18 for 40.19% – 1’s: 15 for 34.09% – 0’s: 3 for 6.82% % proficient

Overall ELA Pilot Summary Data 32 Grade Number of students and percent falling into each score point Total student N/ % %52.90%23.40%6.60%1.50%100% %51.70%27.70%6.80%2.80%100% %36.50%29.90%17.20%4.20%100% %39.10%33.00%17.00%4.50%100% %38.10%35.00%16.90%4.00%100% %39.90%36.00%11.80%3.10%100% %42.10%32.40%14.90%3.40%100% %42.40%31.80%8.20%1.90%100%

Overall Math Pilot Summary Data 33 GradeNumber of students and percent falling into each score point Total student N/ % %34.60%19.30%4.20%1.90%100% %38.10%17.10%4.10%2.80%100% %34.60%17.00%8.40%3.00%100% %37.60%13.20%3.20%0.70%100% %34.30%13.20%3.40%0.60%100% %35.30%14.00%4.50%2.30%100% %39.80%10.60%2.60%1.50%100% %39.00%11.50%4.10%0.60%100%

Key Findings from Phase I Pilot Students should be earning 3s or 4s to demonstrate grade-level mastery of the standards Preponderance of score points 1 and 2 – Incomplete responses – Responses hampered by writing skills – Students did not show work in mathematics and could not explain why they did what they did 34

Implications for the Classroom 35 Clearer directions for students so they understand the expectations of a good response – Complete sentences, good grammar and syntax – Connections – Explanations and rationales Student self-checklists to assist students in assessing their own responses working on tasks Reinforce instructional recommendations to teachers – Instruction aligned with CCGPS content and rigor – Classroom assessments designed with focus on students articulating how they know what they know – Lessons and classroom assessments integrate knowledge; thus, address multiple standards and domains

Resources to Support the Formative Items/Tasks in the OAS About the Formative Assessment Items and Tasks in the OAS provides a summary of information the items and development. Formative Instructional Resources – Released into OAS is a presentation with details on the items and how they were developed, how to access the items in the OAS, and how to use the items in the classroom. 36

Resources to Support the Formative Items/Tasks in the OAS Student Self-Checklist for Written Responses to ELA Constructed Response Items – is a rubric that students can use to evaluate whether they have met the requirements of the task – provides background information and how to use appropriately and when not to use. 37

Student Self- Checklist for ELA 38

Resources to Support the Formative Items/Tasks in the OAS Student Self-Checklist for Mathematics Constructed Response Items – is a rubric that students can use to evaluate whether they have met the requirements of the task. – provides background information and how to use appropriately and when not to use. 39

Student Self- Checklist for Mathematics 40

Resources to Support the Formative Items/Tasks in the OAS About the OAS Formative Instructional Resources Item Search Tool provides information about how to use the tool to locate items that are tagged with multiple standards. Item Search Tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. The OAS shows only primary but this will show all standards associated with a single item/task. 41

OAS on GaDOE 42

Formative OAS Resources 43

Where Do You Find the Items? 44

Select the Tests tab

Select Create a new test

Give the test a name and unique identifier; choose any other desired options, then click Submit

Select an Item Level New CCGPS items are in Level 2

Select a Subject for the test New CCGPS items are found under Language Arts or Math

Select a Grade level New CCGPS items are in grades 3-HS

Select a Standard New CCGPS items are in grades 3-HS

Select a Sub-Standard New CCGPS items are in grades 3-HS Then click on “Show Items”

- OR - Use the “Advanced Search” Method Copy the unique ID here from the Item Search Tool Then click on “Search”

Put a check to the left of any and all items to be included in the test Then click on “Add Selected to Test”

Make any needed changes or add more items (click Add Items to return to selecting items); when finished, click Create Test

Assign Test to a Class Click on the Training tab for more information about assigning tests to a class. 56

How Students Respond Students can respond to multiple choice items regardless of content area within the OAS. Students can respond to constructed-response ELA items within the OAS. Students need to view constructed-response Mathematics items in the OAS but provide written response on paper. – There is no math editor within OAS. – Some items require students to show work or draw a graph, number line, or figure. 57

Scoring Open Ended Items in OAS Scoring Online -All open ended ELA items can be scored online -Some open ended math items can be scored online -Great way to track students’ proficiency -Students can track their own proficiency as well -Prepare students for what’s to possibly come with PARCC Not Scoring Online -A faux student must be created, assigned the test, and take the test to access rubrics and exemplars -All open ended ELA items can be scored off line -All open ended math items can be scored off line -Great idea if computers are not readily available -Great for students that have not mastered keyboarding 58

Log in to the OAS 59

Click on Tests Tab; then click Score open ended items. 60

Highlight the test you want to score; then click Select. 61

Check the Select/De-Select All; then click Next. 62

63 Performance Scoring Student Recorded Response Question Number with Dropdown Feature Show Question & View Sample Paper

64 When the Show Question option is selected the question appears.

65 When the View Sample Paper option is Selected the student exemplars will appear.

66 When the View Sample Paper option is selected the student exemplars will appear

67 This drop down allows for the correct performance score to be given Here is where the teacher can give commentary/feed back Click Submit to continue to next item.

After you have scored and submitted the last item, this screen will appear. Click the tab labeled Student. 68

Click the View teacher-assigned results. 69

Select the test for which want to see the report, and click Next. 70

Select a student whose report you want to see, then click next. 71

All tests that a student has taken will appear. Click the specific test for which you want to see the results. 72

Results page: this page contains analysis of entire test. 73 This test contained 6 items. 4 open ended that we scored using the 4pt. rubric (1,2,4,6) and 2 multiple choice items (3,5). You can click each question and it will display the item and how the student responded or continue to scroll down this page.

Student recorded answers with questions 74

Finding an Item for a Specific Standard OAS only allows coding by a primary standard code. Many of the constructed response items are coded to multiple standards because content knowledge is not discrete and the Common Core expects students to assimilate content across standards and domains. A tool has been created that will allow teachers to search for a standard. 75

OAS Formative Instructional Resources Item Search Tool This tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. All items aligned to a standard can be located, whether it be the primary standard or a secondary standard. The Item Search Tool can be downloaded from the GaDOE website and installed on your desktop. 76

How to Use the Search Tool 77 From the OAS Resources page on the GaDOE website, click “Download Item Search Tool” Open the zip folder and then the folder named “ OASFormativeInstructionalResourcesItemSearchTool” Select and open the “setup.exe” file.

If you do not have Microsoft Access 2007 installed on your computer, Access Run Time 2007 will be automatically installed. This application will allow you to view the database. Step by step instructions for installation are provided in the “About the Item Search Tool” document on the GaDOE website. A shortcut will be saved to your desktop. Double-click the shortcut to open the database. 78 How to Use the Search Tool

Using the drop down menus select Subject, Grade, and Standard (in that order) and click RUN. 79 How to Use the Search Tool

The report will include all items aligned to the selected standard. Copy and paste each Item ID into Advanced Search in OAS. – Use Ctrl-C to copy 80

Formative Assessment Initiatives Bringing a Balanced Assessment Focus to the Classroom 81 Assessment Literacy Professional Learning Interim Benchmark Assessments Formative Assessment Bank First initiative – Phase I items released into OAS; Phase II items will be piloted in early 2013 Another initiative to begin pilot in January 2013 Third initiative – Phase I item development begun with pilot in spring 2013

Georgia Department of Education Assessment and Accountability Melissa Fincher Associate Superintendent Assessment and Accountability Dr. Melodee Davis Director Assessment Research and Development Michael Huneke OAS Manager Dr. Dawn Souter Project Manager. RT Robert Anthony Assessment Specialist, RT3 Formative Item Bank Dr. Jan Reyes Assessment Specialist, RT3 Interim Benchmark Assessments