Assessment Literacy Series 1 -Module 3- Item Specifications

Participants will: 1. Examine multiple choice and constructed response items/tasks. 2. Develop items/tasks that fit the previously created specifications and blueprint in terms of:  content accuracy;  item type;  cognitive load; and  sufficiency. 2 Objectives

Participants may wish to reference the following: Guides Handout #4 – DoK Chart Handout #5 – Item Examples Other “Stuff” DoK Chart II “Smart Book” Textbooks, teacher manuals, and other supplemental materials 3 Helpful Tools

4 Outline of Module 3 Module 3: Item Specifications Multiple Choice Items Constructed Response Items/Tasks Short Constructed Response Extended Constructed Response Process Steps

MC items consist of a stem and answer options Grades K-1 = 3 options, Grades 2-12 = 4 options MC items contain only one correct answer Other options (distractors) are the same structure and length as the answer The distractors should be plausible (realistic) Balance the placement of the correct answer Avoid answer options that provide clues to the answer 5 Multiple Choice (MC) Guidelines

Answer options should be in ascending or descending order when possible Avoid “All of the above” and “None of the above” Directions state what to do, where and how to respond, and point values Refrain from adding directions when test-takers are repeating the same behavior from item to item 6 MC Guidelines (cont.)

Constructed Response (CR) Guidelines Language is appropriate to the age and experience of the students Student expectations are clear- Explain vs. Discuss Describe vs. Comment State the extent of the expected answer Give three reasons vs. give some reasons Directions state what to do, where and how to respond, and point values Refrain from adding directions when test-takers repeat the same behavior from item/task to item/task 7

Short Constructed Response (SCR) items/tasks: One step to solve Requires a brief response (2-5 minutes) Worth up to 3 points Extended Constructed Response (ECR) items/tasks: Multiple steps to solve Requires 5-10 minutes to answer Worth 4 or more points Note: ALL CR items/tasks require a well-developed, scoring rubric. 8 CR Guidelines (cont.)

Scenarios and passages should be: Relatively short Developmentally appropriate Sensitive to readability Performance expectations must state exactly what is to be observed and how it will be measured. Items/tasks are considered secure even in draft form. Copyrights must be handled appropriately. Images, graphs, and charts must be clear and of sufficient size for interpretation. 9 Helpful Hints

Each item/task requires a unique identifier. ◦ Tags contain information used to code and identify items/tasks. ◦ Item tags typically contain: Item number Subject Grade Post test Item type DoK level Content standard number (0008.MTH.GR4.POST.MC-LV2-4OA1) 10 Item Tag Codification Item #Subject Grade Post Item Type DoK Standard ID

1. Review specification tables, blueprint, and targeted standards. 2. Draft the item, passage, prompt, or scenarios. Insert any necessary tables, graphs, or images. 3. For MC items, create the single correct answer and then identify distractors, which reflect common errors/misinterpretations. For CR items/tasks, create a scoring “rubric” that articulates different levels of performance, including a sample response for each level. 4. Specify item/task unique ID: [ e.g., item number. subject. grade. post. item type-DoK level-content standard number ] 5. Repeat Steps 1-4 for the remainder of the items/tasks needed to complete the blueprint. 11 Process Steps

QA Checklist Types, point values, and DoKs match the blueprint. There are sufficient items/tasks to sample the targeted content. The items/tasks are developmentally appropriate for the intended test-takers. The correct answers and/or expected responses are clearly identified. Each item/task is assigned a unique ID. 12

Think-Pair-Share Based on the discussed item specifications, decide what is wrong with the following questions. 1. Some scientists believe that Pluto is __________. A. an escaped moon of Neptune B. usually the most distant planet in the solar system C. the name of Mickey Mouse’s dog D. all of the above 2. The people of Iceland __________. A. a country located just outside the Arctic Circle B. work to keep their culture alive C. claim to be descendants of the Aztecs D. the capital, Reykjavik, where arms talks have been held 13

Summary Module 3: Item Specifications Developed performance measure items/tasks that match the applicable blueprints. Next Steps Module 4: Scoring Keys and Rubrics Given the items/tasks created, develop scoring keys and rubrics. 14 Summary & Next Steps