Reviewing, Revising and Writing Effective Constructed- Response Mathematics Items 1 Writing and scoring test questions based on Ohio’s Academic Content Standards

Reviewing, Revising and Writing Constructed-Response Mathematics Items Learning Objective: To learn to review, correct and write constructed-response Mathematics items based on Ohio’s Academic Content Standards. 2 Writing and scoring test questions based on Ohio’s Academic Content Standards

Reviewing Constructed-Response Mathematics Items Reviewing the Item 3 Writing and scoring test questions based on Ohio’s Academic Content Standards

Let’s review an item together and then I will hand out additional items that you can review in your groups. We will begin by reviewing a Grade 8 Mathematics item. 4 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Standard: Geometry and Spatial Sense Benchmark: Establish the validity of conjectures about geometric objects, their properties and relationships by counterexample, inductive and deductive reasoning, and critiquing arguments made by others. Indicator: Make and test conjectures about characteristics and properties (e.g., sides, angles, symmetry) of two-dimensional figures and three-dimensional objects. Joan says that she can determine whether the figure shown is a square by measuring one side. In your Answer Document, determine if Joan is correct. If so, which side? If not, explain why not. 5 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Yes, because if CD is 5 inches, then it is a square, and if not, then it isn’t. PointsStudent Response 2The focus of this task is to test conjectures about characteristics and properties of a quadrilateral. The response provides the correct answer of yes and the correct side. 1The response shows partial evidence of how to test conjectures about characteristics and properties of a quadrilateral; however, the solution may be incomplete or slightly flawed. 1 point sample answer: For example, the response may: Provide the correct answer of yes, but no (or an incorrect) side. 0The response provides inadequate evidence of how to test conjectures about characteristics and properties of a quadrilateral. The response provides major flaws in explanations or irrelevant information. 0 point sample answer: For example, the response may: Restate the information provided in the item. Be blank or give irrelevant information. 6 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Let’s follow the list on your Constructed Response Review Checklist. Review Step 1: Is the item content correct, accurate, and realistic? 7 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 2: Does the item match the standard, benchmark and indicator listed? 8 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 3: Is the item fair and free from bias and sensitivity issues? 9 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 4: Is the item written at or below the grade level of the students who will be taking the assessment? 10 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 5: Does the item assess significant information? 11 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 6: Is the item written in the passive voice? 12 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Review Step 7: Is the item succinct, and does it contain only the needed information? 13 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Reviewing the Prompt 14 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 8: Does the prompt contain unnecessary technical language? 15 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 9: Does the prompt require a more complex response than a multiple-choice item permits? 16 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 10: Does the prompt require more than a yes-or-no response? 17 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 11: Does the prompt contain enough information to focus the student on the task that must be accomplished? 18 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 12: Does the prompt indicate the amount and type of information that must be Included in the response? 19 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 13: Does the prompt ask for multiple repetitions of the same task to gain additional points? 20 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 14: Does the prompt indicate score-point dependencies? 21 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Review Step 15: Does the prompt present a task that can be completed in the allotted time? 22 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

Reviewing the Rubric 23 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

Review Step 16: Is the rubric aligned with the prompt? 24 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

Review Step 17: Does the rubric permit multiple solutions in the response? 25 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

Review Step 18: Do the score-point descriptors clearly delineate what a student must do to earn each point? 26 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

Review Step 19: Does the rubric contain scoring rules that do not correspond to what is asked in the prompt? 27 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

Review Step 20: Does the rubric contain unambiguous response exemplars? 28 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

As a result of our review, we now know that this item has several flaws. 29 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

ACTIVITY 5 Reviewing Items 30 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

In ACTIVITY 5, we will be reviewing Mathematics items in our small groups. Turn to Activity 5 in your binders. 31 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Please begin reviewing the items. Remember to go through all 20 steps to detect all the flaws in the items. 32 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

When you have finished your review, we will share your group’s review comments with the other groups. 33 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

Let’s go back to the first item we reviewed together. If you remember, we listed the flaws on the flip chart. 34 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

Let’s fix the item together. Let’s look at each flaw we have listed on the flip chart and revise the item to eliminate the flaw. 35 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

ACTIVITY 6 Revising Items to Correct the Flaws 36 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

Go back to the items you reviewed in your groups. Now that you have determined the flaws in these items, work in your groups to correct them. Make sure that after you have corrected a flaw you check to see that you have not introduced any new flaws. If you need to check the accuracy of the content in your items, please feel free to use the computer to do an Internet search. 37 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

If you turn to Activity 6 in your binder, you will see blank item cards. Use these to rewrite the flawed items. 38 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

After we have finished this activity, each group will share their revisions with the other groups. 39 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

Now let’s try writing a 2-point constructed-response item for Grade 5. Let’s write a constructed-response item for the following standard, benchmark and indicator: Standard: Number, Number Sense and Operations Benchmark: Use order of operations, including use of parenthesis and exponents to solve multi-step problems, and verify and interpret the results. Indicator: Identify and use relationships between operations to solve problems. 40 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

Can someone suggest a prompt? Let’s review the prompt to make sure it meets all our guidelines. Can someone suggest an exemplar for the rubric? Can someone suggest wording for the 2-point score descriptor? For the other score point descriptors? Let’s review the rubric together to make sure it meets all our guidelines. 41 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

ACTIVITY 7 Writing Effective Constructed-Response Mathematics Items 42 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

Now we are going to write our own items. Please write a constructed- response item to the standards, benchmarks, and indicators listed in Activity 7 of your binder. 43 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

After you have completed this activity, we will share your group’s items with the other groups. The other groups will act as the reviewers of your group’s items and comment on them using the 20 guidelines we have been using. 44 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

We hope you have enjoyed the day and that you are now more confident that you can write effective constructed response items for your classroom tests. 45 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing, Revising and Writing Constructed-Response Mathematics Items