1. Reviewing, Revising and Writing Effective Constructed- Response Mathematics Items 1 Writing and scoring test questions based on Ohio’s Academic Content Standards

2. Reviewing, Revising and Writing Constructed-Response Mathematics Items Learning Objective: To learn to review, correct and write constructed-response Mathematics items based on Ohio’s Academic Content Standards. 2 Writing and scoring test questions based on Ohio’s Academic Content Standards

3. Reviewing Constructed-Response Mathematics Items Reviewing the Item 3 Writing and scoring test questions based on Ohio’s Academic Content Standards

4. Let’s review an item together and then I will hand out additional items that you can review in your groups. We will begin by reviewing a Grade 8 Mathematics item. 4 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

5. Standard: Geometry and Spatial Sense Benchmark: Establish the validity of conjectures about geometric objects, their properties and relationships by counterexample, inductive and deductive reasoning, and critiquing arguments made by others. Indicator: Make and test conjectures about characteristics and properties (e.g., sides, angles, symmetry) of two-dimensional figures and three-dimensional objects. Joan says that she can determine whether the figure shown is a square by measuring one side. In your Answer Document, determine if Joan is correct. If so, which side? If not, explain why not. 5 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

6. Yes, because if CD is 5 inches, then it is a square, and if not, then it isn’t. PointsStudent Response 2The focus of this task is to test conjectures about characteristics and properties of a quadrilateral. The response provides the correct answer of yes and the correct side. 1The response shows partial evidence of how to test conjectures about characteristics and properties of a quadrilateral; however, the solution may be incomplete or slightly flawed. 1 point sample answer: For example, the response may: Provide the correct answer of yes, but no (or an incorrect) side. 0The response provides inadequate evidence of how to test conjectures about characteristics and properties of a quadrilateral. The response provides major flaws in explanations or irrelevant information. 0 point sample answer: For example, the response may: Restate the information provided in the item. Be blank or give irrelevant information. 6 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

7. Let’s follow the list on your Constructed Response Review Checklist. Review Step 1: Is the item content correct, accurate, and realistic? 7 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

8. Review Step 2: Does the item match the standard, benchmark and indicator listed? 8 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

9. Review Step 3: Is the item fair and free from bias and sensitivity issues? 9 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

10. Review Step 4: Is the item written at or below the grade level of the students who will be taking the assessment? 10 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

11. Review Step 5: Does the item assess significant information? 11 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

12. Review Step 6: Is the item written in the passive voice? 12 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

13. Review Step 7: Is the item succinct, and does it contain only the needed information? 13 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

14. Reviewing the Prompt 14 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

15. Review Step 8: Does the prompt contain unnecessary technical language? 15 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

16. Review Step 9: Does the prompt require a more complex response than a multiple-choice item permits? 16 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

17. Review Step 10: Does the prompt require more than a yes-or-no response? 17 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

18. Review Step 11: Does the prompt contain enough information to focus the student on the task that must be accomplished? 18 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

19. Review Step 12: Does the prompt indicate the amount and type of information that must be Included in the response? 19 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

20. Review Step 13: Does the prompt ask for multiple repetitions of the same task to gain additional points? 20 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

21. Review Step 14: Does the prompt indicate score-point dependencies? 21 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

22. Review Step 15: Does the prompt present a task that can be completed in the allotted time? 22 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Prompts

23. Reviewing the Rubric 23 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

24. Review Step 16: Is the rubric aligned with the prompt? 24 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

25. Review Step 17: Does the rubric permit multiple solutions in the response? 25 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

26. Review Step 18: Do the score-point descriptors clearly delineate what a student must do to earn each point? 26 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

27. Review Step 19: Does the rubric contain scoring rules that do not correspond to what is asked in the prompt? 27 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

28. Review Step 20: Does the rubric contain unambiguous response exemplars? 28 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Rubrics

29. As a result of our review, we now know that this item has several flaws. 29 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

30. ACTIVITY 5 Reviewing Items 30 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

31. In ACTIVITY 5, we will be reviewing Mathematics items in our small groups. Turn to Activity 5 in your binders. 31 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

32. Please begin reviewing the items. Remember to go through all 20 steps to detect all the flaws in the items. 32 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

33. When you have finished your review, we will share your group’s review comments with the other groups. 33 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Constructed-Response Mathematics Items

34. Let’s go back to the first item we reviewed together. If you remember, we listed the flaws on the flip chart. 34 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

35. Let’s fix the item together. Let’s look at each flaw we have listed on the flip chart and revise the item to eliminate the flaw. 35 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

36. ACTIVITY 6 Revising Items to Correct the Flaws 36 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

37. Go back to the items you reviewed in your groups. Now that you have determined the flaws in these items, work in your groups to correct them. Make sure that after you have corrected a flaw you check to see that you have not introduced any new flaws. If you need to check the accuracy of the content in your items, please feel free to use the computer to do an Internet search. 37 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

38. If you turn to Activity 6 in your binder, you will see blank item cards. Use these to rewrite the flawed items. 38 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

39. After we have finished this activity, each group will share their revisions with the other groups. 39 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Constructed-Response Mathematics Items

40. Now let’s try writing a 2-point constructed-response item for Grade 5. Let’s write a constructed-response item for the following standard, benchmark and indicator: Standard: Number, Number Sense and Operations Benchmark: Use order of operations, including use of parenthesis and exponents to solve multi-step problems, and verify and interpret the results. Indicator: Identify and use relationships between operations to solve problems. 40 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

41. Can someone suggest a prompt? Let’s review the prompt to make sure it meets all our guidelines. Can someone suggest an exemplar for the rubric? Can someone suggest wording for the 2-point score descriptor? For the other score point descriptors? Let’s review the rubric together to make sure it meets all our guidelines. 41 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

42. ACTIVITY 7 Writing Effective Constructed-Response Mathematics Items 42 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

43. Now we are going to write our own items. Please write a constructed- response item to the standards, benchmarks, and indicators listed in Activity 7 of your binder. 43 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

44. After you have completed this activity, we will share your group’s items with the other groups. The other groups will act as the reviewers of your group’s items and comment on them using the 20 guidelines we have been using. 44 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Constructed-Response Mathematics Items

45. We hope you have enjoyed the day and that you are now more confident that you can write effective constructed response items for your classroom tests. 45 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing, Revising and Writing Constructed-Response Mathematics Items