Reviewing, Revising and Writing Mathematics Multiple- Choice Items 1 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing, Revising and Writing Effective Mathematics Multiple-Choice Items

Reviewing, Revising and Writing Effective Mathematics Multiple-Choice Items Learning Objective: To learn how to review, correct and write effective Mathematics multiple-choice items. 2 Writing and scoring test questions based on Ohio’s Academic Content Standards

Reviewing Mathematics Multiple-Choice Items Reviewing the Item 3 Writing and scoring test questions based on Ohio’s Academic Content Standards

Let’s review an item together and then I will hand out additional items that you can review in your groups. We will begin by reviewing a Grade 5 Mathematics item. 4 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Standard: Data Analysis and Probability Benchmark: Read, create and use line graphs, histograms, circle graphs, box-and-whisker plots, stem-and-leaf plots, and other representations when appropriate. Indicator: Determine appropriate data to be collected to answer questions posed by students or teacher, collect and display data, and clearly communicate findings. Answer Key: D Toni wants to know how large the families of her classmates are. The question that would be most appropriate for Toni to ask is: A. Do you have a father? B. How old is your mother? C. Do you have sisters? D. How many brothers and sisters do you have? 5 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Let’s follow the list on your Multiple Choice Items Review Checklist. Review Step 1: Is the item content correct, accurate and realistic? 6 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 2: Does the item match the standard, benchmark and indicator listed? 7 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 3: Is the item fair and free from bias and sensitivity issues? 8 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 4: Is the item written at or below the grade level of the students who will be taking the assessment? 9 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 5: Does the item assess significant information? 10 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 6: Is the item written in the passive voice? 11 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Review Step 7: Is the item succinct, and does it contain only the needed information? 12 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Reviewing the Stem 13 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Stems

Review Step 8: Does the item have a closed stem? 14 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Stems

Review Step 9: Does the item have a negative stem? 15 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Stems

Review Step 10: Does the item have a directed stem? 16 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Stems

Review Step 11: Does the stem avoid unnecessary technical language? 17 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Stems

Reviewing the Options 18 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Options

Review Step 12: Are the options the same length or ordered according to length, alpha order, or in numeric order? 19 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Options

Review Step 13: Do the options avoid words such as “all” or “only”? 20 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Options

Review Step 14: Are the options grammatically parallel? 21 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Options

Review Step 15: Do the options avoid repeated information? 22 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Options

Reviewing the Key 23 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Keys

Review Step 16: Does the key answer the question posed in the stem? 24 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Keys

Review Step 17: Is the key correct and the only correct option? 25 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Keys

Review Step 18: Does the key contain significant language from the stem? 26 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Keys

Review Step 19: Is the key obvious because it is the only positively or negatively worded option? 27 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Keys

Reviewing the Distractors 28 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Distractors

Review Step 20: Are the distractors plausible but incorrect? 29 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Distractors

Review Step 21: Do the distractors present logical misunderstandings or misconceptions? 30 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Distractors

Review Step 22: Do the distractors contain opposing statements? 31 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Distractors

Review Step 23: Do the distractors avoid using the words “all of the above” or “none of the above”? 32 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Distractors

As a result of our review, we now know that this item has several flaws. 33 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

ACTIVITY 7 Reviewing Items 34 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

35 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items Turn to Activity 7 in your binder. In ACTIVITY 7, we will be reviewing Mathematics items in our small groups.

Please begin reviewing the items. Remember to go through all 23 steps to detect all the flaws in the items. 36 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

When you have finished your review, we will share each group’s review comments with the other groups. 37 Writing and scoring test questions based on Ohio’s Academic Content Standards Reviewing Mathematics Multiple-Choice Items

Let’s go back to the first item we reviewed together. If you remember, we listed its flaws on the flip chart. 38 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

Let’s fix the item together. Let’s look at each flaw we have listed on the flip chart and revise the item to eliminate the flaw. 39 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

ACTIVITY 8 Revising Items to Correct the Flaws 40 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

Go back to the items you reviewed in your groups. Now that you have determined the flaws in these items, work in your groups to correct them. Make sure that after you have corrected a flaw you check to see that you have not introduced any new flaws. If you need to check the accuracy of the content in your items, please feel free to use the computer to do an Internet search. 41 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

If you turn to Activity 8 in your binder, you will see blank item cards. Use these to rewrite the flawed items. 42 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

After we have finished this activity, each group will share its corrections with the other groups. 43 Writing and scoring test questions based on Ohio’s Academic Content Standards Revising Mathematics Multiple-Choice Items

Now let’s try writing our own item for the fifth grade. Let’s write a multiple-choice item for the following standard, benchmark and indicator. Standard: Data Analysis and Probability Benchmark: Read, create and use line graphs, histograms, circle graphs, box-and-whisker plots, stem- and-leaf plots, and other representations when appropriate. Indicator: Determine appropriate data to be collected to answer questions posed by students or teacher, collect and display data, and clearly communicate findings. 44 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Mathematics Multiple-Choice Items

Can someone suggest a stem? Let’s review the stem and make sure it meets all our guidelines. Can someone suggest a key? Again, let’s review the key. Can someone suggest distractors? And let’s review them. 45 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Mathematics Multiple-Choice Items

ACTIVITY 9 Writing Effective Multiple-Choice Items 46 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Mathematics Multiple-Choice Items

Now we are going to write our own items. Please write a multiple-choice item to the standards, benchmarks and indicators listed under Activity 9 in your binder. 47 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Mathematics Multiple-Choice Items

After you have completed this activity, we will share your group’s items with the other groups. The other groups will act as the reviewers of your group’s items and comment on them using the 23 guidelines we have been using. 48 Writing and scoring test questions based on Ohio’s Academic Content Standards Writing Mathematics Multiple-Choice Items

Reviewing, Revising and Writing Mathematics Multiple-Choice Items We hope you have enjoyed the day and that you are now more confident that you can write effective multiple-choice items for your classroom tests. 49 Writing and scoring test questions based on Ohio’s Academic Content Standards