1. Distractor Efficiency
Multiple-choice questions

2. NRT Item Analysis What is the purpose of our test items?
To spread people out
If every student gets an item correct, what does it tell us?
Nothing
If every student gets an item incorrect, what does it tell us?

3. NRT Item Selection Choose items with: Discard items with:
IF between .30 and .70.
ID as high as possible
Discard items with:
IF above .80 or below .20.
Negative ID
BUT, always consider actual item as well as statistics

4. Distractor efficiency analysis
Is each distractor ‘pulling its weight’ on the test?
Which distractors are ‘more proficient’ students selecting?
Which distractors are ‘less proficient’ students selecting?
Are poor distractors hurting an otherwise good item?

5. Brown (p. 72)