CONSTRUCTING OBJECTIVE TEST ITEMS: MULTIPLE-CHOICE FORMS CONSTRUCTING OBJECTIVE TEST ITEMS: MULTIPLE-CHOICE FORMS CHAPTER 8 AMY L. BLACKWELL JUNE 19, 2007

Objective test items are not limited to the measurement of simple learning outcomes. The multiple choice item can measure both knowledge and understanding levels and is free of many of the limitations of other forms of objective items. Objective test items are not limited to the measurement of simple learning outcomes. The multiple choice item can measure both knowledge and understanding levels and is free of many of the limitations of other forms of objective items. Chapter 8 covers: Characteristics of Multiple-Choice Items Characteristics of Multiple-Choice Items Uses of Multiple-Choice Items Uses of Multiple-Choice Items Advantages and Limitations of Multiple-Choice Items Advantages and Limitations of Multiple-Choice Items Suggestions for Constructing Multiple-Choice Items Suggestions for Constructing Multiple-Choice Items

Multiple-Choice Items Most widely used Most widely used Measure simple learning outcomes Measure simple learning outcomes Measure complex learning outcomes (knowledge, understanding, and application) Measure complex learning outcomes (knowledge, understanding, and application) Flexible, high quality items adaptable to most subject-matter content Flexible, high quality items adaptable to most subject-matter content Used extensively in achievement testing Used extensively in achievement testing

Characteristics of Multiple-Choice Items Consists of a problem (stem) and a list of suggested solutions (alternatives, choices, or options) Consists of a problem (stem) and a list of suggested solutions (alternatives, choices, or options) Answers other than the correct answer are called distracters (decoys or foils) Answers other than the correct answer are called distracters (decoys or foils) Items can be stated in two ways. Items can be stated in two ways. 1) Direct questions 1) Direct questions a) easier to write a) easier to write b) more natural for younger students b) more natural for younger students c) present a clearly formatted problem c) present a clearly formatted problem 2) Incomplete sentences 2) Incomplete sentences a) more concise a) more concise b) present a well defined problem if phrased well b) present a well defined problem if phrased well

Correct Answer Type and Best Answer Type Correct Answer Type and Best Answer Type The correct answer type has only one possible The correct answer type has only one possible correct answer (recall factual information). correct answer (recall factual information). The best answer type measures learning outcomes The best answer type measures learning outcomes that require the understanding, application, or that require the understanding, application, or interpretation of factual information (measures more interpretation of factual information (measures more complex learning and is more difficult). complex learning and is more difficult). When dealing with the best answer variety, make sure your best answers are those that are agreed on by experts. This will allow you to defend your answers as the best possible choice.

USES OF MULTIPLE-CHOICE ITEMS Measuring Knowledge Outcomes 1) Knowledge of Terminology 1) Knowledge of Terminology 2) Knowledge of Specific Facts 2) Knowledge of Specific Facts 3) Knowledge of Principles 3) Knowledge of Principles 4) Knowledge of Methods and Procedures 4) Knowledge of Methods and Procedures Measuring Outcomes at the Understanding and Application Levels 1) Ability to Identify Application of Fact and Principles 1) Ability to Identify Application of Fact and Principles 2) Ability to Interpret Cause-and-Effect Relationships 2) Ability to Interpret Cause-and-Effect Relationships 3) Ability to Justify Methods and Procedures 3) Ability to Justify Methods and Procedures

Advantages and Limitations of Multiple-Choice Items Advantages Measures achievement and complex learning outcomes. Measures achievement and complex learning outcomes. Structure of alternatives eliminate vagueness and ambiguity Structure of alternatives eliminate vagueness and ambiguity Knowledge of content area is measured without concern for spelling errors Knowledge of content area is measured without concern for spelling errors Multiple-choice requires students to choose the correct or best answer while true-false tests allow students to get credit for knowing a statement is not correct. Multiple-choice requires students to choose the correct or best answer while true-false tests allow students to get credit for knowing a statement is not correct. Multiple-choice items have a greater reliability than true-false Multiple-choice items have a greater reliability than true-false Multiple-choice items measure a single idea while matching exercises require a series of related ideas Multiple-choice items measure a single idea while matching exercises require a series of related ideas Multiple-choice items are usually free of response sets Multiple-choice items are usually free of response sets Incorrect answers in multiple-choice items can usually allow for diagnosis of errors and misunderstandings that need correction Incorrect answers in multiple-choice items can usually allow for diagnosis of errors and misunderstandings that need correction

Disadvantages Limited to outcomes at the verbal level Limited to outcomes at the verbal level Requires selection of the correct answer and therefore it does not measure problem solving skills in math and science or the ability to organize and present ideas Requires selection of the correct answer and therefore it does not measure problem solving skills in math and science or the ability to organize and present ideas It is difficult to find a sufficient number of reasonable alternatives or distracters (especially at the primary level) It is difficult to find a sufficient number of reasonable alternatives or distracters (especially at the primary level) Learning Exercise (page 209) Learning Exercise (page 209) #4 #4

Suggestions For Constructing Multiple Choice Items The stem of the item should be meaningful by itself and should present a definite problem. The stem of the item should be meaningful by itself and should present a definite problem. The item stem should include as much of the item as possible and should be free of irrelevant material. The item stem should include as much of the item as possible and should be free of irrelevant material. Use a negatively stated stem only when significant learning outcomes require it. Use a negatively stated stem only when significant learning outcomes require it. All the alternatives should be grammatically consistent with the stem of the item. All the alternatives should be grammatically consistent with the stem of the item. An item should contain only one correct or clearly best answer. An item should contain only one correct or clearly best answer. Items used to measure understanding should contain some novelty, but beware of too much. Items used to measure understanding should contain some novelty, but beware of too much. All distracters should be plausible. The purpose is to distract the uninformed from the correct answer. All distracters should be plausible. The purpose is to distract the uninformed from the correct answer. Verbal associations between the stem and the correct answer should be avoided. Verbal associations between the stem and the correct answer should be avoided. The relative length of the alternative should not provide a clue to the answer. The relative length of the alternative should not provide a clue to the answer. The correct answer should appear in each of the alternative positions an approximately equal number of times but in random order. The correct answer should appear in each of the alternative positions an approximately equal number of times but in random order. Use sparingly “none of the above” or “all of the above.” Use sparingly “none of the above” or “all of the above.” Do not use multiple-choice items when other items are more appropriate. Do not use multiple-choice items when other items are more appropriate.