Ginny Price CETL TEST DEVELOPMENT

WRITING MULTIPLE CHOICE ITEMS Write only a few items at a time Immediately after preparing class lesson or after class discussions Focus on important points Spread out work time increasing depth of questions Build a bank of questions on Angel Include a few new items each time test is given Use item analysis to determine worth of test items

ANATOMY OF ITEM Stem Responses Correct response Three or four plausible distractors

STEM Students can read and understand stem Students can formulate an answer without reading responses Can be written as an incomplete sentence or a question Focus on important concept or principle Trivial details or obscure facts should not be used

DISTRACTORS Use four responses unless doing so would require using distractors that are implausible Use student misconceptions and common errors to write plausible distractors Give a short answer quiz to obtain sample of students’ incorrect responses to use as distractors

GUIDELINES Stem presents problem clearly and concisely Include a verb in the statement Contains all information necessary to understand problem Responses should not contain repetitive words or phrases. Include these items in the stem Write correct response first then distractors Best answer (reduces disagreements) Agreed upon answer by authorities in field Correct answer Facts: anatomy, terminology

GUIDELINES 2 Avoid giving clues to answers Measure test taking skills not learning Reduces test’s validity Types of clues Length of responses: correct answer is either longer or shorter than distractors Verbal association: using similar words in stem and correct response Grammatical clue: correct answer is only grammatically correct one Specific determiners: always, never, all, none Instead use: usually, sometimes, typically, maybe

GUIDELINES 3 Use positive statements as negative statements confuse students If negative statements are used the negative word should be bold and in caps Refrain from using all of the above too easy often the correct response Refrain from using none of the above options Do not use with negative stem; creates confusing double negative Does not reinforce correct answer as correct answer is not present Use only with correct answer format instead of best answer (absolutes) Can be used to increase item difficulty

GUIDELINES 4 Each item is independent No clues in one item for other items No item dependent on previous item’s response For responses: use a logical order or randomize Avoids patterns Response “A” rarely correct answer; “C” most often correct answer Cite authority in stem if item controversial Avoid quoting source verbatim in item Encourages memorization instead of understanding Arrange responses in vertical column

CONCENTRATIONS Purpose of test Reading material Understanding material Synthesizing material Applying material Analyzing material Can adjust weights of items based on purpose Important concepts and principles in course Develop novel or thought provoking situations in items Use students’ misconceptions as distractors

TECHNIQUES Premise → consequences Premise + premise → consequence Analogy Classification Proposition 1 & proposition 2 → 1 correct but not 2, 2 correct but not 1, both correct, both incorrect Context dependent item set

PREMISE → CONSEQUENCES p → c A premise (stem) Followed by consequences (responses) Evaluate Example: What could happen if a liquid is instilled into an animal’s external ear canal without determining the patency of the animal’s tympanic membrane? Damage to the animal’s hearing What must be examined before instilling substances into an animal’s external ear canal? Patency of the tympanic membrane

PREMISE + PREMISE → CONSEQUENCE p + p → c Premise + premise (stem) Followed by a consequence (responses) Evaluate Example: If an animal presented with a body temperature of 104.0°F and they walked to the hospital with their owner on a 90°F day it can be concluded the hyperthermia was caused by Environmental factors Stress Infection Shock

ANALOGY a : b = c : d a is to b as c is to d Precise relationships between two sets of words Analysis Example: Operant conditioning is to voluntary as classical conditioning is to Involuntary Association Habituation Salivating

CLASSIFICATION Classifies terms, names and statements Comprehension Example Pavlov would be classified as a Behaviorist Physiologist Biologist Sociologist

P1 P2 → 1 CORRECT BUT NOT 2, 2 CORRECT BUT NOT 1, BOTH CORRECT, BOTH INCORRECT Two propositions are presented in the stem. It must be decided whether both are true, neither are true, A but not B is true or B but not A is true. Example: A fearful dog with passive defense reflexes comes into the hospital for an exam. This dog will exhibit (A) appeasement body language (B) agonistic body language. A. A but not B B. B but not A C. neither A nor B D. both A and B

CONTEXT DEPENDENT ITEM SET A stimulus is used to generate a set of questions Written scenario, graph, chart, map, picture, table, article Measures high level cognitive skills: comprehension, analysis and application Use a novel stimulus to gain an idea of student’s quality of learning Example on next slide

The purpose of the porous tape in this picture is to prevent the catheter from sliding out of the bladder allow sutures to be placed attaching the tape to the skin prevent the animal from chewing out the catheter allow the technician to better monitor catheter length when inserting it CONTENT DEPENDENT ITEM

RESOURCES Jacobs. L.C. & Chase, C.I. (1992). Developing and using tests effectively: A guide for faculty.