Writing Meaningful Test Questions Kurt Gilliland, Ph.D. Dept. of Cell Biology and Physiology Ryan Madanick, M.D. Division of Gastroenterology and Hepatology Dept. of Medicine

Writing Meaningful Test Questions Format of stems, lead-ins, and options »What are common flaws to avoid? Content of stems, lead-ins, and options »How can your items reflect learning objectives? »How can you develop items that test application instead of recall?

1. The primary purpose of the stam is to remove which of the following? A. Carm B. Denton C. Menice D. Stam bar

Word repeat: A word or phrase is included in the stem and in the correct answer. Item Flaw ✔

2. Which of the following pairs has won the greatest number of Abby awards? A. Jones and Smith B. Smith and Taylor C. Smith and White D. White and Allen

Convergence strategy: The correct answer included the most elements in common with other options. Item Flaw ✔

3. How many pounds of pressure are exerted by a callam? A. 260 B. 150 C. 2.6 D. 2600

Language in the options is not parallel: Options are in an illogical order, in this case. Item Flaws ✔ Logical cues: A subset of the options are collectively exhaustible. Often, the correct answer is in the middle of the range.

4. The stanon is aided by a A. anstel B. immon C. octal D. port

Item Flaw ✔ Grammatical cues: One or more distractors don’t follow grammatically from the stem.

5. Why does the stanon frequently overheat? A. All grestels are belious. B. No immons are directly fectitious. C. Octals are usually casable. D. Ports are always actial.

Item Flaws ✔ Absolute terms: Terms such as “always” or “never” are used in options. Frequency terms in the topics are vague: Terms such as “rarely” or “usually” not consistently defined or interpreted, even by experts.

6. Stamation normally occurs when the… A. anstels rupture. B. immon falls and the denton is in place. C. octal rotates easily. D. ports pass over the carm.

Item Flaw ✔ Long correct answer: Correct answer is longer, more specific, or more complete than other options.

BONUS. Find 4 flaws. Regarding syphilis, all are true except… A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain. B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in T. pallidum infections but become negative in tertiary syphilis. C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic centers and a surrounding rim of plasma cells. D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta with aortitis and aneurysm. E. None of the above

Item Flaws 1. Short stem but long, heterogeneous options. 2. A negative term in the stem. 3. Unfocused stem. 4. Use of “none of the above” as option.

Frequency Terms What do terms like “usually” or “frequently” mean to you? What do terms like “usually” or “frequently” mean to you?

Frequency Terms

The most important prognostic factor in AIDS is… A. CD4:CD8 ratio B. presence of Kaposi’s sarcoma C. absolute leukocyte count D. absolute CD4 count E. level of GP120 antibody

Unfocused stem Use of absolute terms such as “most”

What is the cause of muscle soreness that is present the day or two after exercise? A. Accumulation of lactic acid B. Microscopic muscle damage leading to an inflammatory response including the release of bradykinin, PGE2, and histamine which sensitive nerve endings C. Muscle stiffness D. Use of anabolic steroids

Long distractor Heterogeneous distractors

The tophi of gouty arthritis: A. are a form of erythema nodosum B. have a fibrinoid center surrounded by peripheral histiocytes C. are composed of prominent osteophytes at the DIP joints of fingers D. none of the above

Unfocused stem Short stem, long distractors “None of the above”

Sigmund Freud’s Interpretation of Dreams was published in what year? A B C D E. 1915

Correct answer (1913) in middle of range Question is recall of minutia and not application

Aspirin should NOT be prescribed when the patient has which of the following conditions? A. Peptic ulcers B. Headache C. Fever D. Pain E. Inflammation

Use of “not” is acceptable in this case since the teaching point was in fact a contraindication.

Which of the following is true regarding lysosomal storage disease? A. 会意字 / 會意字 B. ربي/عربى C. 日本語 D. में जन्मजात स्वतन्त्रता और

Unfocused stem