Formatting the Answers: Avoiding Common Errors ABCRS Question Writing Workshop Glenn T. Ault, MD, MSEd Associate Professor of Surgery -Program Director –Colorectal Residency Senior Associate Dean, Clinical Administration (LAC+USC)- Univ. of Southern California 1

No disclosures

Options – all choices provided after the stem (includes the key and the distractors) The Key – the correct answer in the list of options The Distractors – the incorrect answers in the list of options Parts of the Answers:

Which of the following is a manifestation of adrenal insufficiency? A. Hypertension B. Hypokalemia C. Hypothermia D. Tachycardia

Well written stem is important Start with the key There should be one and only one correct answer or clearly the best answer on which experts would agree. Writing the Options

Generally more difficult than writing the stem Strategically designed to attract examinees who haven’t completely mastered the content and skill Plausible options Incorporate common misunderstandings or errors Make length of options similar (biased cueing) Writing Options - Distractors

Avoid negatives and/or double negatives Avoid “The Absolutes” - words as always, never and all Avoid highlighting any options with quotation marks or parenthesis Avoid vague frequency terms – sometimes, often, seldom, usually, frequently, likely No longer use format of questions that includes “all of the above”, “both a and c above”, or “none of the above” Writing Options - Distractors

Distractors – Common Flaws Overlapping Alternatives During which age period is thumb- sucking likely to produce the greatest psychological trauma? A.Infancy B.Preschool period C.Before adolescence D.During adolescence During which age period is thumb- sucking likely to produce the greatest psychological trauma? A.From birth to 2 years old B.From 2 to 5 years old C.From 5 to 12 years old D.From 12 to 20 years old

Distractors – Common Flaws “Grammatical Clues” Albert Einstein was a: A.Anthropologist B.Astronomer C.Chemist D.Mathmetician Who was Albert Einstein? A.An anthropologist B.An astronomer C.A chemist D.A mathmetician

Distractors – Common Flaws “Grammatical Clues” A 60-year-old man is brought to the emergency department after being found unconscious on the sidewalk. After verifying that the airway is open, the next step should be the IV administration of A.phenytoin B.diazepam C.Glucose with Vitamin B1 D.Tylenol

A word or phrase that is in the stem also appears in the correct answer A 58-year-old with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called A. Depersonalization B. Derailment C. Derealization D. Signal anxiety Distractors- Common Flaws “Clanging”

The options list three diagnostic studies and one method of treatment Test wise examinee will discard treatment and focus on diagnostics If treatment options used – escalate options from observation – medical management to surgery/ or least invasive to more invasive Distractors- Common Flaws “Non-parallel Structure”

Examples – Non-parallel Structure A 29-year old pregnant patient presents with right lower quadrant pain. What is the next best step? A.MRI B.CT-scan C.Ultrasound D.Laparoscopy A 29-year old pregnant patient presents with right lower quadrant pain. What is the next best step? A.KUB B.Ultrasound C.CT-scan D.MRI

We as authors tend to make the key longer because we want to include more detail, more information Fall into this trap because we want to qualify the answer to make it “completely” correct All answers should be similar in length to avoid this flaw… Options – Common Flaws Answer is the longest option (Bias Cueing)

Example –Bias Clueing Which of the following is a consequence of secondary gain? A.Previously unreported physical symptoms B.Prolonging of illness associated with advantages such as attention, relief from responsibility, or financial advantage C.Obsessive-Compulsive Disorder D.Organic brain damage Which of the following is a consequence of secondary gain? A.Previously unreported physical symptoms B.Prolonging of illness C.Obsessive-compulsive disorder D.Organic brain damage

What property of methadone differentiates it from other medications classified as narcotic analgesics? A. Methadone has a longer half-life B. Methadone has a shorter half-life C. Methadone is less likely to cause nausea D. Methadone is a narcotic antagonist Options – Common Flaws “Logical Clueing”

We tend to write the key first and then make the distractors plausible by making it partially correct. What is the total amount of kcal provided by one can of Ensure? A. 26 B. 150 C. 260 D Options – Common Flaws “Convergence”

Another example of convergence… Testing which nerves provides the most useful information to establish the diagnosis: A.Median motor, peroneal motor, facial, and sural B.Median motor, peroneal motor, tibial H reflex, and facial C.Median motor, ulnar motor, facial and sural D.Peroneal motor, tibial motor, peroneal F wave, and facial In addition to testing the median motor, peroneal motor, and facial nerves, evaluation of which of the following is likely to be most useful for establishing the diagnosis? A.Sural B.Ulnar motor nerve C.Tibial H reflex D.Peroneal F wave

Following a second episode of diverticulitis, what is the likelihood that another episode will occur? A. Less than 20% B. 20 to 30% C. Greater than 50% D. 90% We should expect examinees to demonstrate their knowledge, not encourage them to try and game the system through flawed test items. “Overlapping Numeric Options”

In a normal adult, what percent of blood volume loss will result in irreversible organ damage? A. 10% B. 20% C. 30% D. 40% In a normal adult, what is the minimum percent of blood volume loss that will result in irreversible organ damage? “Overlapping Numeric Options”

Options: Parallel in structure Fit logically and grammatically with the stem Worded concisely, clear and simple – delete unnecessary words! Inclusive that they logically eliminate another more restricted option from being a possible answer The “Answers”- Conclusions

The Key: Correct and best answer Answers the question posed in the stem Should not be too obvious relative to the other alternatives (should be shortened, lengthened, given greater number of details, made less concrete) The “Answers”- Conclusions

The Distractors: Be plausible but should not be justified as an acceptable correct answer Plausible enough to be attractive to examinees who are misinformed or ill-prepared Does not call attention to the key (should not merely state the reverse of the key, resembles key too closely, involves opposites) The “Answers”- Conclusions

Glenn T. Ault, MD, MSEd THANK YOU!