Georgetown University Center for New Designs in Learning & Scholarship and Office of Faculty and Academic Affairs Writing Multiple Choice Questions Aviad Haramati, PhD Peggy A. Weissinger, EdD Georgetown University School of Medicine 9/14/2011 CNDLS & OFAA Georgetown University

Congruence! Objectives Activities Assessment Adapted from Felder & Brent ( 1991)

TLISI - Georgetown University What am I testing? Core Principles: Decide exactly what you want to test. Eliminate as many ambiguities as possible. Create distractors based firmly on what you want to test. After it is over, use it as a learning opportunity! Give feedback for your distracters which explains why they’re wrong. Give feedback for the correct answer explaining why it’s right. Adapted from http://hotpot.uvia.ca/howto/mcquestion.htm TLISI - Georgetown University

TLISI - Georgetown University Technical Flaws Issues Related to Irrelevant Difficulty Options long, complicated, or double Numeric data not stated consistently Terms in options are vague Language in options not parallel Options in non-logical order Stems are tricky or unnecessarily complicated Reference: Case & Swanson (NBME) Constructing Written Test Questions TLISI - Georgetown University

TLISI - Georgetown University Item Shape Appropriately Shaped Item Long Stem a. b. c. d. e. Poorly Shaped Item Short Stem a. b. c. d. 5/21/08 TLISI - Georgetown University e.

TLISI - Georgetown University When Writing Options... …make sure that they are: Homogenous in content Incorrect or inferior to the correct answer Plausible and attractive to the uninformed Similar to the correct answer in construction Grammatically consistent and logically compatible with the stem Reference: Case & Swanson (NBME) Constructing Written Test Questions TLISI - Georgetown University

TLISI - Georgetown University General Guidelines Bulk of question in the stem. Try “cover-the-options” test. Avoid superfluous information. Avoid “tricky” and overly complex items. Write grammatically consistent options. Avoid absolutes! (e.g. always, never, all) And most important: Focus on important concepts. Don’t waste time testing trivial facts. Reference: Case & Swanson (NBME) Constructing Written Test Questions TLISI - Georgetown University

Pulling it together! Summary: Decide what you want to test. Well-written objectives make the job easier. Use guidelines for writing quality MCQ. Local resources here to help!

Georgetown University Contact information Aviad Haramati haramati@georgetown.edu 202-687-1021 Peg Weissinger paw33@georgetown.edu 202-687-4556 9/14/2011 CNDLS & OFAA Georgetown University