Writing Effective MCQs Marc J Kahn, MD, MBA Peterman-Prosser Professor Sr. Associate Dean Tulane University School of Medicine

Disclosures Nothing to disclose

There are no perfect questions, only perfect answers

MCQ Types A through R and X USMLE Step 1 only uses A-type, Step 2 also uses R- type A-type = single best answer MCQ, 4 or more options R-type = extended matching in sets of 2-20

K-type Hyperleuckocytosis is typically associated with (1)infection (2)acute leukemia (3)chronic leukemia (4)baseball A=1,2,3B=1,3 C=2,4 D=4E=all

Problem with multiple T/F Totally WrongTotally Correct

Problem with multiple T/F Totally WrongTotally Correct

Psychometrics How precise is my test? Kuder Richardson—tests reliability for measures with dichotomous choices. Ranges 0.00 to 1.00 KR=[N/(N-1)][1-{M(N-M)}]/(N*V) N= number of items M=arithmetic means of test scores V= variance of raw scores > 0.7 is good

Point Biserial Measures correlation of individual question with overall examinee performance How good is my question at distinguishing high from low performers? r pb =M 1 -M 0 /s n-1 * [n 1 n 0 /n(n-1)]^ 1/2 Want POSITIVE value (-1.00 to 1.00) >0.2 very good

TEST Time

MCQ rules Focus on potentially catastrophic or common problems Assess application of knowledge not a factoid Stem should be clear All distractors should be homogenous Avoid technical flaws

Anatomy of a MCQ STEM A.Distractor B.Distractor C.Correct Answer D.Distractor E.Distractor Lead in

Bad Stem Beta blockers are associated with what common side effect?

Better stem A 67 year old woman complains of one month duration of severe shortness of breath associated with a dry cough. Her symptoms are worse with exercise. She has been diagnosed with severe glaucoma that is being treated with timolol, dipivefrine, travaprost, and brimonidine eye drops. She is also taking oral acetazolamide. Which of her medications is most likely responsible for her symptoms?

Other STEM problems Too much information Unnecessary data Red herrings Should all follow same format: Age and gender Complaint and duration PMH Physical Labs

Lead-in problems Multiple negatives All of the following are not contraindications to beta blockers except for: Disguised TRUE/FALSE Which of the following are true about his condition?

Avoid Factoids

GLIB = GLAB

A glib is purveyed Which of the following are also purveyed: A.Binks B.Astons C. Glabs D.Wims E.Creys

What has been learned??

Answer Choices

The perils of “all” and “none of the above” A. mitochondria B. endoplasmic reticulum C. Nucleus D. None of the above E. All of the above

Schrodinger’s Cat DeadAlive Or in between?

Bad Distractor A.red blood cell B.lymphocyte C. neutrophil D.eosinophil E.basophil

Universality A.Prevalence is highest in childhood B.Prevalence is highest in adulthood C.Prevalence is highest in the elderly D.Is autosomal dominant E.Prevalence is highest in women

Non-homogeneity A.review peripheral smear B.give IV glucose C.discuss end of life care D. Consult a cardiologist E.order a blood gas

Other key points Perform psychometrics Keep a secure item bank Add and retire questions (~30% new questions per year) Share with peers for comment Exam item Review Committee Insure clinical relevance

It is better to give than to receive