1. Writing Effective MCQs Marc J Kahn, MD, MBA Peterman-Prosser Professor Sr. Associate Dean Tulane University School of Medicine

2. Disclosures Nothing to disclose

3. There are no perfect questions, only perfect answers

4. MCQ Types A through R and X USMLE Step 1 only uses A-type, Step 2 also uses R- type A-type = single best answer MCQ, 4 or more options R-type = extended matching in sets of 2-20

5. K-type Hyperleuckocytosis is typically associated with (1)infection (2)acute leukemia (3)chronic leukemia (4)baseball A=1,2,3B=1,3 C=2,4 D=4E=all

6. Problem with multiple T/F Totally WrongTotally Correct 123123 4

7. Problem with multiple T/F Totally WrongTotally Correct 1313 4 2

8. Psychometrics How precise is my test? Kuder Richardson—tests reliability for measures with dichotomous choices. Ranges 0.00 to 1.00 KR=[N/(N-1)][1-{M(N-M)}]/(N*V) N= number of items M=arithmetic means of test scores V= variance of raw scores > 0.7 is good

9. Point Biserial Measures correlation of individual question with overall examinee performance How good is my question at distinguishing high from low performers? r pb =M 1 -M 0 /s n-1 * [n 1 n 0 /n(n-1)]^ 1/2 Want POSITIVE value (-1.00 to 1.00) >0.2 very good

10. TEST Time

11. MCQ rules Focus on potentially catastrophic or common problems Assess application of knowledge not a factoid Stem should be clear All distractors should be homogenous Avoid technical flaws

12. Anatomy of a MCQ STEM A.Distractor B.Distractor C.Correct Answer D.Distractor E.Distractor Lead in

13. Bad Stem Beta blockers are associated with what common side effect?

14. Better stem A 67 year old woman complains of one month duration of severe shortness of breath associated with a dry cough. Her symptoms are worse with exercise. She has been diagnosed with severe glaucoma that is being treated with timolol, dipivefrine, travaprost, and brimonidine eye drops. She is also taking oral acetazolamide. Which of her medications is most likely responsible for her symptoms?

15. Other STEM problems Too much information Unnecessary data Red herrings Should all follow same format: Age and gender Complaint and duration PMH Physical Labs

16. Lead-in problems Multiple negatives All of the following are not contraindications to beta blockers except for: Disguised TRUE/FALSE Which of the following are true about his condition?

17. Avoid Factoids

18. GLIB = GLAB

19. A glib is purveyed Which of the following are also purveyed: A.Binks B.Astons C. Glabs D.Wims E.Creys

20. What has been learned??

21. Answer Choices

22. The perils of “all” and “none of the above” A. mitochondria B. endoplasmic reticulum C. Nucleus D. None of the above E. All of the above

23. Schrodinger’s Cat DeadAlive Or in between?

24. Bad Distractor A.red blood cell B.lymphocyte C. neutrophil D.eosinophil E.basophil

25. Universality A.Prevalence is highest in childhood B.Prevalence is highest in adulthood C.Prevalence is highest in the elderly D.Is autosomal dominant E.Prevalence is highest in women

26. Non-homogeneity A.review peripheral smear B.give IV glucose C.discuss end of life care D. Consult a cardiologist E.order a blood gas

27. Other key points Perform psychometrics Keep a secure item bank Add and retire questions (~30% new questions per year) Share with peers for comment Exam item Review Committee Insure clinical relevance

28. It is better to give than to receive