1. Writing Better MCQs

2. Why Multiple Choice Questions? Most commonly used format in the College Most commonly used format on licensing examinations This session will NOT cover essays, fill-in-the-blank items, true/false, or any other question types

3. Strengths of MCQ Can assessment achievement of learning from simple to complex More reliable than True/False items Scoring is easy, reliable, and objective Can cover a lot of material Zimmaro (2004)

4. Limitations of MCQs Time consuming to write good questions Acceptable distractors are sometimes hard to find Is not the best way to evaluate problem-solving Sometimes there is more than one correct answer May encourage guessing Zimmaro (2004)

5. Why Good Questions Matter Strengths of MCQs are only valid if constructed well Use in national licensure and credentialing examinations  Example: BPS examinations Well written items can assess more than recall of knowledge MCQs become “tricky” when rewritten from the original best format

6. Parts of a MCQ Stem – the text of the question Options – the choices provided after the stem Key – the CORRECT answer in the list of options Distracters – the INCORRECT answers in the list of options

7. Start with a Good Stem The stem should have a single, definitive statement or question It should be clear from the stem what is expected The majority of the information should be in the stem, not in the key Avoid unnecessary and irrelevant information Use clear, straightforward language Avoid use of negatives, and if you use them capitalize, underscore, or bold to make it clear what you are asking (i.e. EXCEPT, not )

8. What’s wrong with this stem? A nurse is assessing a client who has pneumonia. Which of these assessment findings indicates that the client does NOT need to be suctioned? a. Diminished breath sounds b. *Absence of adventitious breath sounds c. Inability to cough up sputum d. Wheezing following bronchodilator therapy Which of these assessment findings, if identified in a client who has pneumonia, indicates that the client needs suctioned? a. Absence of adventitious breath sounds b. Respiratory rate of 18 breaths/minute c. *Inability to cough up sputum d. Wheezing prior to bronchodilator therapy

9. What’s wrong with this stem? Mitochondria evolved from free- living bacteria that could carry our oxidative phosphorylation. For this reason, they have circular genomes that reproduce independently of the nuclear genome. What characteristic is relatively constant in mitochondrial genomes across species? a. Content (i.e., types of genes) b. Organization c. Size What characteristic is relatively constant in mitochondrial genomes across species? a. Content (i.e., types of genes) b. Organization c. Size

10. What’s wrong with this stem? Polysaccarides a. Are made up of thousands of smaller units called monosaccharides b. Are NOT found in the aloe vera leaf c. Are created during photosynthesis d. Can be described by the chmical formula CHHOH Polysaccharides of the plant cell wall are synthesized mainly in the a. Endoplasmic reticulum b. Cytosol c. Plasma membrane d. Golgi complex

11. Writing Keys and Distractors There should be only one best answer. Ideally there is only one right answer. Avoid K-type and A-type items that include multiple answers (i.e. All of the above or Both A and C ) Wrong answers should be plausible. Avoid funny or silly distractors. Keep the length of options the same. Vary the location of the correct answer. Maintain formatting and avoid overlapping answers. Don’t repeat words from the stem. Avoid grammar clues. Avoid absolute statements (e.g. Always, Never, All)

12. What’s wrong with the key/distractors? During what age period is thumb- sucking likely to produce the greatest psychological trauma? a. Infancy b. Preschool period c. Before adolescence d. During adolescence e. After adolescence During what age period is thumb- sucking likely to produce the greatest psychological trauma? a. From birth to 2 years old b. From 2 years to 5 years old c. From 5 years to 12 years old d. From 12 years to 20 years old

13. What’s wrong with the key/ distractors? Albert Einstein was a: a. Anthropologist b. Astronomer c. Chemist d. Mathematician Who was Albert Einstein? a. An anthropologist b. An astronomer c. A chemist d. A mathematician

14. What’s wrong with the key/ distractors? Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Warhol b. Flinstone c. Michelangelo d. Santa Claus Which of the following artists is known for painting the ceiling of the Sistine Chapel? a. Botticelli b. Da Vinci c. Michelangelo d. Raphael

15. What’s wrong with the key/ distractors? The term operant conditioning refers to the learning situation in which: a. A familiar response is associated with a new stimulus b. Individual associations are linked together in sequence c. *A response of the learner is instrumental in leading to a subsequent reinforcing event d. Verbal responses are made to verbal stimuli The term operant conditioning refers to the learning situation in which: a. A familiar response is associated with a new stimulus b. Individual associations are linked together in sequence c. *The learner’s response leads to reinforcement d. Verbal responses are made to verbal stimuli

16. What’s wrong with the key/ distractors? Who received a Noble Prize for discovering the structure of DNA? a. Francis Crick b. James Watson c. Rosalind Franklin d. A and B e. B and C f. A and C Who received a Noble Prize for discovering the structure of DNA? a. Paul D. Boyer b. Francis H. C. Crick c. Robert F. Curl Jr. d. Rosalind E. Franklin e. Dorothy C. Hodgkins f. Sir Robert Robinson

17. Tips and Pearls Good stem One-best answer Test application vs. recall of isolated facts Cover the options rule Options of similar length and structure Good distractors Avoiding  Always, never, rarely, usually, none of the above, all of the above

18. Time to Review Your Own Test

19. Resources Constructing Written Test Questions for the Basic and Clinical Sciences (NBME) Constructing Written Test Questions for the Basic and Clinical Sciences (NBME) Dos and Don’ts of Writing MCQs Good, Better, best: Multiple Choice Exam Construction Writing Good Multiple Choice Test Questions Writing Multiple Choice Questions: An Introductory Tutorial (NBME) Writing Multiple Choice Questions: An Introductory Tutorial (NBME)