1. Oscar Vergara Chihlee Institute of Technology July 28, 2014

2.   Lecturer at CIT since 2012  More than 10 years ESL/EFL teaching experience  Co-author of a variety of EFL textbooks & test data banks  Certified IELTS examiner About Me

3.   Some Guiding Principles  Multiple Choice Test Questions  Workshop Activity: Suggestions for Creating Good Tests  Q & A Today’s Content

4.   Clarity of questions & instructions  Reliability & Validity  Fairness Some Guiding Principles

5.   Instructions  Clearly state what is required  Use simple language / unambiguous  Questions / Stems  Content or structure should not prevent an informed student from answering correctly  Don’t include distracting or unnecessary details Clarity

6.   Do the exam questions measure what they purport to measure?  Do the questions test your course goals?  Does the exam accurately reflect the achievement of what you intended to teach?  Classes may differ, so different versions may be needed. Reliability & Validity

7.   Provide clear expectations about student performance  Provide examples / practice with mock tests or past tests  Students should know expectations of how their grade on tests reflects their skills (not extraneous factors) Fairness

8.   Difficult to achieve due to:  Demands on time  Grading resources  Require many versions Guiding Principles

9.   Taxonomy of Educational Objectives  Specifies different abilities and behaviors related to thinking processes  Contains 6 distinct categories Guiding Principles: Bloom’s Taxonomy

11. Knowledge Comprehension Application Analysis Synthesis Evaluation - Rote memorization - Recall -Interpret information -Test questions on facts, rules & principles - Apply concepts to situations - Distinguish or differentiate between ideas - Formulate or modify ideas - Assess, criticize, justify

12.   Knowledge: Remember  Knowledge of terms and concepts  Recall of information  Comprehension: Understand  Comprehension including translating, summarizing, demonstrating, discussing  Application: Apply  Apply what was learned  Use problem-solving methods Guiding Principles

13.   Please work in pairs or small groups  Do only part A  Discuss Worksheet

14.   Made up of a single question called a stem  Many possible choices with one correct answer  Several incorrect answers called distractors  Distractors are plausible but not possible Multiple choice

15.   Match your learning goals  Appropriate level of difficulty  Be aware of common errors How to Write Good Multiple Choice Stems

16.   Please work in groups of 3 or 4  Do part B  There is no single correct answer; only note what you think could be improved Worksheet: Group Activity

17.  1.More than one possibility; change (d) to different word formation 2.Double negative is confusing 3.Answers should all be of a similar length; (d) is too long and obvious 4.Too complex; time-consuming; frustrates some test- takers; item value Worksheet Part A: Suggestions

18.  5.Grammar can give away answer; always use a(n) as necessary 6.Negatives should be emphasized; ie – NOT, EXCEPT, etc. 7.Too many blanks; item value; misspelled or nonsensical/ non-existent words 8.Avoid absolutes including All, None or more than one answer Worksheet Part A: Suggestions

19.  Multiple Choice: Summary What to avoid in the stem  Long complex sentences  Negatives / Double negatives  Unintentional clues What to use in the stem  Your own words (if possible)  Single idea & clearly formulated question

20.  Multiple Choice: Summary What to avoid in the choices  Statements too close to being correct  Completely implausible answers  Absolute answers (ie – All of the above) What to use in the choices  Plausible & homogeneous distractors  Same option lengths  True statements that do not answer the question  Answers distributed evenly

21.   Composed only of statements with two possible answers  Assess familiarity with course content and general misconceptions  Test a range of broad concepts and can quickly respond  Easy to grade, but time-consuming to create True / False

22.  What to avoid  Negatives / Double negatives  Long complex sentences  Ambiguous or trivial material What to use  Your own words  50/50 or 60/40 in favor of false (students more likely to answer true)  One idea per item

23.   Contains equal number of stems and choices  Assess recognition and recall  Important if acquisition of detailed knowledge is a learning goal  Easy to grade, but students may require more time than equal number of m/c or t/f Matching

24.  What to avoid  Long stems and options  Heterogeneous content (ie – testing grammar and vocabulary)  Implausible responses What to use  Short responses; 10 to 15 items per page  Clear directions  Ordered choices (ie – alphabetical or chronological)

25.  Q & A

26.   Bloom, Benjamin S. Taxonomy of Educational Objectives (1956). Published by Allyn and Bacon, Boston, MA. Copyright (c) 1984 by Pearson Education.  Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer, Richard E.; Pintrich, Paul R.; Raths, James; Wittrock, Merlin C. (2000). Anderson, Lorin W.; Krathwohl, David R., eds. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Allyn and Bacon. ISBN 978-0-8013-1903-7. References

27.   E-mail: oscarv@mail.chihlee.edu.tw Thank you!