Oscar Vergara Chihlee Institute of Technology July 28, 2014

  Lecturer at CIT since 2012  More than 10 years ESL/EFL teaching experience  Co-author of a variety of EFL textbooks & test data banks  Certified IELTS examiner About Me

  Some Guiding Principles  Multiple Choice Test Questions  Workshop Activity: Suggestions for Creating Good Tests  Q & A Today’s Content

  Clarity of questions & instructions  Reliability & Validity  Fairness Some Guiding Principles

  Instructions  Clearly state what is required  Use simple language / unambiguous  Questions / Stems  Content or structure should not prevent an informed student from answering correctly  Don’t include distracting or unnecessary details Clarity

  Do the exam questions measure what they purport to measure?  Do the questions test your course goals?  Does the exam accurately reflect the achievement of what you intended to teach?  Classes may differ, so different versions may be needed. Reliability & Validity

  Provide clear expectations about student performance  Provide examples / practice with mock tests or past tests  Students should know expectations of how their grade on tests reflects their skills (not extraneous factors) Fairness

  Difficult to achieve due to:  Demands on time  Grading resources  Require many versions Guiding Principles

  Taxonomy of Educational Objectives  Specifies different abilities and behaviors related to thinking processes  Contains 6 distinct categories Guiding Principles: Bloom’s Taxonomy

Knowledge Comprehension Application Analysis Synthesis Evaluation - Rote memorization - Recall -Interpret information -Test questions on facts, rules & principles - Apply concepts to situations - Distinguish or differentiate between ideas - Formulate or modify ideas - Assess, criticize, justify

  Knowledge: Remember  Knowledge of terms and concepts  Recall of information  Comprehension: Understand  Comprehension including translating, summarizing, demonstrating, discussing  Application: Apply  Apply what was learned  Use problem-solving methods Guiding Principles

  Please work in pairs or small groups  Do only part A  Discuss Worksheet

  Made up of a single question called a stem  Many possible choices with one correct answer  Several incorrect answers called distractors  Distractors are plausible but not possible Multiple choice

  Match your learning goals  Appropriate level of difficulty  Be aware of common errors How to Write Good Multiple Choice Stems

  Please work in groups of 3 or 4  Do part B  There is no single correct answer; only note what you think could be improved Worksheet: Group Activity

 1.More than one possibility; change (d) to different word formation 2.Double negative is confusing 3.Answers should all be of a similar length; (d) is too long and obvious 4.Too complex; time-consuming; frustrates some test- takers; item value Worksheet Part A: Suggestions

 5.Grammar can give away answer; always use a(n) as necessary 6.Negatives should be emphasized; ie – NOT, EXCEPT, etc. 7.Too many blanks; item value; misspelled or nonsensical/ non-existent words 8.Avoid absolutes including All, None or more than one answer Worksheet Part A: Suggestions

 Multiple Choice: Summary What to avoid in the stem  Long complex sentences  Negatives / Double negatives  Unintentional clues What to use in the stem  Your own words (if possible)  Single idea & clearly formulated question

 Multiple Choice: Summary What to avoid in the choices  Statements too close to being correct  Completely implausible answers  Absolute answers (ie – All of the above) What to use in the choices  Plausible & homogeneous distractors  Same option lengths  True statements that do not answer the question  Answers distributed evenly

  Composed only of statements with two possible answers  Assess familiarity with course content and general misconceptions  Test a range of broad concepts and can quickly respond  Easy to grade, but time-consuming to create True / False

 What to avoid  Negatives / Double negatives  Long complex sentences  Ambiguous or trivial material What to use  Your own words  50/50 or 60/40 in favor of false (students more likely to answer true)  One idea per item

  Contains equal number of stems and choices  Assess recognition and recall  Important if acquisition of detailed knowledge is a learning goal  Easy to grade, but students may require more time than equal number of m/c or t/f Matching

 What to avoid  Long stems and options  Heterogeneous content (ie – testing grammar and vocabulary)  Implausible responses What to use  Short responses; 10 to 15 items per page  Clear directions  Ordered choices (ie – alphabetical or chronological)

 Q & A

  Bloom, Benjamin S. Taxonomy of Educational Objectives (1956). Published by Allyn and Bacon, Boston, MA. Copyright (c) 1984 by Pearson Education.  Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer, Richard E.; Pintrich, Paul R.; Raths, James; Wittrock, Merlin C. (2000). Anderson, Lorin W.; Krathwohl, David R., eds. A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Allyn and Bacon. ISBN References

  Thank you!