1. 1 Testing and Grading Ensuring that we meet the objectives

2. 2 Desired Learning Levels Teaching Philosophy Students AbilitiesCourse Level and Scope Write Course and Topic Objectives Select Effective Teaching Methods How Students Learn BA 9200 Course Overview The Lecture Active Learning Testing and Grading

3. 3 Testing Overview A. Preparing Exams 1.Type of Tests 2.Number of Tests 3.Questions per Test, and 4.Table of Specification B. Writing Questions 1.Effective Stems and Distracters 2.Structured Essay Questions and Answers C. Grading 1.Scoring Key and Partial Credit for Multiple Choice D. Improving Exams 1.Survey and Difficulty and Discrimination Indices

4. 4 A. Preparing Exams 1. Type of Tests Norm BasedCriterion Based Goal Sort and Classify StudentsAssess Student Competency Scores Must Have Large Spread in Scores Spread in Scores Not Relevant Difficulty Questions Must Range from Easy to Difficult Questions Must Reflect Demands of Profession Timing All Students May Not Finish Exam All (or Most) Should Finish Exam

5. 5 A. Preparing Exams 2. Number of Tests Not Effective: One Exam at the end Effective: Unit Tests/Quizzes followed by a final exam.

6. 6 A. Preparing Exams 3. Criterion-Based Tests: Number of Questions per Test Rote Multiple choice Short Answer 2 minutes 4 minutes MI Level Multiple Choice Short Answer 4 minutes 8-10 minutes Critical Thinking Multiple Choice Short Answer 10-15 minutes 15-30 minutes Rough guideline for time per question at each level.

7. 7 A. Preparing Exams 4. Table of Specification RoteExerciseMI Measures of Center 15%25% Measures of Spread 10%15%10% Shape of Distribution 10%15%

8. 8 Testing Overview A. Preparing Exams 1.Type of Tests 2.Number of Tests 3.Questions per Test, and 4.Table of Specification B. Writing Questions 1.Effective Stems and Distracters 2.Structured Essay Questions and Answers C. Grading 1.Scoring Key and Partial Credit for Multiple Choice D. Improving Exams 1.Survey and Difficulty and Discrimination Indices

9. 9 B. Writing Questions 1. Multiple Choice Questions Essay Questions a.Require the Student to Remember Information. b.Are Scored More Reliably. c.Are More Difficult to Construct. d.Are Easier to Construct. The Median is a.the Average Value. b.the 25% Percentile Value. c.the 75% Percentile Value. d.that Number such that Approximately 50% of the Numbers Are Higher and 50% are Lower than the Median Value.

10. 10 B. Writing Questions Multiple Choice Questions The Term Used in Connection with Sound that Corresponds to Color with Reference to Light is a.Pitch. b.Hue. c.Timbre. d.Intensity. One Basic Advantage of Using Short Answer Essay Questions as Opposed to Long Answer Ones is that Short Answer Essay Items a.Generating Questions is Easier. b.Allow for a Broader Sample of Questions. c.Students Prefer Them. d.They Relate More Closely to the Course Objectives.

11. 11 B. Writing Questions Writing Effective Stems Ideal Stem Answerable Without Seeing Choices. Questions Address Important Concepts. Relate Each Question to One or More Topic Objectives. Question Stem Followed by Cognitive Objectives that Question Addresses{O12, O15} Cite Authority in Stem where applicaple

12. 12 B. Writing Questions Writing Effective Stems Solve Math Problems to Ensure All Data in Stem. State Stem in Positive Language. Same Phrase Appears in Each Answer, Shift It to Stem. Use Complete Sentence Format.

13. 13 B. Writing Questions Writing Effective Distractors Use Best-Answer Format. Write Best or Correct Answer First. For Concept Question Correct Answer to Similar Concept. Have Correct Elements, But Also Incorrect Elements From Students' Answers to Essay Version of Question.

14. 14 B. Writing Questions Writing Effective Distractors For Math Question Reflect Mistakes That Students Make. Use "Not Sufficient Information to Solve Problem" Option. Avoid “None of Above” for Math Problems

15. 15 B. Writing Questions Writing Effective Distracters Correct Answer Should Not Be Longest Answer. All Distractors Should Form Complete and Grammatically-Correct Sentences With Stem. Avoid Using Never or Always. Avoid an “All of the Above” Option. Avoid Repeating Words From Stem. Vary Correct Answer Among Four Possible Answer Positions.

16. 16 B. Writing Questions 2. Writing Short-Answer Questions Base on Objectives. Use “Structured Essay” Format. Uses Precise Verbs. Helps Students Frame Answer. Use Iterative Procedure. Write Ideal Answer. Could Students Achieve Ideal Given Question Wording? Questions Address Important Concepts. Solve Math Problems.

17. 17 Compare and Contrast “Amadeus” and “The Shawshank Redemption”. John Locke: The Key to David Hume? Discuss. Is the Central Limit Theorem Critical to Constructing a Confidence Interval on a Population Mean? Discuss and Defend. B. Writing Questions Unstructured Short-Answer Questions

18. 18 B. Writing Questions The All-Bluff Answer and the Need for the Structured Format  Is Central Limit Theorem Critical to Constructing a Confidence Interval on a Population Mean?  Dr. Deming Noted that the Central Limit Theorem is Essential.  As You Said in Class, “The Central Limit Theorem is a Critical Topic.”  The Central Limit Theorem Says that ……..  Finally, Hypothesis Testing is Similar to Confidence Intervals in that...

19. 19 B. Writing Questions Structured Answer Format Compare and Contrast “Amadeus” and “The Shawshank Redemption”. Basis 1 is _________ Basis 1 is Important Because _________ Basis 2 is __________ Basis 2 is Important Because __________

20. 20 B. Writing Questions Structured Answer Format Students then Complete the Table.

21. 21 Can the Bias in a forecast ever be greater than the MAD value? [O15] 10 Points Select One Response Below and Complete. Yes Because No Because B. Writing Questions Structured Answer Format

22. 22 Suppose the Central Limit Theorem was not true. Would It be Possible to Construct a Confidence Interval on a Population Mean? [O23] [15 Points] Select One of the Responses Below and Complete. Would Never be Possible Because Would Always be Possible Because Would Only be Possible When B. Writing Questions Structured Answer Format

23. 23 Suggest a modification to the analogy method that would increase the likelihood that a Sensing-dominant manager would use the Method. [O45, O56] [25 Points] Describe the Modification. This Modification Increases the Likelihood that a Sensing-Dominant Manager Would Use the Analogy Method Because … B. Writing Questions Structured Answer Format

24. 24 Testing Overview A. Preparing Exams 1.Type of Tests 2.Number of Tests 3.Questions per Test, and 4.Table of Specification B. Writing Questions 1.Effective Stems and Distracters 2.Structured Essay Questions and Answers C. Grading 1.Scoring Key and Partial Credit for Multiple Choice D. Improving Exams 1.Survey and Difficulty and Discrimination Indices

25. 25 C. Grading Grading Issues for All Types of Exams Written Comments Are Highly Motivating. “Excellent Improvement” “Sorry Did So Poorly. Please See Me.” Return Within One Class (MC) or One Week (Short Answer). Provide Key After Quiz, if Possible.

26. 26 C. Grading Grading Issues on Multiple Choice Exams Partial Credit for Multiple Choice? On Numerical Problems: Provide Optional White Space and Only Check if Wrong Option Selected. Defend Concept Answer to Complex Question: Provide Optional White Space and Only Check if Wrong Option Selected.

27. 27 C. Grading Grading Issues on Short Answer Exams For Small Classes, Provide Audio Feedback. Students Hand In Audio Cassette to Instructor. Write Annotated Comments on Exam. Use a Detailed Scoring Key. Grade One Question at a Time. Shuffle Papers After Each Question.

28. 28 Testing Overview A. Preparing Exams 1.Type of Tests 2.Number of Tests 3.Questions per Test, and 4.Table of Specification B. Writing Questions 1.Effective Stems and Distracters 2.Structured Essay Questions and Answers C. Grading 1.Scoring Key and Partial Credit for Multiple Choice D. Improving Exams 1.Survey and Difficulty and Discrimination Indices

29. 29 D. Improving Exams Use Test Assessment Instrument During Term. 1.Have exams reflected topic objectives? 2.Have exams reflected important subject content? 3.Has there been sufficient time to complete exams? 4.Has instructor reacted positively to student concerns on questions, grading, etc.?__________________________________________ 1234 5 Not at All Moderately Very Much 5.How could the exams be improved?

30. 30 Additional Readings Bloom, B., Hastings, T., and Madaus, G. Handbook of Formative and Summative Evaluation for Student Learning. New York: McGraw Hill, 1971. Jacobs, L and Chase, C. Developing and Using Tests Effectively: A Guide for Faculty. San Francisco, CA.: Jossey- Bass, 1992.

31. 31 Desired Learning Levels Teaching Philosophy Students AbilitiesCourse Level and Scope Write Course and Topic Objectives Select Effective Teaching Methods How Students Learn BA 9200 Course Overview The Lecture Active Learning Testing and Grading