2. Biology Partnership 2012-2013 Assessment Pedagogy Session Saturday, September 29, 2012 Dr. Susan Butler

3. FEAPs: 4. Assessment Analyzes and applies data from multiple assessments and measures to diagnose student’s learning needs, informs instruction based on those needs, and drives the learning process. Designs and aligns formative and summative assessments that match learning objectives and lead to mastery.

4. DEVELOPING SELECTED RESPONSE QUESTIONS: A REVIEW Part I

6. True False Test 1.? 2.? 3.? 4.? 5.? 6. ? 7. ? 8. ? 9. ? 10. ? Number papers 1-10. Answer True or False for each question.

7. True-False Test 1.True 2.True 3.False 4.False 5.True 6. False 7. True 8. True 9. False 10. False Number papers 1-10. Answer True or False for each question. What does a high score on a True-False test really tell you about the student’s knowledge and understanding of concepts?

8. Multiple Choice Items Multiple-choice items can be used to measure one or more learning objectives. Like True-False, they are easy to grade. They are NOT objective, as no test/assessment is totally objective. (The instructor/designer determines content, objectives, and amount of emphasis placed on various topics.) They can be scored objectively.

9. Multiple Choice Questions Multiple-choice items contain three parts: 1)stem. This is the actual question. 2) correct answer. 3) distractors. These are incorrect, but plausible answers.

10. Multiple Choice Items Comparing True-False to Multiple- Choice – True-False Panama City is located in Bay County. – Multiple-Choice In which county is Panama City located? – Franklin – Gulf – Calhoun – Bay distractor answer stem

11. Writing Appropriate Distractors To be plausible, the distractor must have the potential for being selected as the correct answer. Two distractors are as effective as three if one of the three is not plausible (highly unlikely to be selected). In which county is Panama City located? Franklin Gulf Cedar Grove Bay answer Less plausible distractor distractor WHY?

12. Multiple Choice “DOs” 1.Use the same number of distractors for every question 2.Use plausible distractors that are related to the stem and are similar in character ( no cute, throw- away distractors ) 3.Have all distractors and the answer about the same length WHY?

13. Multiple Choice “DOs” 4.If the stem is an incomplete sentence, each distractor should grammatically complete the sentence. 5.Put all distractors in a single column, not side by side. 6.Attempt to always or sometimes to some extent avoid ambiguity (mostly, generally sometimes, etc.). Why?

14. Multiple Choice “DOs” 7.State the authority to be used in items calling for judgment (Based on….) 8.Vary the position of the correct answer. (What is most usual position?) 9.Avoid subtle clues in word choice or phrasing. Why?

15. “DO” Activity You have 10 minutes to do the following: Work with a partner. (How many is that in a group?) Review the FTCE question books and find questions that are NOT “DOs.” Let’s shoot for 4. Be prepared to explain why each is not a “DO.” Refer to the number (1-9) with which the question fails to comply.

16. Multiple Choice “DON’Ts” A.Don’t use specific determiners in distractors as “all, none, only, alone.” Why? B.Don’t use “generally, often, usually, most, and may.” Why? C.Don’t use negatives, as “without” or “except.” (If you must use a negative, highlight it.) Why? D. Don’t use clues in the stems as “a or an” at the end. (Put these in the distractors.) Why?

17. Multiple Choice “DON’Ts” E.Don’t use “all of the above.” F.Don’t use “none of the above.” Why? If you absolutely can’t control yourself and MUST use them, make them as frequently incorrect answers as they are correct answers.

18. Use the “DOs” & “DON’Ts” You have 10 minutes to do the following: Work with a partner. In any order (but don’t label them as DO or DON’T): Write 1 very bad multiple choice question. (Prize for the one that has the most DON’Ts & least DOs.) Write 2 very good multiple choice questions. (No prize, just the satisfaction of excellent work!) Write questions on chart paper and post.

19. Writing Multiple-Choice Items "Best" answer items are usually more difficult for the students than "correct" answer items. These items assume that your instruction has included activities in which students are required to compare and evaluate information. If students have been instructed that a particular aspect is the most important one, the question measures only knowledge or recall of information. Think before you write: What will this question tell me about a student’s level of understanding?

20. Do you remember Fractographing Mexicathon? It is very important that you learn about mexicathon. Mexicathon is a new form of diodine. It is factographed in Guanocartera. The Guanocarterrans marborate large amounts of pavoline and then montisorate it to bractorize mexicathon. Mexicathon may well be one of our most symbonized chethorens in the future because of our fastulator zenorith. Given this question: Who factographs Mexicathon? –A. Pavolines –B. Diodines –C. Guanocarterans –D. Marborates If students got the “right” answer, what would this tell you about their knowledge or understanding?

21. ASSESSING CONSTRUCTED RESPONSE QUESTIONS Part II

22. Simple versus Complex Constructed Response Simple Fill in the Blank (no word bank) Venn Diagram Open Response Short Answer Question Flow Chart Food Chain Complex Research Paper Lab Report Project Open Ended Essay Question Debate Food Web

23. Rubric Definition Scoring guides that contain the criteria for high quality work Contain descriptors of different proficiency levels Provide mechanisms or strategies students can use to improve work Are instructional in nature; student can use to self-assess own work

24. Advantages/Disadvantages Advantages Save time during grading Clearly inform students of the teacher’s expectations Provide formative feedback Disadvantages Require lots of up- front construction time Require frequent revision Require a plan to convert rubric scores to grades

25. Restaurant Rubric CriteriaPoorAcceptableExcellent CleanlinessSeveral (>5) people have died in the last month after eating here A few (<5) people have died in the last month after eating in this restaurant; 10 more have sought medical help No one has died in the last month after eating in this restaurant and <10 have sought medical help Work with your partner. Select an appropriate criterion by which to judge a restaurant. Write the “Excellent” descriptor for this criterion.

26. Sample Rubrics Choose one of the 4 sample Science Rubrics found on pages 40-50 (Figures 4.6, 4.7, 4.8 & 4.9). Choose one of the Application exercises (p. 38 or p. 39). For p. 38, use the Criteria for High Quality Rubrics handout For p. 39, only answer questions 1, 2, 3, & 6.

27. Parting is Such Sweet Sorrow! Brown: Review one important learning point from today’s session. Green: One idea you learned today that you’ll use in the classroom. Yellow: Something that surprised you. Red: One barrier that will make it difficult for you to implement your new knowledge. Orange: Your favorite quote from today. Blue: One thing you’d like to hear more about.