1. Summative Assessment Clarissa Dirks The Evergreen State College Olympia, Washington

2. Smith et al 2005 PREVIOUS THINKING ABOUT TEACHING AND LEARNING

3. OUTCOME

5. CURRENT THINKING ABOUT TEACHING AND LEARNING Misconceptions(formative assessments) Conceptual Frameworks(backward design; scholarly teaching) Metacognition(formative assessments; testing effect) AND SOMETHING ELSE...

6. Summative Assessments Drive Learning

7. THE MONTILLATION AND USES OF TRAXOLINE It is very important to learn about traxoline. Traxoline is a new form of zionter. It is montilled in Ceristanna. The Ceristannians found that they could gristerlate large amounts of fervon and then bracter it to quasel traxoline. This new, more efficient bracterillation process has the potential to make traxoline one of the most useful products within the molecular family of lukizes snezlaus. QUIZ: 1.What is traxoline? 2. Where is it montilled? 3. How is traxoline quaseled? 4. Why is traxoline important? What were the learning outcomes related to this content?

8. Misalignments Practice HOC LOC or none HOC Testing LOC HOC Outcome for Student Frustration Excitement and Learning Lower Order Cognitive Skills = LOCS Higher Order Cognitive Skills = HOCS

9. Benjamin S. Bloom Taxonomy of educational objectives. Published by Allyn and Bacon, Boston, MA. Copyright (c) 1984 by Pearson Education. Higher order cognitive skills (HOCS) Lower order cognitive skills (LOCS) Bloom’s Taxonomy

10. Cognitive Level Bloom Level A Simple Phrase to Guide Categorization Verbs Typically Associated with the Category HOCEvaluate “Defend or judge a concept or idea” Synthesize “Create something new” Analyze “Distinguish parts and make inferences” LOC/HOCApply “Use information or concepts in new ways” LOCComprehend “Explain information or concepts” Know “Recall information”

11. Partial Hierarchy of Bloom’s Taxonomy

12. Practice Using Bloom’s Taxonomy Work with a neighbor to categorize your first five test questions as either a LOC or HOC. Practice Using Bloom’s Taxonomy Work with a neighbor to categorize your first five test questions as either a LOC or HOC. What did you have to consider when doing this? Could you easily categorize someone else’s questions in biology? What about in another discipline?

13. Writing Multiple-Choice Questions That Test High Order Cognitive Skills Warning! These activities may be hazardous to your colleague’s health as a result of undue stress. Faculty often get distracted by the content of a question and miss out on the process of what is being presented. Let’s focus on the process!

14. Structures of Multiple-Choice Questions (MCQs) Standard MCQ

15. MCQ FormatAdvantagesDisadvantages StandardUseful for measuring HOC skills if a strong distractors are used If not well written the structure can cue a student to the answer Context Dependent Useful for measuring HOC skills if strong distractors are used Not time efficient Two-TieredThe second tier may survey students’ misconceptions Not time efficient Answer to one tier can cue the answer to the other tier. Complex (K-Type) Requires analytical skills of comparing and contrasting Not time efficient Tends to be very misleading MatchingTime efficientTypically not useful for measuring HOC skills True/FalseTime efficientTrue/false with less than 5 items have increased errors associated with guessing Alternate Choice Useful for measuring HOC skills Not time efficient Less discriminating than standard format

16. Take a few minutes and evaluate your exam. What kinds of questions are you asking? Free response, multiple-choice, or both? Do you have questions that help you to identify student misconceptions? Do you have any free response questions that requires a student to explain their reasoning or two-tiered MCQs. Using Two-tiered questions: A MCQ followed by a free response requiring students to explain why they answered the question the way they did will be very revealing!

17. Enchanted Valley is located within the Olympic National park of Washington state and is home to many black bears. A researcher is studying the different types of berries that 20 Enchanted Valley black bears eat. Which of the following passages describes the researcher’s data that would be graphed as a histogram? A) amount of each type of berry the bears ate in a week B) amount of salmon berries eaten by each bear in a week C) amount of huckleberries eaten by male and female bears in a week In the space provided below, please explain your reasoning for selecting the answer to the above question.

18. Review the graph right. Which of the following passages describes the graph? A) membrane permeability to urea is only dependent on the amount of membrane cholesterol B) amount of membrane cholesterol is only dependent on membrane permeability to urea C) membrane permeability to urea is dependent on temperature and the amount of membrane cholesterol D) amount of membrane cholesterol and membrane permeability to urea are dependent on one another

19. Which of the following best describes why you selected the answer that you did? The graph shows... A) two dependent variables and one independent variable B) one dependent variable and two independent variables C) one dependent variable and one independent variable D) two dependent variables and two independent variables

20. Q: Why do we care about student misconceptions? A1: They make us crazy! A2: We would like to engage in Scholarly Teaching and/or Discipline-Based Education Research Concept Inventories Two ways of thinking about this terminology: 1. An outline of the key concepts that a course or curriculum will cover. 2. A MCQ test based of a specific content area that uses common student misconceptions as distractors.

21. Concept Inventories in Biology Genetics Concept Assessment (GCA)Smith, et al. 2008. Genetics Literacy Assessment Instrument 2 (GLAI-2)Bowling et al., 2008a; 2008b Genetics Concept InventoryElrod, 2007 Conceptual Inventory of Natural Selection (CINS)Anderson, 2003 Anderson et al., 2002 Biology literacy (http://bioliteracy.net/)Klymkowsky and Garvin-Doxas, 2008 Diagnostic Question Clusters: BiologyWilson et al., 2006 D’Avanzo, 2008 Host-Pathogen Interactions (HPI)Marbach-Ad et al., 2009 Introductory Molecular and Cell Biology Assessment (IMCA) Shi et al., 2010 You too can create a concept inventory. If you do, I hope you would publish it! It would require that the inventory is valid and reliable.

22. VALIDITY Validity: Addresses whether the test measures what is suppose to measure. Content Validity: The test presents the key concepts and misconceptions for a given domain as affirmed by experts. Construct Validity: The item measures what it is intended to measure. It requires affirmation by experts. Face Validity: The test has the appearance of measuring what it intends to measure, thereby motivating students to persevere in doing their best.

23. RELIABILITY Reliability: Addresses the consistency of a set of measures; it is based on whether the test results are repeatable and not based on random error. Inter- and Intra-Rater Reliability: The consistency of scoring free-response items by different instructors or the same instructor, respectively. (GREAT FOR RUBRIC DEVELOPMENT!) Internal Consistency: It measures whether pair-wise items on the same test, measure the same latent variable. Test-Re-Test Reliability: Each student should give relatively the same result when they take the test at two different times. Alternate Form Reliability: Each student should give relatively the same result when they take alternate forms of the same test at two different times.

24. What are we doing for the rest of the day?

25. Applying the concepts you learned this morning for creating teaching materials. Work in groups to do the following in this order: 1. Create a few learning outcomes using Bloom’s Taxonomy Target misconceptions? HOCS? 2. Write 2 summative assessments that test the learning outcomes HOCS? MCQS? 3. Design 1 formative assessment that gives students practice at working with the material at the same cognitive level at which you are testing them Help students address their misconceptions, build a framework and develop metacognition. POST THESE TO THE WIKI PAGE!