The Assessment Toolbox Linda Suskie Middle States Commission on Higher Education AB Tech February 2005

Today... What is assessment? The assessment toolbox Rubrics (scoring guides) Prompts (assignments) Multiple-choice tests Reflective writing Using assessment results to improve teaching

What is Assessment? Deciding what we want our students to learn Making sure they learn it! --Jane Wolfson, Director, Environmental Science & Studies Program, Towson University

The Teaching-Learning-Assessment Cycle Learning Goals Using Results Learning Opportunities Assessment

1. Learning Goals What is a good learning goal? Outcomes – what students should be able to do AFTER they pass the course Observable – action words Clear – no fuzzy terms Skills – thinking, performance Important - meet student/employer needs Not observable: Understanding Clear: Demonstrate – depends on what you’re demonstrating Good (nursing): Demonstrate skill in drawing a blood sample from a patient Not good: Demonstrate effective communication skills

2. Aligning Assignments with Goals If you don’t, you can’t assess how effective they are and you won’t know what to change. Also, the teaching/learning strategies often suggest good assessment strategies. Example: info lit example from Gen Ed curriculum. How, specifically, do students develop professional ethics in your program? How do they develop oral communication skills? Understand and appreciate the diversity of perspectives and practices around the globe. Describe how global understanding enhances self-understanding. Have a lasting curiosity about other places, people, and cultural experiences. Know how to maintain and build their knowledge of other people, places, and cultures. I haven’t seen your syllabi, only your course outlines. Only you know if your assignments and exercises REALLY help students achieve the learning goals you’ve listed. HANDOUT 01: Exercise 1 Show of handouts—how many of you put checks next to all of your goals? The rest of you…and you’re not alone! Need to rethink your assessment strategies for you goals, which is why we’re here today!

3. The Assessment Toolbox HANDOUT 2: Examples of evidence of academic quality

Rubrics A list of things you’re looking for when you’re grading tests, papers, or projects Often with guidelines or standards for evaluating them

Rating Scale Rubrics A scale showing the degree to which the things you’re looking for are present. HANDOUTS 3 & 4: Example Give to students when giving them assignment! Makes grading go faster. Pro: Fairly easy to construct; easy to complete Con: Not clear what’s “outstanding” vs. “very good” Possible inconsistencies with multiple graders If this is an issue…

Descriptive Rubrics More detailed descriptions of each possible rating. HANDOUTS 5 & 6: Examples Pros: No more arguments, greater consistency Con: Takes a lot more time to create You choose the format that you think will work best for you. Can also say must score minimum on every criterion in order to pass. Or must score minimum on certain criteria in order to pass. If you’re uncomfortable with rubrics, or if capstone projects vary so much that no one rubric will work for them all…

Holistic Scoring Guides A single score that reflects an overall impression of performance Scores are defined by descriptions or model answers HANDOUT 7: Critical thinking holistic scoring guide

Write a Rubric! HANDOUT 1, Exercise 2

Prompts: Creating Effective Assignments Prompts = assignments HANDOUT 8: Examples of assignments beyond papers Important to give a variety of assignments because of students’ varying skills and learning styles e.g., imagine a biology course with all assignments oral presentations…or all essay tests HANDOUT 09: Writing effective prompts READING: Mehrens & Lehman HANDOUT 01: Exercise 3

Creating Effective Multiple Choice Tests Start with a test blueprint. HANDOUTS 10: Example

Definitions Objective test Stem Alternatives/ responses/ options

Multiple Choice Advantages Efficient Fast and easy to score Options can diagnose difficulties Disadvantages Hard to write Often requires reading skills Guessing Can’t measure some thinking skills Cannot measure: Synthesis Organization Problem definition Creativity

Use Multiple Choice Items for... Conceptual understanding Application Identify correct application or example Analysis Identify correct cause, effect, or element Identify why something occurs or is best

Interpretive Exercise = context-dependent item = enhanced multiple choice item One new stimulus (paragraph, chart) that students must read or examine to be able to answer all the objective items that follow HANDOUT 11: Examples of interpretive exercises

Examples of Interpretive Material Reading passage they haven’t seen Description of lab experiment Material from historical period (letter, document) Description of patient’s symptoms Chart, diagram, drawing Any scenario (“You are...”)

Use Interpretive Exercises to... Apply knowledge and understanding to new material or novel situations. Identify correct generalization, inference, or conclusion. Use problem-solving and analysis skills. Prepare for standardized tests.

Writing Good Multiple Choice Items HANDOUTS 12 & 13 (and ANSWER HANDOUT):

More Ways to Make Multiple Choice Tests Effective Open-book, open-note Throw out items that half your students get wrong. Review only items that many students got wrong. Ask them WHY they got them wrong. Open-book, open-note: eliminates rote memorization; all items will be thinking skills or at least understanding.

Reflective Writing Why ask students to reflect? Encourage metacognition Learning how to learn by thinking about how we learn Assess attitudes and values HANDOUT 14 HANDOUT 1 Exercise 4 (if time)

4. Using Assessment Results to Improve Teaching Goals Curriculum Pedagogy Assessments

Look at your learning goals. Do you have too many goals? Do your goals need to be clarified? Are your goals inappropriate or overly ambitious? 1. Can’t turn terrible writer into great one in one three-credit course 2. e.g, goal that students can “write effectively” or “think critically” HANDOUT: What is critical thinking? 3. Esp in community colleges—can’t accomplish 33 goals in a 11 courses

Look at your curriculum. Including placement and developmental education. Does the curriculum adequately address each learning goal? maybe most common problem

Look at your teaching methods. How do students learn best? HANDOUT 16: how to achieve deep, lasting learning EXERCISES

Look at your assessments. Are they poorly written and misinterpreted? Do they match your key learning goals? Are they too difficult for most responsible students?

Isn’t Poor Performance the Student’s Fault? Sometimes, but usually a minority Suskie’s “50% rule”

Time to Reflect! Reflection sheets including minute-paper questions HANDOUT 17: Bibliography