1. The Assessment Toolbox
Linda Suskie
Middle States Commission on Higher Education
AB Tech
February 2005

2. Today... What is assessment? The assessment toolbox
Rubrics (scoring guides)
Prompts (assignments)
Multiple-choice tests
Reflective writing
Using assessment results to improve teaching

3. What is Assessment? Deciding what we want our students to learn
Making sure they learn it!
--Jane Wolfson, Director, Environmental Science & Studies Program, Towson University

4. The Teaching-Learning-Assessment Cycle
Learning Goals
Using Results
Learning Opportunities
Assessment

5. 1. Learning Goals What is a good learning goal?
Outcomes – what students should be able to do AFTER they pass the course
Observable – action words
Clear – no fuzzy terms
Skills – thinking, performance
Important - meet student/employer needs
Not observable: Understanding
Clear: Demonstrate – depends on what you’re demonstrating
Good (nursing): Demonstrate skill in drawing a blood sample from a patient
Not good: Demonstrate effective communication skills

6. 2. Aligning Assignments with Goals
If you don’t,
you can’t assess how effective they are
and you won’t know what to change.
Also, the teaching/learning strategies often suggest good assessment strategies.
Example: info lit example from Gen Ed curriculum.
How, specifically, do students develop professional ethics in your program?
How do they develop oral communication skills?
Understand and appreciate the diversity of perspectives and practices around the globe.
Describe how global understanding enhances self-understanding.
Have a lasting curiosity about other places, people, and cultural experiences.
Know how to maintain and build their knowledge of other people, places, and cultures.
I haven’t seen your syllabi, only your course outlines.
Only you know if your assignments and exercises REALLY help students achieve the learning goals you’ve listed.
HANDOUT 01: Exercise 1
Show of handouts—how many of you put checks next to all of your goals?
The rest of you…and you’re not alone! Need to rethink your assessment strategies for you goals, which is why we’re here today!

7. 3. The Assessment Toolbox
HANDOUT 2: Examples of evidence of academic quality

8. Rubrics
A list of things you’re looking for when you’re grading tests, papers, or projects
Often with guidelines or standards for evaluating them

9. Rating Scale Rubrics
A scale showing the degree to which the things you’re looking for are present.
HANDOUTS 3 & 4: Example
Give to students when giving them assignment!
Makes grading go faster.
Pro: Fairly easy to construct; easy to complete
Con: Not clear what’s “outstanding” vs. “very good”
Possible inconsistencies with multiple graders
If this is an issue…

10. Descriptive Rubrics
More detailed descriptions of each possible rating.
HANDOUTS 5 & 6: Examples
Pros: No more arguments, greater consistency
Con: Takes a lot more time to create
You choose the format that you think will work best for you.
Can also say must score minimum on every criterion in order to pass.
Or must score minimum on certain criteria in order to pass.
If you’re uncomfortable with rubrics,
or if capstone projects vary so much that no one rubric will work for them all…

11. Holistic Scoring Guides
A single score that reflects an overall impression of performance
Scores are defined by
descriptions or
model answers
HANDOUT 7: Critical thinking holistic scoring guide

12. Write a Rubric!
HANDOUT 1, Exercise 2

13. Prompts: Creating Effective Assignments
Prompts = assignments
HANDOUT 8: Examples of assignments beyond papers
Important to give a variety of assignments because of students’ varying skills and learning styles
e.g., imagine a biology course with all assignments oral presentations…or all essay tests
HANDOUT 09: Writing effective prompts
READING: Mehrens & Lehman
HANDOUT 01: Exercise 3

14. Creating Effective Multiple Choice Tests
Start with a test blueprint.
HANDOUTS 10: Example

15. Definitions
Objective test
Stem
Alternatives/ responses/ options

16. Multiple Choice Advantages Efficient Fast and easy to score
Options can diagnose difficulties
Disadvantages
Hard to write
Often requires reading skills
Guessing
Can’t measure some thinking skills
Cannot measure:
Synthesis
Organization
Problem definition
Creativity

17. Use Multiple Choice Items for...
Conceptual understanding
Application
Identify correct application or example
Analysis
Identify correct cause, effect, or element
Identify why something occurs or is best

18. Interpretive Exercise
= context-dependent item
= enhanced multiple choice item
One new stimulus (paragraph, chart)
that students must read or examine
to be able to answer all
the objective items that follow
HANDOUT 11: Examples of interpretive exercises

19. Examples of Interpretive Material
Reading passage they haven’t seen
Description of lab experiment
Material from historical period (letter, document)
Description of patient’s symptoms
Chart, diagram, drawing
Any scenario (“You are...”)

20. Use Interpretive Exercises to...
Apply knowledge and understanding to new material or novel situations.
Identify correct generalization, inference, or conclusion.
Use problem-solving and analysis skills.
Prepare for standardized tests.

21. Writing Good Multiple Choice Items
HANDOUTS 12 & 13 (and ANSWER HANDOUT):

22. More Ways to Make Multiple Choice Tests Effective
Open-book, open-note
Throw out items that half your students get wrong.
Review only items that many students got wrong.
Ask them WHY they got them wrong.
Open-book, open-note: eliminates rote memorization; all items will be thinking skills or at least understanding.

23. Reflective Writing Why ask students to reflect?
Encourage metacognition
Learning how to learn by thinking about how we learn
Assess attitudes and values
HANDOUT 14
HANDOUT 1 Exercise 4 (if time)

24. 4. Using Assessment Results to Improve Teaching
Goals
Curriculum
Pedagogy
Assessments

25. Look at your learning goals.
Do you have too many goals?
Do your goals need to be clarified?
Are your goals inappropriate or overly ambitious?
1. Can’t turn terrible writer into great one in one three-credit course
2. e.g, goal that students can “write effectively” or “think critically”
HANDOUT: What is critical thinking?
3. Esp in community colleges—can’t accomplish 33 goals in a 11 courses

26. Look at your curriculum.
Including placement and developmental education.
Does the curriculum adequately address each learning goal?
maybe most common problem

27. Look at your teaching methods.
How do students learn best?
HANDOUT 16: how to achieve deep, lasting learning EXERCISES

28. Look at your assessments.
Are they poorly written and misinterpreted?
Do they match your key learning goals?
Are they too difficult for most responsible students?

29. Isn’t Poor Performance the Student’s Fault?
Sometimes, but usually a minority
Suskie’s “50% rule”

30. Time to Reflect! Reflection sheets including minute-paper questions
HANDOUT 17: Bibliography