1. BEST PRACTICES IN LIBRARY INSTRUCTION FORUM November 7, 2007
HOW DOES YOUR ASSESSMENT MEASURE UP?
Karen Brown
Dominican University

2. Goals: Discuss different levels of evaluation
Examine the potential of outcomes-based evaluation
Apply outcomes-based strategies to library user instruction

3. Kirkpatrick’s Levels of Evaluation
LEVEL ONE: Reaction
LEVEL TWO: Learning
LEVEL THREE: Transfer
LEVEL FOUR: Results

4. LEVEL 1: REACTION
How did the participants react to the session?

5. LEVEL 2: LEARNING
Have the skills, knowledge, or attitudes of the participants increased, improved, or advanced?

6. LEVEL 3: BEHAVIOR (or transfer)
Do the participants use their newly acquired skills or knowledge in their jobs, courses, or personal endeavors?

7. LEVEL 4: RESULTS
What impact has the instruction achieved?

8. OUTCOMES-BASED EVALUATION – Why has it gained so much attention?
Ripple effect of “accountability culture”
Influence of business models
Aligns a program’s goals with the institution’s goals
Integrated approach to assessment

9. Plan … Implement … Assess
“If you don’t know where you are going,
you will wind up somewhere else.”
- Yogi Berra

10. Elements of Outcomes-Based Evaluation
Documents benefits to program’s participants
- What would participants recognize as
the benefits?
Focuses on outcomes vs. outputs
Output: 51 course-related instruction sessions provided
Outcome: Instructors report better use of research materials in students’ papers

11. Considers changes in: Skill: What someone can do
Attitude: What someone feels/thinks about something
Skill: What someone can do
Knowledge: What someone knows
Behavior: How someone acts
Status: Someone’s social or professional condition
Life condition: Someone’s physical condition
From: IMLS, “Outcome Based Evaluation”:

12. Important Questions to Ask
1. If we had an excellent and successful library user instruction program, what would be the impact or benefits?
- administration
- faculty
- students
- library staff
- other stakeholders/constituent groups?

13. 2. What would be the characteristics or indicators of success or excellence?
- attitudes
- skills
- knowledge
- behavior
- status
- life condition

14. 3. How might we document this success or excellence?
- Examples: interviews
observation
focus groups
performance assessments
journals or logs
library use statistics

15. Application to Library User Instruction
Students
Faculty
Library staff
Library services and programs
Administration of the institution

16. APPLICATION
If you had a successful library instruction program, what would be the impact or benefits?
- Describe how the success benefits the people served by the program
What would be the characteristics or indicators of success?
- Identify concrete and observable changes in attitudes, skills, knowledge, behaviors, status, and/or conditions
How might you document or demonstrate the success?
- What tools or methods would you use to gather data and information?