1. MEASURES OF SUCCESS: Assessment and Evaluation
CIRTL Measures for Success
MEASURES OF SUCCESS: Assessment and Evaluation
Chris Pfund
based on presentations from Carol L. Colbeck and
Sue Daffinrud
Colbeck Copyright

2. Definitions Assessment Assessment instrument Evaluation
The process of measuring something (e.g., student knowledge, skills, and attitudes within a classroom setting)
Assessment instrument
The method or device for measuring (e.g., formally - a test; informally – observations of students, interaction with students)
Evaluation
A judgment of something based upon results from data gathered from assessment

3. CIRTL Measures for Success
LEVELS OF EVALUATION
Participation
Satisfaction
Learning
Application
Impact
Colbeck Copyright

4. CIRTL Measures for Success
PARTICIPATION
Who?
What?
Why?
Demographics
Disciplinary Major
Race/ Ethnicity/Gender
GPA
Experience
# of years/ classes in discipline
Other professional development experience
Motivation
Own goals
External encouragement
Evaluation and Research questions:
What % of the targeted population are participating?
What are demographics, experience, motivation of non participants?
Does participation change over time? If so, among which groups?
How can we increase participation?
Colbeck Copyright

5. CIRTL Measures for Success
SATISFACTION
Value for participants
Content
Instruction/facilitation
Did program address participants’ reasons for coming?
Own goals
External encouragement
Were learning objectives specified and addressed?
Did process foster participant interest?
Instructor style
Teaching methods
Did participants like the program? Consider it worthwhile?
Did the program deliver the content the developers hoped and planned to deliver?
Did the instructor’s style engage participants, convince them of his/her suitability to deliver the information, model the desired attitudes and skills?
How can process be improved to keep participants coming? Attract new participants?
Colbeck Copyright

6. CIRTL Measures for Success
LEARNING
Knowledge gains
Conceptual change
Skill development
Has awareness increased?
Content Knowledge
Theoretical foundations
Empirical research
Are there changes in?
Attitudes
Beliefs
Confidence
Can tasks be performed?
Behaviors
Did participants learn what was intended?
To what extent did the program contribute to their learning? What else contributed?
Colbeck Copyright

7. CIRTL Measures for Success
APPLICATION
Attempt
Implement
Adjust
Are new knowledge, attitudes, skills tried independently?
Does application follow desired processes & procedures?
Does participant analyze & adjust appropriately for context?
Colbeck Copyright

8. CIRTL Measures for Success
IMPACT
On undergraduate students
Increased satisfaction
Deeper learning
Application: in lab, on job, on GRE’s
On graduate student participants
Increased confidence in teaching
Wider array of career opportunities
More integration of teaching and research
On institution
Improved recruitment/retention of graduate students
Improved recruitment/retention of undergraduate students
Stronger community around research, teaching, and learning
Colbeck
Is program worthwhile?
Colbeck Copyright

9. SAMPLE EVALUATION QUESTIONS
CIRTL Measures for Success
SAMPLE EVALUATION QUESTIONS
What types of students are most likely to participate?
How do relationships between participant characteristics, experience, and teaching methods affect their satisfaction? Learning?
How do relationships between institutional context, class content, and individual characteristics affect participants’ independent application of learning?
Under what conditions are undergraduate students most likely to report gains in satisfaction and learning from a participant who has undergone professional development training?
Colbeck
Colbeck Copyright

10. Student Assessment of Learning Gains (SALG)
SALG instrument asks students to rate the extent to which course aspects helped them to learn and the extent to which they are achieving objectives of course
Website enables you to survey students on-line
Available at:

11. The SALG has five main questions:
How much did each of the following aspects of the course help you in your learning?
How well do you think that you now understand each of the following?
How has this class added to your skills in each of the following?
To what extent did you make gains in each of the following?
How much of the following will you carry with you to your other classes?