MEASURES OF SUCCESS: Assessment and Evaluation CIRTL Measures for Success MEASURES OF SUCCESS: Assessment and Evaluation Chris Pfund based on presentations from Carol L. Colbeck and Sue Daffinrud Colbeck Copyright 2002

Definitions Assessment Assessment instrument Evaluation The process of measuring something (e.g., student knowledge, skills, and attitudes within a classroom setting) Assessment instrument The method or device for measuring (e.g., formally - a test; informally – observations of students, interaction with students) Evaluation A judgment of something based upon results from data gathered from assessment

CIRTL Measures for Success LEVELS OF EVALUATION Participation Satisfaction Learning Application Impact Colbeck Copyright 2002

CIRTL Measures for Success PARTICIPATION Who? What? Why? Demographics Disciplinary Major Race/ Ethnicity/Gender GPA Experience # of years/ classes in discipline Other professional development experience Motivation Own goals External encouragement Evaluation and Research questions: What % of the targeted population are participating? What are demographics, experience, motivation of non participants? Does participation change over time? If so, among which groups? How can we increase participation? Colbeck Copyright 2002

CIRTL Measures for Success SATISFACTION Value for participants Content Instruction/facilitation Did program address participants’ reasons for coming? Own goals External encouragement Were learning objectives specified and addressed? Did process foster participant interest? Instructor style Teaching methods Did participants like the program? Consider it worthwhile? Did the program deliver the content the developers hoped and planned to deliver? Did the instructor’s style engage participants, convince them of his/her suitability to deliver the information, model the desired attitudes and skills? How can process be improved to keep participants coming? Attract new participants? Colbeck Copyright 2002

CIRTL Measures for Success LEARNING Knowledge gains Conceptual change Skill development Has awareness increased? Content Knowledge Theoretical foundations Empirical research Are there changes in? Attitudes Beliefs Confidence Can tasks be performed? Behaviors Did participants learn what was intended? To what extent did the program contribute to their learning? What else contributed? Colbeck Copyright 2002

CIRTL Measures for Success APPLICATION Attempt Implement Adjust Are new knowledge, attitudes, skills tried independently? Does application follow desired processes & procedures? Does participant analyze & adjust appropriately for context? Colbeck Copyright 2002

CIRTL Measures for Success IMPACT On undergraduate students Increased satisfaction Deeper learning Application: in lab, on job, on GRE’s On graduate student participants Increased confidence in teaching Wider array of career opportunities More integration of teaching and research On institution Improved recruitment/retention of graduate students Improved recruitment/retention of undergraduate students Stronger community around research, teaching, and learning Colbeck 2003 Is program worthwhile? Colbeck Copyright 2002

SAMPLE EVALUATION QUESTIONS CIRTL Measures for Success SAMPLE EVALUATION QUESTIONS What types of students are most likely to participate? How do relationships between participant characteristics, experience, and teaching methods affect their satisfaction? Learning? How do relationships between institutional context, class content, and individual characteristics affect participants’ independent application of learning? Under what conditions are undergraduate students most likely to report gains in satisfaction and learning from a participant who has undergone professional development training? Colbeck 2003 Colbeck Copyright 2002

Student Assessment of Learning Gains (SALG) SALG instrument asks students to rate the extent to which course aspects helped them to learn and the extent to which they are achieving objectives of course Website enables you to survey students on-line Available at: http://www.salgsite.org/

The SALG has five main questions: How much did each of the following aspects of the course help you in your learning? How well do you think that you now understand each of the following? How has this class added to your skills in each of the following? To what extent did you make gains in each of the following? How much of the following will you carry with you to your other classes?