1. NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT Shannon M. Sexton, Julia M. Williams, & Timothy Chow Rose-Hulman Institute of Technology

2. Introduction & Workshop Goals RHIT/IRPA - NASPA 2009  Introduction  Goals 1. Discuss the process of developing institute student learning outcomes as well as common difficulties. 2. Assist participants in developing their own learning outcomes 3. Discuss evidence that can be used to evaluate student learning outcomes achievement. 4. Assist participants in identifying articles of evidence for their learning outcomes. 5. Discuss using an e-portfolio as a means of collecting and rating evidence. 6. Allow participants to rate sample evidence using a sample rating rubric. 7. Discuss how to use the rating results to close the loop and improve student learning.

3. Group Activity #1 RHIT/IRPA - NASPA 2009  Define Teamwork  Include characteristics of teamwork  Include traits of those characteristics  Time allotted 10 minutes

4. About Rose-Hulman  Terre Haute, Indiana  1800+ undergraduate students  B.S. degrees in engineering, science, and mathematics  80%+ engineering students  Accredited through ABET and North Central RHIT/IRPA - NASPA 2009

5. CASO RHIT/IRPA - NASPA 2009  Commission on the Assessment of Student Outcomes  Faculty committee with 1 representative from each academic department  Regular meetings throughout the academic year  Tasked with developing Institute-wide student learning outcomes and a way to measure them

6. Historic Timeline RHIT/IRPA - NASPA 2009  1996 – 1998  1999 – 2000  2001 – 2006  2007 – present

7. Removing the Jargon RHIT/IRPA - NASPA 2009  Domains = Categories of skills  Learning Outcomes = Definition of the skills students should have  Performance Criteria = Characteristics of the learning outcome  Rubrics = Use of a scale to identify traits in a document

8. Domains and Performance Criteria 8 DomainStudent Learning Outcome Performance Criteria Technical Skills Outcomes developed by each department. Levels A – C indicate the level of performance achieved by each student. Professional Skills 1. Leadership 2. Teamwork 3. Communication Levels A – C indicate the level of performance achieved by each student. World Citizenship 4. Cultural & Global Awareness 5. Ethics 6. Service Levels A – C indicate the level of performance achieved by each student. RHIT/IRPA - NASPA 2009

9. Developing Outcomes RHIT/IRPA - NASPA 2009  Importance of preliminary research  Teams of faculty developers  Developing in steps & gaining institute buy-in  Importance of measurability

10. Writing Outcomes – Activity #2 RHIT/IRPA - NASPA 2009  Revise your teamwork definition into a student learning outcome with rubrics  Time allotted 15-20 minutes

11. 10 minutes to stretch your legs Take a Break RHIT/IRPA - NASPA 2009

12. Evidence of Outcomes RHIT/IRPA - NASPA 2009  Choosing appropriate evidence  New vs. Current  Importance of Rubric  Collecting evidence  How to physically collect & store evidence  Course Mapping  Tracking where submissions are coming from  Accountability

13. Criterion Description and Rubric  Teamwork B1: Demonstrate how you reached a decision as a team. Primary Traits: A passing submission for this criterion must: 1. Describe the team goal. 2. Provide a description of a specific team decision and describe the following: - the process of making the decision - how multiple team members contributed to the outcome - how team members’ ideas were critically evaluated 3. Show that the ultimate outcome was consistent with the team’s decision-making process. Potential documents: Documents appropriate for this criterion include (but are not limited to): Memo or reflective statement on team process from a lab group design team, debate team, service project. Additional information: 13 RHIT/IRPA - NASPA 2009

14. Course Mapping 14 Course mapping RHIT/IRPA - NASPA 2009

15. Identifying Evidence – Activity #3  Where do you already have existing assignments that could be used as articles of evidence for the Teamwork outcome?  Group Brainstorming Activity 15 RHIT/IRPA - NASPA 2009

16. Domains and Performance Criteria 16 Performance Criteria Level C – associated with awareness, application, or evaluation. Level B – associated with application. Level A – associated with knowledge or comprehension. A-Level Criteria B-Level Criteria C-Level Criteria RHIT/IRPA - NASPA 2009

17. Collecting Evidence RHIT/IRPA - NASPA 2009  Use of an E-portfolio system  Faculty input  Student input  Assessment staff input

18. Rating Evidence RHIT/IRPA - NASPA 2009  Rating preparation  Administrative side of rating  Paying faculty volunteers, tech support, number of volunteers, etc.  Process of rating days

19. Portfolio Rating Methodology  Evaluation rubric, A, B, C  Inter-rater reliability: initial, subsequent  Document rating: Pass/Fail/Exemplary  Wholistic rating, NOT grading RHIT/IRPA - NASPA 2009

22. Rating Documents – Activity #4 RHIT/IRPA - NASPA 2009  Rate 2-3 teamwork documents using the provided rubric.  15 - 20 minutes

23. Using Rating Results RHIT/IRPA - NASPA 2009  Identifying constituents  How to communicate results to institute  What to do with the results

24. Example Result RHIT/IRPA - NASPA 2009

25. Please contact us for more information after the conference. RHIT/IRPA - NASPA 2009