1. Take a Bite Out of Learning Outcomes - The Development Process Presented by Debra L. Hammond Executive Director University Student Union, Inc. California State University, Northridge debra.hammond@csun.edu usu.csun.edu

2. LEARNING OUTCOMES Participants will learn how to:  Define a LEARNING OUTCOME  Use a PROCESS to develop an ASSESSMENT CYCLE  Write a LEARNING OUTCOMES STATEMENT  Develop STRATEGIES, OBJECTIVES and INTERVENTIONS to meet the learning outcomes  Determine ASSESSMENT METHODS for learning outcomes  Avoid the PITFALLS of the LEARNING OUTCOMES PROCESS

3. ASSESSMENT MYTHS AND FACTS

4. “It is an embarrassment that we can tell people anything about education except how well students are learning.” Patrick M. Callan National Center for Public Policy & Higher Education

5. ASSESSMENT Any effort to gather, analyze and interpret evidence which describes INDIVIDUAL, institutional, departmental, divisional or agency effectiveness Schuh/Upcraft, 1996 EVALUATION Any effort to use assessment evidence to improve INDIVIDUAL, institutional, department division or agency effectiveness Schuh/Upcraft, 1996

6. Assessment is about LEARNING Personal and Organizational and its about improvement

7. What is a Learning Outcome? A learning outcome is a statement of what A learner is expected to know, understand or be able to do as a result of a learning process. (http://www.ssdd.uce.ac.uk/outcomes/)

8. Assessment Mindset  Conscious about learning continuously  Envisioning a future that will shape the present Champion Upper Level Advocate Team Supporters Organizational Mindset

9. Assessment Plan Considerations  Plans and processes cannot be imported from other institutions and applied directly to a new context  Assessment efforts should collect both quantitative and qualitative data  Assessment needs to focus on processes and outcomes

10. The Assessment Cycle California State University, Northridge, University Student Union, Inc. 1. Conduct Needs Assessment 2. Define Learning Outcomes 3. Develop Intervention Methods & Strategies 4. Market the Program 5. Implement Intervention Methods & Strategies 6. Implement Assessment Plan 7. Analyze Results 8. Draw Conclusions 9. Report Results/ Conclusions

11. Assessment Cycle Step 1: Needs Assessment How are decisions made? (current)  Group “think”  Lots of unanimous decisions  Limited in-depth discussion  WIIFM How do we want decisions to be made? (futur e)  Analysis of the issue(s)  Reflective, critical thinking  Focus on the big picture  WIIFU

12. EXERCISE Using the theme of “team” Develop a learning outcome for your student employees or volunteers

13. The Process 1. What do we want our students to learn? 2. What do students learn from participating in our organization or programs? 3. What are the common learning outcomes shared by all?

14. Assessment Cycle Step 2: Defining Learning Outcomes California State University, Northridge Division of Student Affairs Common Learning Theme 5: Applying Knowledge and Practical Competence Outcome 1: The student will learn how to work effectively in teams Capability 1: The student will learn how to contribute to the functioning of the team. Capability 2: The student will learn how to accomplish a task as part of a team. Capability 3: The student will learn how to develop and assess goals. Capability 4: The student will learn how to make decisions as part of a team.

15. Assessment Cycle Step 3: Develop Intervention Methods and Strategies Training and Development: Board & Committee Chair Orientations, Retreats & Workshops: Servant Leadership & Accountability Meeting Facilitation/Management Goal Setting/Assessment Parliamentary Procedure Decision Making Effective Teams Other: Committee Meeting Discussions, Work, and Recommendations Journaling Intranet Posting Leadership Toolbox (on-line learning)

16. Assessment Cycle Step 4: Market The Program  Board of Directors Orientation / Trainings / Retreats / Meetings  Staff Meetings  Web site, Intranet  Bulletin Boards  E-mail Correspondence

18. Assessing a Learning Outcome: TOOLS  Pre-Assessment – Self Appraisals  Learning Outcomes Discussion/Quarterly Meetings  Post-Assessment – Self Appraisals  Focus Groups  Journaling  Observation Rubric  Exit Interviews

19. Assessment Cycle Step 5: Implement Intervention Methods and Strategies 2009-10 PLAN  Servant Leadership Workshop at Board Retreat July  Committee Chair/Executive Secretary Training August  Chair/Advisor Discuss Goals for the Year August  SMART Goals Workshop/Committee Orientation September  Committee Sets GoalsSeptember  Parliamentary Procedure Training September  Journaling October -March  Decision-Making Training October  Effective Team Training November  Mid-Year Assessment of Goals December  Review of Key Concepts from Training January/February  End of Year Goals ReviewMay Committees meet every 3 weeks September-December / January-May Advisors meet with committee Chairs at least monthly

20. Assessment Process Step 6: Implement the Assessment Plan Selected students will journal about their experiences and answer questions (October – March) All committee members will complete a survey regarding skills gained (April) All advisors complete an observation matrix and analysis (April)

21. SAMPLE JOURNAL QUESTIONS MAIN QUESTION SUPPLEMENTAL QUESTION OUTCOME > Describe the process used to develop your committee’s goals. What was your role in the development process? What was the role of others? GOAL DEVELOPMENT > Talk about a time when you voted without completely understanding the issue. DECISION MAKING SAMPLE SURVEY QUESTIONS How often do you do the following - get agreement on solutions to team problems? How often do you do the following - ask for input from team members? Never Rarely Occasionally Often Always Does not apply or I don't know

22. Assessment Process Step 7: Analyze Results University Student Union, Inc. California State University, Northridge Selected Survey Results PREPOST Relationship with others in my work group 58%68% improved as a result of team activities Shared their opinion in a group setting76%86% Considered multiple perspectives before 85%93% making a decision Achieved objective of increasing by 2% the percentage of students who often/always work with others to achieve common goals

23. Assessment Process Step 8: Draw Conclusions 2008-09 Don’t claim a 100% causational relationship between training and outcomes Do believe intentional training and development activities had a significant impact on student learning 2009-10 Continue the same efforts for next year and add additional trainings Add journaling activity to obtain qualitative data

24. Assessment Process Step 9: Market Results/Conclusions  Results published in Student Affairs Annual Report (via University Web site)  Results will be included in WASC accreditation Educational Effectiveness Report  Results published in USU Annual Report

25. Pitfalls – How to Avoid Them  Rushing the process  Asking lengthy “pre” and “post” self-assessment surveys  Asking the wrong survey questions  Asking double-barreled questions  Providing no training or inadequate training  Assigning assessment responsibility to one person  Trying to do too much, too fast  Making unfounded claims

26. Things to Keep in Mind  Begin with the end in mind  Evolutionary process  Obtain training, develop in-house expertise  Team approach/include students (Learning Outcomes Work Group)  Time commitment is required  Utilize tools/expertise such as StudentVoice  Be reasonable  Document and communicate results  Use a systematic process  Educate as you go  Continue to refine  START

27. Learning Outcomes Resource List Banta, Trudy W. (Ed.) (2004). Hallmarks of effective outcomes assessment. San Francisco: Jossey-Bass. Keeling, Richard P., M.D. (Ed.) (2004). Learning reconsidered: a campus-wide focus on the student experience. Washington, D.C.: ACPA and NASPA. Keeling, Richard P., M.D. (Ed.) (2006). Learning reconsidered 2: implementing a campus-wide focus on the student experience. Washington, D.C.: ACPA, ACUHO-I, ACUI, NACA, NACADA, NASPA and NIRSA. Love, Patrick G. and Estanek, Sandra M. (2004). Rethinking student affairs practice. San Francisco: Jossey-Bass. Perozzi, Brett (Ed.) (2009). Enhancing student learning through college employment. Bloomington, IN: Association of College Unions International.

28. Learning Outcomes Resource List – cont. Schuh, John H. and Associates (2009). Assessment methods for student affairs. San Francisco: Jossey-Bass. Schuh, John H. and Upcraft, M. Lee (2001). Assessment practice in student affairs. San Francisco: Jossey-Bass. Strayhorn, Terrell L. (2006). Frameworks for assessing learning and development outcomes. Washington, D.C.: Council for the Advancement of Standards in Higher Education. Suskie, Linda (2009). Assessing student learning a common sense guide, 2 nd. Ed. San Francisco: Jossey-Bass. Walvoord, Barbara E. (2004). Assessment clear and simple. San Francisco: Jossey-Bass. StudentVoice - http://www.studentvoice.com/app/Views/Home/Default.aspxhttp://www.studentvoice.com/app/Views/Home/Default.aspx

29. Questions and Answers