1. 2009 TACUSPA Fall Conference October 5, 2009 El Paso, Texas

2.  Setting the Scene  The heart of the matter: Learning and program outcomes  Direct vs. indirect measures of learning  The good old stand by measures  New assessment measures  Issues of staff development  Sharing

3.  Push for accountability in higher education ◦ Federal and state governments ◦ Regional accrediting bodies ◦ Parents, alumni, donors and students

4.  Program/process/performance outcomes – The end result of what a program or process is to do, achieve, or accomplish.  Learning Outcome: An easily identified action that a student is expected to demonstrate in terms of knowledge, skills, and attitudes upon completion of a program/activity/experience. 

5.  Direct Methods - Any process employed to gather data which requires subjects to display their knowledge, behavior, or thought processes ◦ Student work samples ◦ Collections of student work (e.g. Portfolios) ◦ Recorded observations of student behavior ◦ Evaluation of performance on a case study/problem ◦ Locally developed tests ◦ Standardized tests ◦ Pre-and post-tests ◦ Essay tests blind scored across units ◦ Supervisor/advisor evaluations ◦ Certificates of completion/compliance ◦ Evaluation of/response to purpose-driven reflective Journals

6.  Indirect Methods - Any process employed to gather data which asks subjects to reflect upon their knowledge, behaviors, or thought processes. ◦ Alumni, employer, student surveys ◦ Focus groups and interviews ◦ Exit Interviews with graduates or leaders leaving their positions ◦ Graduate follow-up studies ◦ Benchmarking ◦ Student leadership transcripts/portfolios ◦ Timelines and budgets ◦ Tracking use of services (attendance, ticket sales, clients, etc.) ◦ Tracking program participation by desired demographics ◦ Document analysis (e.g., meeting minutes, policies, handbooks)

7.  Benefits of direct measures to Student Affairs ◦ Provides faculty and administrators more confidence in the measures and their results because it is the academic standard ◦ Helps with accreditors who are looking for student learning in the co-curricular and measures of same  Challenges ◦ Time and expertise needed to develop ◦ Assessing the “fuzzies” – attitudes and values ◦ Small “n’s” and the validity of the studies  Bottom line ◦ Use multiple indirect measures to offset the lack of direct measures

8.  Surveys to inform of needs and satisfaction  Surveys to assess outcomes  Focus groups/interviews to inform future assessments or to dig more deeply into existing survey results  Tracking use and participation (numbers)

9.  Learning contracts  Pre and post tests of knowledge  Rubrics  Reflective journals  Reflective conversations/writing  Photography  Observations with documentation

10.  Reflective questions for closing meetings ◦ Apprises leadership of what membership is learning or understanding and what corrections or changes are needed to redirect or refocus ◦ Can be done on paper or 4”x6” cards ◦ Must be recorded ◦ Look for changes over time ◦ Examples  Greatest strength/weakness of a meeting or the organization;  Misconception/preconception - value or mission of the organization; purpose of an event; what’s expected to be accomplished  Muddiest point

11.  Reflective questions for staff  Reflective experiences for staff  Professional development opportunities

12.  Does your department or student affairs division have an agreed upon definition or understand about student learning?  Does your department or student affairs division have an agreed upon understanding of student learning in the co-curricular?  Do students understand what learning is?  Do students understand application of learning from the classroom to the co-curricular and from the co-curricular to the classroom?  What is life long learning and do students understand it?

14.  What was the most valuable learning from this session?  What do you still need to know or understand?

15. Sandi Osters, Ph.D. Director, Student Life Studies Texas A&M University C321 Cain Hall, Mail Stop 1254 979-862-5624 sandio@tamu.edu studentlifestudies.tamu.edu