1. Assessment of Student Learning Co-Curricular Programs
Jo Lynn Autry Digranes
Coordinator for Assessment

2. Digranes, OCU Assessment 2014
Presentation Outline
Definition
Principles of Good Practice for Assessing Student Learning – American Association for Higher Education (AAHE)
Why Do We Assess? HLC Statement on Student Learning, Assessment, and Accreditation – Fundamental Questions Linked to Reading Process
Assessment -- Co-Curricular Programs
Digranes, OCU Assessment 2014

3. Assessment of Student Learning
The systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about how to improve learning.
Walvoord, B. E. (2004). Assessment clear and simple A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-bass A Wiley Imprint.
Digranes, OCU Assessment 2014

4. 9 Principles of Good Practice for Assessing Student Learning
The assessment of student learning begins with educational values.
Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
American Association for Higher Education (AAHE). Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hurchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright, 1992.
Digranes, OCU Assessment 2014

5. 9 Principles of Good Practice for Assessing Student Learning
Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
American Association for Higher Education (AAHE). Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hurchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright, 1992.
Digranes, OCU Assessment 2014

6. 9 Principles of Good Practice for Assessing Student Learning
Assessment works best when it is ongoing not episodic.
Assessment fosters wider improvement when representatives from across the educational community are involved.
American Association for Higher Education (AAHE). Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hurchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright, 1992.
Digranes, OCU Assessment 2014

7. 9 Principles of Good Practice for Assessing Student Learning
Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
American Association for Higher Education (AAHE). Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hurchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright, 1992.
Digranes, OCU Assessment 2014

8. 9 Principles of Good Practice for Assessing Student Learning
Through assessment, educators meet responsibilities to students and to the public.
Banta, Lund, Black & Oblander, 1996:
Assessment is most effective when undertaken in an environment that is receptive, supportive, and enabling.
American Association for Higher Education (AAHE). Authors: Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hurchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright, 1992.
Digranes, OCU Assessment 2014

9. HLC Statement on Student Learning, Assessment, and Accreditation
Fundamental Questions for Conversations on Student Learning
Six fundamental questions serve as prompts for conversations about student learning and the role of assessment in affirming and improving that learning:
How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?
What evidence do you have that students achieve your stated learning outcomes?
In what ways do you analyze and use evidence of student learning?
How do you ensure shared responsibility for student learning and for assessment of student learning?
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
In what ways do you inform the public and other stakeholders about what students are learning---and how well?
Higher Learning Commission. (2007) Statement on Student Learning, Assessment and Accreditation. HLC Website:
Digranes, OCU Assessment 2014

10. HLC Fundamental Questions
How are your stated student learning outcomes appropriate to your mission, programs, degrees, and students?
OCU Mission Emphasis
Scholarship
Service
Culturally rich community
Moral and spiritual development
Rigorous curriculum
Effective leaders
Digranes, OCU Assessment 2014

11. HLC Fundamental Questions
What evidence do you have that students achieve your stated learning outcomes?
Stated learning outcomes should be measureable.
Data can come from both direct and indirect measures, but always attempt to incorporate direct assessment.
Digranes, OCU Assessment 2014

12. HLC Fundamental Questions
In what ways do you analyze and use evidence of student learning?
Use multiple measures of direct and indirect assessment. (Grades are typically not adequate measures.)
Do you utilize evidence for reflecting upon program outcomes?
Do you utilize evidence for indication of student learning?
Do you utilize evidence for planning and change?
Digranes, OCU Assessment 2014

13. HLC Fundamental Questions
How do you ensure shared responsibility for student learning and for assessment of student learning?
How many faculty/staff members are involved?
How many courses/programs are assessed?
How many students are assessed?
How often are learning outcomes assessed?
Are external stakeholders involved in assessment, such as in service learning or internships?
Digranes, OCU Assessment 2014

14. HLC Fundamental Questions
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
What is the plan for improvement, if needed?
How does the plan for improvement link to strategic planning or budget requests?
How do you know that last year’s plans worked?
How did Readers’ recommendations impact or improve effectiveness?
Digranes, OCU Assessment 2014

15. HLC Fundamental Questions
In what ways do you inform the public and other stakeholders about what students are learning---and how well?
How are students informed of assessment results?
How are internal stakeholders informed of assessment results?
How are external stakeholders informed of assessment results?
Digranes, OCU Assessment 2014

16. Assessment of Co-Curricular Programs
“Learning” is not exclusively classroom-based
Many valued outcomes are not taught exclusively in the classroom
Many valued outcomes are the result of processes outside the classroom
“Learning” is a process based on three interdependent student experiences:
Understanding academic content and processes
Student development
Identity formation
Assessing Co-Curricular Learning PowerPoint
Robert Mundhenk, Visiting Scholar
The Higher Learning Commission
Digranes, OCU Assessment 2014

17. Assessment of Co-Curricular Programs
Student learning in co-curricular programs should be:
Intentional
Planned
Part of the structure of a student’s experience
Assessed
Assessing Co-Curricular Learning PowerPoint
Robert Mundhenk, Visiting Scholar
The Higher Learning Commission
Digranes, OCU Assessment 2014

18. Assessment of Co-Curricular Programs
Traditional -- Efficiency models:
Focus on process
How well is this office/service functioning?
Focus on numbers:
Clients served
Graduation rates
Tutorial visits
Attendance at activities
Student/staff ratios
Example: Indiana East University
Uses frequency data to identify areas with greatest potential impact
Academic Advising, Student Support Services, and Athletics
Assessing Co-Curricula Learning PowerPoint
Jonathan Keiser, Seniors Officer, OASIS and Robert Mundhenk, Visiting Scholar, The Higher Learning Commission
Digranes, OCU Assessment 2014

19. Assessment of Co-Curricular Programs
Newer Approaches -- Effectiveness Models: Indirect
Based on surveys and other indirect indicators (NSSE)
Often rely on student self-reporting
Tend to skew positively on outcomes, if not always on the processes that led to them
Example: Saint Xavier University
Gen Ed & Student Affairs host Campus Conversation Day
Observation: “62% of our students would find it ‘Very Difficult’ to ask instructors for help when struggling with course assignments.”
Result: Biannual meetings to discuss first year data
Assessing Co-Curricula Learning PowerPoint
Jonathan Keiser, Seniors Officer, OASIS and Robert Mundhenk, Visiting Scholar, The Higher Learning Commission
Digranes, OCU Assessment 2014

20. Assessment of Co-Curricular Programs
Newer Approaches -- Effectiveness Models: Direct
Focuses on student performance
Can be based on observation or objective measures
Require carefully designed and consistent measuring practices
Example: Drake University
Student Life Lecture Series
Topics informed by Gen Ed Outcomes
Integrated Direct Assessment Measures
Forwarded to Assessment Committee for Analysis and Communication
Assessing Co-Curricula Learning PowerPoint
Jonathan Keiser, Seniors Officer, OASIS and Robert Mundhenk, Visiting Scholar, The Higher Learning Commission
Digranes, OCU Assessment 2014

21. Assessment of Co-Curricular Programs
A Framework
University Mission Statement
Student Services Mission Statement
Mapping to Mission(s) and Levels
Program Goals and Measureable Objectives
Identify Measures
Identify “Successful Completion”
Plan Development and Implementation
Report Development
Some information from Azusa Pacific University Co-Curricular Assessment Handbook, 2010/2011
Digranes, OCU Assessment 2014

22. Assessment of Co-Curricular Programs
A Framework – The University Mission Statement
Oklahoma City University embraces the United Methodist tradition of scholarship and service and welcomes all faiths in a culturally rich community dedicated to student success. Men and women pursue academic excellence through a rigorous curriculum focused on students’ intellectual, moral, and spiritual development to prepare them to become effective leaders in service to their communities.
Digranes, OCU Assessment 2014

23. Assessment of Co-Curricular Programs
A Framework -- University Vision
Oklahoma City University, a private, church-related institution, aspires to be a premier university for the liberal arts and the professions, with respect to:
Academic excellence that cultivates character,
Student success and welfare,
Personalized education that encourages service, leadership, and spiritual growth,
Local community and economic development, including the use of global relationships, and
Cultural leadership in our community and state.
Digranes, OCU Assessment 2014

24. Assessment of Co-Curricular Programs
A Framework -- Student Services Mission Statement – Overall or by Department
OCU Multicultural Student Affairs provides vision, leadership, coordination and long-range planning for a comprehensive program of diversity workshops, student leadership development, student career development, the Clara Luper Scholarship program, the American Indian Scholars Program and the Multicultural Student Association as well as other cultural campus organizations.  Multicultural Student Affairs promotes understanding and appreciation of cultural diversity and fosters positive human relations for all students, faculty and staff while paying particular attention to the needs of student populations that may be historically underrepresented or undeserved based on race/ethnicity, sexual orientation, religion, gender, national origin, disability, class and age.  Multicultural Student Affairs approaches diversity issues by expressing the importance of our community being culturally proficient.
Digranes, OCU Assessment 2014

25. Assessment of Co-Curricular Programs
A Framework -- Mapping to Mission(s) & Levels
Assessment of General Education Goals by Method and Level of Learning
General Education Goal
Introduction at Basic Level of Learning
Development/Enhancement Level of Learning
Mastery Level of Learning
SS Department/Goal
Timeline
1. Develop fundamental thinking skills, including the ability to critique, analyze, solve problems creatively, and both discover and effectively use information.
Student Government Association
2. Acquire a broad base of knowledge, including the arts, history, languages, literature, mathematics, philosophy, religion, and both the natural and social sciences.
Multicultural Student Affairs
3. Acquire the capacity to communicate well, orally and in writing, both personally and publicly.
Career Services
4. Cultivate the capacity to recognize and reflect upon ethical issues.
Religious Life
5. Use their knowledge and skills to assist others in our wider communities.
Service Learning
6. Be exposed to cultural perspectives other than the students’ own.
Study Abroad
Digranes, OCU Assessment 2014

26. Assessment of Co-Curricular Programs
A Framework -- Program Goals and Measureable Objectives
Career Services: As a result of interactions with the Career Services Office, students and alumni will:
Identify their skills, abilities, and strengths in order to make knowledgeable career decisions
Have the necessary resources and skills to prepare for life-long post-graduate experiences
Assessing Co-Curricular Learning PowerPoint
Robert Mundhenk, Visiting Scholar
The Higher Learning Commission
Digranes, OCU Assessment 2014

27. Direct and Indirect Assessment
Direct assessment involves an analysis of products or behaviors that demonstrate the extent of students’ mastery of learning outcomes.
Indirect assessment involves people’s opinions, and these opinions can richly supplement what is learned in direct assessment studies.
Allen, M. J. (2006). Assessing general education programs. San Francisco, CA: Jossey-Bass, A Wiley Imprint.
Digranes, OCU Assessment 2014

28. Assessment of Co-Curricular Programs
Direct
Student Artifact/Special Projects
Student Demonstration
Exam
Portfolio
Case Studies
Digranes, OCU Assessment 2014

29. Assessment of Co-Curricular Programs
Indirect
Surveys – Students, alumni, employers, and other external constituents
Interviews
Focus Groups
Retention Rates
Graduation Rates
Employment/Placement Rates
Student research activities
Honors, awards, scholarships, and other forms of public recognition earned by students and alumni
Digranes, OCU Assessment 2014

30. Assessment of Co-Curricular Programs
Indirect Examples from OCU NSSE Survey
Career Services “Talked about career plans with a faculty member or advisor”
Significantly higher for Freshmen T 2.50 than US NEWS Top Ten (2.25), Carnegie Class (2.24), and NSSE 2012 (2.23)
For Seniors at 2.58 higher than Carnegie (2.67) and NSSE 2012 (2.44) and lower than US NEWS Top Ten (2.67)
Religious Life “Participated in activities to enhance your spirituality (worship, meditation, prayer, etc.)”
Significantly higher for Freshmen AT 2:60 than US NEWS Top Ten (2.20), Carnegie Class (2.05), and NSSE 2012 (2.06)
For Seniors at 2.37 was significantly higher than US NEWS Top Ten (2.13), Carnegie Class (2.13), and NSSE 2012 (2.14)
Digranes, OCU Assessment 2014

31. Assessment of Co-Curricular Programs
Indirect Examples from OCU NSSE Survey
Study Abroad “Study Abroad”
Was higher for Freshmen at .04 than US NEWS Top Ten (.02) and NSSE 2012 (.03); same as Carnegie Class at .04
For Seniors at .12 was significantly lower than US NEWS Top Ten (.33), same as Carnegie Class, and lower than NSSE 2012 (.14)
Digranes, OCU Assessment 2014

32. Assessment of Co-Curricular Programs
A Framework -- Identify “Successful Completion”
80% of eligible students will participate
85% will pass exam on content area presented
90% will rate the University in a specific area on a survey at good or excellent
All students will score at Level 4 or 5 on a rubric measuring leadership skills (with 5 being highest)
University students will surpass scores on survey questions related to mission as compared to peer and/or benchmark institutions
All agencies at which students perform service learning activities will rate satisfaction at Level 4 or 5 (with 5 being highest)
Digranes, OCU Assessment 2014

33. Assessment of Co-Curricular Programs
A Framework -- Mapping to Mission(s) & Levels
Can also do this for various departments within Student Services – mapping back to Student Services Mission – or mapping goals/objectives to departmental mission
Assessment of General Education Goals by Method and Level of Learning
General Education Goal
Introduction at Basic Level of Learning
Development/Enhancement Level of Learning
Mastery Level of Learning
SS Department/Goal
Timeline
1. Develop fundamental thinking skills, including the ability to critique, analyze, solve problems creatively, and both discover and effectively use information.
Student Government Association
2. Acquire a broad base of knowledge, including the arts, history, languages, literature, mathematics, philosophy, religion, and both the natural and social sciences.
Multicultural Student Affairs
3. Acquire the capacity to communicate well, orally and in writing, both personally and publicly.
Career Services
4. Cultivate the capacity to recognize and reflect upon ethical issues.
Religious Life
5. Use their knowledge and skills to assist others in our wider communities.
Service Learning
6. Be exposed to cultural perspectives other than the students’ own.
Study Abroad
Digranes, OCU Assessment 2014

34. Assessment of Co-Curricular Programs
A Framework -- Plan Development and Implementation
All co-curricular offices are expected to develop an Overall Assessment Plan—a written plan that out- lines how the office intends to assess its stated outcomes/goals. The Overall Plan serves as a “catalog,” listing all the office outcomes/goals, where they are addressed, the intended assess- ment methods (measures), and how often each outcome/goal is assessed. The overall plan is con- structed once and updated based on assessment feedback (data/evidence).
Some information from Azusa Pacific University Co-Curricular Assessment Handbook, 2010/2011
Digranes, OCU Assessment 2014

35. Assessment of Co-Curricular Programs
A Framework -- Plan Development and Implementation
The outcomes/goals might have many measures, but you do not need to use every measure every year. Some measures might only be used once every two years.
Some information from Azusa Pacific University Co-Curricular Assessment Handbook, 2010/2011
Digranes, OCU Assessment 2014

36. Assessment of Co-Curricular Programs
A Framework -- Report Development
Annual Report on Assessment of Student Learning in Co-Curricular Programs. Each student learning outcome and program goal assessed during the past year must contain the following:
Summary of Findings—what you learned.
Recommendations—the actions taken or planned based on the assessment evidence.
Reflections/ Notes—the lessons learned (about the program, assessment, etc.).
Acceptable Target Achievement—indications of whether the acceptable target results defined in the Annual Plan were met, not met, or exceeded.
Ideal Target Achievement—choose whether you are moving way from, approaching, or exceeding your ideal target.
Substantiating Evidence—include documents that will help support your findings.
Forms to be developed
Feedback process to be developed
Due dates to be developed
Some information from Azusa Pacific University Co-Curricular Assessment Handbook, 2010/2011
Digranes, OCU Assessment 2014

37. Digranes, OCU Assessment 2014
Questions?
Thank You!
Digranes, OCU Assessment 2014