1. Assessment of Academic Advising 2011 NACADA Summer Institute Charlie Nutt The presenters acknowledge and appreciate the contributions of NACADA colleagues Susan Campbell, Rich Robbins, Mike Kirk-Kuwaye, Lynn Higa, Tom Grites, and Eric White in preparation of materials for this presentation NACADA Executive Office Kansas State University 2323 Anderson Ave, Suite 225 Manhattan, KS 66502-2912 Phone: (785) 532-5717 Fax: (785) 532-7732 e-mail: nacada@ksu.edunacada@ksu.edu © 2011 National Academic Advising Association The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. The Global Community for Academic Advising

2. Review of assessment cycle Identifying stakeholders Identifying student learning outcomes Mapping the students’ learning experiences Identifying measures Using the CAS standards Assessment and research Overview The Global Community for Academic Advising

3. The Assessment Cycle (Maki, 2002, 2004) Gather Evidence Interpret Evidence Identify Outcomes Implement Change Mission/Purposes Educational Objectives The Global Community for Academic Advising

4. The Assessment Flowchart (adapted from Darling, 2005) Student Learning Outcomes Cognitive, Psychomotor, AffectiveProcess/Delivery Outcomes Mapping the Experience What experiences? When or by when? Gathering Evidence When gathered? Where gathered? How often gathered? From whom gathered? How gathered? Minimum performance criteria for success? Values Vision Mission Goals Programmatic Outcomes Sharing and Acting Upon the Results Interpret how results inform practice How and with whom to share interpretation Follow up on implemented changes Start the process all over again! The Global Community for Academic Advising

5.  Mapping of Outcomes  The Assessment Matrix/Table Institutional Mission Statement Local Mission Statement Specific Goal or Objective Specific Process/Delivery Outcome or Student Learning Outcome Where Outcome Occurs When or By When Outcome Occurs Outcome Measure Minimum Performance Criteria for Success Data Instruments Action(s) Based on Outcome Data The Global Community for Academic Advising

6. Identifying Key Stakeholders: Who Should Be Involved? Colleagues, faculty, administrators, institutional researchers, staff, students, and institutional community Decide how the assessment team will interact, overlap, and/or support other institutional efforts Encourage stakeholders on and off campus Continuous communication and feedback is a must! The Global Community for Academic Advising

7. Building of a shared trust Building of a shared motivation Building of a shared language Building of support for academic advising institutionally-wide The result is a shared ownership and belief in the process Benefits of a Collective and Collaborative Process The Global Community for Academic Advising

8. Exercise 1: Identify Your Stakeholders The Global Community for Academic Advising

9. Key Terms Process/Delivery Outcomes – Articulate the expectations for how academic advising is delivered and what information should be delivered through the academic advising experience Student Learning Outcomes – Articulate what students are expected to know, do, and appreciate as a result of involvement in the academic advising experience Mapping – The process of determining when, where and how the outcomes for advising will be accomplished over the students’ academic career and beyond The Global Community for Academic Advising

10. Process/Delivery Outcomes Typically what is evaluated via student satisfaction surveys e.g., “the advisor provided accurate information” e.g., “the setting for the advising session was appropriate” etc. Do not assess student learning, but evaluate what processes occurred during the advising interaction The Global Community for Academic Advising

11. Focus on Student Learning Outcomes The Global Community for Academic Advising

12. It’s All About Student Learning Understanding what, how and when students learn through academic advising The Global Community for Academic Advising

13. Advising is Teaching The Global Community for Academic Advising

14. Advisor as Teacher facilitator of communication coordinator of learning experiences referral agent who connects students with all of the institution’s resources and co-curricular opportunities that can help them be successful The Global Community for Academic Advising

15. Teaching Knowledge of subject matter Preparation - planning and organization of course material Advising Knowledge of institutional policies, procedures, programs, resources Preparing for advising meetings The Global Community for Academic Advising

16. Teaching Engaging students in participation of their learning Giving students feedback on their progress Helping students learn to analyze and problem solve Advising Guiding students to be self-directed and autonomous Working with student to regularly evaluate the student’s goals and progress Assisting students in decision-making skills The Global Community for Academic Advising

17. Teaching Clarity of presentation of subject matter Establishment of dialogue with students in the classroom Demonstration of excellent listening skills Advising Sharing information in a clear manner Leading students to question and interact with the advisor Listening to what advisees are saying verbally and non-verbally The Global Community for Academic Advising

18. Teaching Assigning out-of class work Working from a developmental perspective Providing a learner- centered environment Advising Assigning out-of-session tasks Working from a developmental perspective Providing a learner- centered environment The Global Community for Academic Advising

19. Teaching Identification and communication of student learning outcomes for the curriculum or course Use of a course syllabus Advising Identification and communication of student learning outcomes for the advising process/interaction Use of an advising syllabus The Global Community for Academic Advising

20. Focus on the Learner/Advisee As a result of academic advising, what do we want students to demonstrate that they… Know (cognitive student learning outcome) Are able to do (behavioral student learning outcome) Value and appreciate (affective student learning outcome) The Global Community for Academic Advising

21.  Mapping of Outcomes  The Assessment Matrix/Table Institutional Mission Statement Local Mission Statement Specific Goal or Objective Specific Process/Delivery Outcome or Student Learning Outcome Where Outcome Occurs When or By When Outcome Occurs Outcome Measure Minimum Performance Criteria for Success Data Instruments Action(s) Based on Outcome Data The Global Community for Academic Advising

22. Cognitive SLOs What do we want students to KNOW as a result of participating in academic advising? Know general education requirements Know about academic support services Know how to use the student information system to register Know how to use the catalog Etc……. The Global Community for Academic Advising

23. Exercise 2: Identify Three Things You Want Students to Know as a Result of Academic Advising The Global Community for Academic Advising

24. Tips on Developing SLOs avoid compound SLOs, e.g., “students understand and value the purpose of a liberal arts education,” “students understand and utilize their degree progress reports” as you are developing each SLO, think about where and when the opportunities for student learning may occur for that desired SLO as you are developing each SLO, think about how you might measure whether or not the SLO has been achieved The Global Community for Academic Advising

25. Behavioral/Psychomotor SLOs What do we want students to Doas a result of participating in academic advising? Generate their degree audit Make advising appointments Keep advising appointments Ask for help Access course descriptions and degree requirements using the online catalog Etc…. The Global Community for Academic Advising

26. Exercise 3: Identify Three Things You Want Students to Do as a Result of Academic Advising The Global Community for Academic Advising

27. Affective SLOs What do we want students to Value or Appreciate as a result of participating in academic advising? Value/Appreciate general education Value/Appreciate the advising relationship Value/Appreciate the process of learning Etc…. The Global Community for Academic Advising

28. Exercise 4: Identify Three Things You Want Students to Value or Appreciate as a Result of Academic Advising The Global Community for Academic Advising

29. Clear and measurable student learning outcomes for academic advising may also be derived from the NACADA Concept of Academic Advising the NACADA Core Values the CAS Standards for Advising All are included in your Summer Institute Guide The Global Community for Academic Advising

30. Using the CAS Standards in Program Assessment The Global Community for Academic Advising

31. The Council for the Advancement of Standards in Higher Education (CAS) has developed roughly three dozen different “self assessment” guides available in all areas of Student Services. They range from assessment of advising to housing to help desks… The Global Community for Academic Advising

32. The CAS Standards for Academic Advising can easily be utilized as outcome statements for your advising program The Global Community for Academic Advising

33. It is not necessary to attempt to use all of the CAS Standards as your outcome statements… …for those you may choose to use, be sure to develop related student learning outcomes as well as process/delivery outcomes The Global Community for Academic Advising

34. Mapping the Learning Experience What should be learned: e.g., student will know the components of the institution’s General Education requirements Where it should be learned: e.g., orientation workshops, advising sessions, personal reading of catalog or curriculum guide When or by when it should be learned: e.g., prior to first year (orientation); by end of first semester; by end of first year; before declaring a major, etc. The Global Community for Academic Advising

35.  Mapping of Outcomes  The Assessment Matrix/Table Institutional Mission Statement Local Mission Statement Specific Goal or Objective Specific Process/Delivery Outcome or Student Learning Outcome Where Outcome Occurs When or By When Outcome Occurs Outcome Measure Minimum Performance Criteria for Success Data Instruments Action(s) Based on Outcome Data The Global Community for Academic Advising

36. Exercise 5: Mapping Your Desired Outcomes: Identifying Where/When the Process/Delivery Occurs OR Where/When the Student Learning Opportunities Occur The Global Community for Academic Advising

37. Once the desired process/delivery and student learning outcomes have been identified, as well as when and where they will occur, the next step is to determine who or what will be measured and how the data will be gathered… …using multiple measures of varying types The Global Community for Academic Advising

38.  Mapping of Outcomes  The Assessment Matrix/Table Institutional Mission Statement Local Mission Statement Specific Goal or Objective Specific Process/Delivery Outcome or Student Learning Outcome Where Outcome Occurs When or By When Outcome Occurs Outcome Measure Minimum Performance Criteria for Success Data Instruments Action(s) Based on Outcome Data The Global Community for Academic Advising

39. Types of Measurement and Data Qualitative Quantitative Direct Indirect Multiple measures!!! The Global Community for Academic Advising

40. Measures can (and should) include existing institutional data Information from Institutional Research, Admissions, Registrar, etc. can provide tracking data, GPAs, retention rates, and other information you can utilize as assessment data – this can be a source of some of the multiple measures utilized (in addition to formal instruments, satisfaction inventories, and others) The Global Community for Academic Advising

41. For both process/delivery and student learning outcomes, you need to identify the minimum criteria for success of the outcome measure, e. g., number of students exhibiting a specific learning performance percentage of students exhibiting a specific learning performance advisor rating of student performance student rating of specific aspect of advising process advisor rating of specific aspect of advising process etc. The Global Community for Academic Advising

42. Exercise 6: Identifying Multiple Measures Select one process/delivery or one student learning outcome, and identify three ways to measure whether or not that specific outcome has been met The Global Community for Academic Advising

43. So I Have The Data – Now What? Sharing and Acting Upon Results –Interpret results regarding how they inform the advising process/delivery, student learning, and decision-making – Determine with whom and how the results are reported – Decide how you will implement changes based on the results – Start assessment cycle again… The Global Community for Academic Advising

44. Determine How and With Whom Results Are Shared Administration: President, Provost, various committees – via annual report, strategic plan, white paper, Web sites, etc. Faculty: all faculty, curricular committees, faculty advisors – via performance reviews, annual reports, strategic plans, Web sites, etc. Students: all students, student advisees, student senate, student groups – via newsletters, annual reports, Web sites, etc. Budgeting entities – via annual reports, budget requests, Web sites, etc. Accreditors – via self-studies, accreditation reports, Web sites, etc. The Global Community for Academic Advising

45. Revise pedagogy or curriculum Develop/revise advisor training programs Design more effective programming – advising, orientation, mentoring, etc. Increase out-of-class learning opportunities Shape institutional decision making – planning, resource allocation Other… Interpret How Results Will Inform Decision Making The Global Community for Academic Advising

46. Decide How You Will Follow-up on Implemented Changes Timetable to implement changes – implement all or specific components on a schedule Assessment of implemented changes – repeat assessment cycle again Continuous assessment – assessment is on-going The Global Community for Academic Advising

47. Final Exercise: Complete One of Your Outcome Matrices The Global Community for Academic Advising

48. THANK YOU! 2011 NACADA Summer Institute Assessment of Academic Advising Dr. Rich Robbins Bucknell University The Global Community for Academic Advising