1. Beyond satisfaction, more than a survey: Assessment to practice
Heather Ammons and Danielle Clarke
NACADA Annual 2016 – Atlanta, Georgia

2. What our students are saying (part I)
How satisfied are you with the quality of your advisor?
Year
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree 3% 1% 2%
12%
82% 4% 5%
14%
75%
How likely are you to recommend your advisor to another student?
Year
Very Unlikely
Unlikely
Undecided
Likely
Very Likely 4% 1%
12%
79% 5% 3%
13%
75%

3. What our Students are saying (Part II)
During advising appointment, I felt comfortable talking with my advisor.
Year
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree 2%
15%
79% 5% 3%
14%
76%
So we delved deeper and could have just left it here, but…

4. What our Students are Saying (Part II)
What would have made you feel more comfortable talking with this advisor?
If the advisor made any attempt to show enthusiasm about her job or say anything other than "yeah", "mhmm", or "nah" then I would have felt a little more at ease knowing an advisor cared about my appointment.
It seemed as if he was having a long day, but I do not see what his attitude of the day had to do with my advising.
Her actually caring about my session. She barely spoke to me and did not make the meeting personal at all. I did not feel confident about my academic situation going into course selection after this meeting and I had to schedule another with someone else in order to know anything about my class options.
It being individualized. she didn't even ask my name or major. She was just checking me off of a list.
If he had not been rude, and acted like he genuinely cared.
This question was only asked to those students that indicated they felt uncomfortable

5. What our Students are saying (Part II)
Did you tell your advisor about factors (personal, financial, academic, etc.) that influence your success at UA?
Year
Yes No
64%
34%
Again, not too bad. Our students like us, they tell us things.

6. What our students are saying (Part II)
In what ways did your advisor address these factors with you?
He gave me a direct vision of where I am and where I need to be.
In listening, understanding, and planning accordingly
My advisor took my individual career goals and current situation into account and made me an updated, personal plan.
My advisor was made aware of my major time commitments outside of the classroom, as well as some personal issues I am dealing with. He then advised me on how to create a schedule that was manageable and beneficial to me with those other commitments in mind.
What would have made you more likely to discuss these factors?
If he asked. At one point he suggested I take a full course load over the summer, which is not financially feasible. I did not feel comfortable enough to express my concerns with that.
I didn't feel like I needed to talk about those factors.
If advisor had asked
If there was a reason to discuss them. Our conversations did not pertain to ability to attend, it was more focused on ensuring all of my classes were accounted for in the semesters they were offered since I have added a second major.
There was no opportunity to bring any of these factors up, as we only discussed class options for next semester.
Reading over this it’s easy to maybe toss the comments out as maybe an outlier student that is disappointed or a one off comment. Or maybe just an opportunity to work with advisors on how we engage with students. We took it as a view that shows the differing perspectives of advising and that we were missing out on a couple of our outcomes. First column – for those that answered yes. Second column – for those that answered no

7. Why satisfaction is not enough

8. What do you know about your office?
Students are happy with our services!
They like us!
They think we are nice and easy to talk to!
BUT…..
They can be happy and never know that they aren’t growing (problematic)
Advising is teaching…
What do you know about your office?
This is not to say that satisfaction is not of value, because that is not true, but instead to think through exactly what that is telling you about your office.

9. IT all goes back to the foundation
What is the purpose of your office?
Are there clearly established goals and outcomes?
What are you current practices?
What information do you currently track about your students? Why?
Are you actually doing what you say you are supposed to do?
When was the last time you asked your students about their advising encounters?
Essentially, what is the defined vision, mission, goals, and outcomes for your office?
This is where we started – at the foundation. Danielle will talk, briefly, about what we did a few summers ago. But really why does a focus on this matter?

10. Value of multiple measures
How do you know if you are meeting your mission, vision, goals, and outcomes? You assess it, but there are so many ways you can do that. Let’s go through different types and why they are of value.

11. Types of measures Direct Indirect
Method of gathering information that asks student/advisor to reflect on what has been learned rather than to demonstrate it.
Survey
Focus Groups
Interviews
Document Analysis
Direct
Indirect
Method of gathering information that allows student/advisor to demonstrate their knowledge of skills.
Appt #’s
Degree plans
Demographics
GPA
Progression to degree

12. Why use multiple measures?
Cannot access everything, but you can access a lot
Build powerful cases by pairing/contrasting
Tailor your story to the audience
Perception v. reality
Options to explore:
Focus Groups
Interviews
Mining what you already have
Self-assessment

13. Approaching assessment

14. Our Assessment Plan
Vision, Mission, Shared Values, Goals, and Outcomes reviewed every 5 years
Outcomes divided over five years, so we are not trying to assess everything at once
Each year we develop instruments and other measure that specifically address each of our emphasized outcomes
Yearly review of assessment outcomes with entire office
Yearly annual report that summarizes findings (Task Stream)
Self study at end of five years using CAS

15. What measures do we use? Direct Indirect Appt #s and reason codes
Businessatbama s
DegreeWorks plans #
Who created
Course registration audit
Surveys
Student and Advisor outcomes
Includes missed appointments, change of division
Focus Groups

16. Assessment to Practice
Assessment included in annual reviews
# of appointments
Survey and focus group feedback
Opportunities to address needs
Annual Report
What are we accomplishing
The good, the not so good, and the bad
Make it the entire office’s prerogative to define and work at areas of focus
Altered development and training
It’s hard to own up that it’s not the policies, it’s the conversations
How have our practices changed?

17. Making sense of it all

18. There is a lot to this Assessment is a work in progress
Cannot measure everything at once
Constantly review!
Continuously thinking of why you are doing this
It’s about change, growth, development
Every item, every step, every word in assessment relates back to something that the office values
Unique to every office
Never done, never static. Sheet with outcome here. Show them one of our outcomes and how we assess it.

19. Let’s get to brainstorming!
Assessment in progress
Just beginning
Do you have a vision and mission?
Yes: Is it part of your assessment?
No: Tease it out.
Do you have a vision and mission?
Yes: Great! How can you use it in assessment?
No: Tease it out.
Now that we’ve discussed why move beyond satisfaction and what we do in our office – how about you start thinking about your office. Let’s pass out the handout here.

20. How to contact us Heather Ammons Danielle Clarke hammons@
culverhouse.ua.edu
culverhouse.ua.edu