1. ASSESSMENT OF STUDENT SUPPORT SERVICES Kimberly Gargiulo, Coordinator of Assessment Office of Institutional Research and Assessment

2. Cycle of Assessment II. Administrative Goals III. Means Of Assessment And Criteria For Success IV. Summary of Data Collected V. Use of Results To Implement Change or Improvement I. Institutional & Unit Mission/Goal Reference

3. 5 Column Model* *: Adapted from Institutional Effectiveness Associates Institutional Mission Unit Mission Administrative Goals Means of Assessment & Criteria for Success Results Use Of Results

4. Cycle of Assessment II. Administrative Goals III. Means Of Assessment And Criteria For Success IV. Summary of Data Collected V. Use of Results To Implement Change or Improvement I. Institutional & Unit Mission/Goal Reference

5. Administrative Goals  Linked to Unit Mission  Realistic  Limited in number ( 3-5 per assessment period)  Measurable

6. Interconnectedness: Administrative Goals-  Related to the mission and goals of the college  Supports Unit’s mission

7. Deciding on Goals- Short & Long List Short List: What currently is being assessed Can be assessed within one assessment period Linked to Unit’s Mission Long List: Other important goals – input from staff Refer back to every year Cycle into Short List

8. Outcome, Process or Satisfaction? Outcome: “Students who meet with advisors will graduate faster…” Process: “The AATC will increase the number of students served…” Satisfaction: “Students will be satisfied with our services…”

9. Principles of good goals:  Singular  Measurable  Observable Use action words  Reasonable

10. Observable Use action words  Students WILL: “be satisfied” “be aware” “demonstrate” “understand”

11. Working Example: Freshmen will be satisfied with AATC advising services.

12. Cycle of Assessment II. Administrative Goals III. Means Of Assessment And Criteria For Success IV. Summary of Data Collected V. Use of Results To Implement Change or Improvement I. Institutional & Unit Mission/Goal Reference

13. Means of Assessment & Criteria for Success: How do you know: Students met the goals? And to what extent?

14. Examples of Assessment- Direct Evidence  Surveys  Focus Groups  Student Interviews  Suggestion Boxes  1 minute paper

15. SURVEYS  Measures attitudes about service  Measures satisfaction  Can measure knowledge too (quiz)  Determine weak areas  Quick feedback  Open-ended or multiple choice  Created locally or standardized

16. FOCUS GROUPS  Questions asked to a group  7-12 participants  Attain qualitative data  Could be video or audio taped or notes can be taken during session  Facilitator needed

17. Indirect Evidence Examples:  GPA of students who attend AATC workshops or advisement  Graduation Rates  Retention Rates  Transfer Rates Compare participants versus non-participants

18. Criteria for Success Overall Primary total rating Component Secondary, more detailed. Identify scores which would elicit further review

19. Establishing Criteria for Success  Reasonable expectation of responses  Expressed in specific terms (overall) “The average rating of the workshop will be at least 4.0” (overall) “At least 75% of respondents will be satisfied or very satisfied with their advisement session.”  (component) “No more than 5% of respondents will be very dissatisfied with their advisement session.”

20. REMEMBER  Means of Assessment must be directly linked to goals Make sure instrument will give you the answers you need  Criteria for Success Check wording in tool and in criteria for success (rating scale, what rating scale??)

21. Working Example- Means of Assessment Goal: Freshmen will be satisfied with AATC advising services. Means of Assessment: 1) Survey administered to freshmen immediately following their advisement session will ask students about their level of satisfaction with the session.

22. Working Example- Criteria for Success Assessment Tool 1) Survey Criteria for Success 1) At least 80% of the respondents will be satisfied or very satisfied with their advisement session; no more than 5% of respondents will state they left the session still unclear about graduation requirements.

23. Cycle of Assessment II. Administrative Goals III. Means Of Assessment And Criteria For Success IV. Summary of Data Collected V. Use of Results To Implement Change or Improvement I. Institutional & Unit Mission/Goal Reference

24. Summary of Data Collected  What were your results?  Did you meet, exceed or fall below your target?  What do the results tell you? Hypothesize

25. Results Should:  Show to what extent goals were accomplished  Be linked to assessment means used  Be detailed to show the assessment took place  Justify the “use of results”

26. Results Should Not:  Be statistically unlikely  Match criteria of success exactly  NOT BE USED!

27. Working Example- Summary of Data Collected Criteria for Success 1) At least 80% of the respondents will be satisfied or very satisfied with their advisement session; no more than 5% of respondents will state they left the session still unclear about graduation requirements. Summary of Data Collected 1) 82% of respondents stated they were at least satisfied with their advisement session 2) 10% of respondents answered they left the session unclear about graduation requirements 3) The average rating of the session was a 3.5 out of 5.0 4) Response rate: 30%

28. Cycle of Assessment II. Administrative Goals III. Means Of Assessment And Criteria For Success IV. Summary of Data Collected V. Use of Results To Implement Change or Improvement I. Institutional & Unit Mission/Goal Reference

29. Using Results -Are the use of results substantive enough? -Detailed enough? -Do you need to modify your services, goals, targets or assessment tools? -Communicated to the appropriate parties?

30. Working Example- Using Results to Implement Change Summary of Data Collected 1) 82% of respondents stated they were at least satisfied with their advisement session 2) 10% of respondents answered they left the session unclear about graduation requirements 3) The average rating of the session was a 3.5 out of 5.0 4) Response rate: 30% Use of Results 1) Target of 80% was met; add a fill in question asking to detail why they answered this question as they did 2) Target of less than 5% not met; hold training for advisors about graduation requirements for incoming freshmen 3) Hold brainstorming sessions with staff about how to improve advising sessions (new goal to cycle in?) 4) Review mode of administration of survey, think about ways to increase rate.

31. If Results are Positive:  Celebrate your successes!  Let colleagues know!  Cycle in new goals?

32. If Results are Not So Positive:  What needs to be modified?  Work with colleagues  Don’t get discouraged!

33. Next Steps:  Continue the assessment cycle  Share with department  Create an assessment process  Visit Office of Institutional Research & Assessment with any questions!

34. EXERCISE 1) Two goals for the AATC. 2) Means of Assessment & Criteria for Success for these goals.

35. Assessment Time!  Please take a moment to fill out a brief survey. Thank You!

36. Questions? Kimberly Gargiulo, Coordinator of Assessment Office of Institutional Research and Assessment S 752 220-8331 kgargiulo@bmcc.cuny.edu