Counselor Assessment: 2016-2018

“….any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness” (Schuh and Upcraft, 1996, p. 18) Guides good practice Typically has implications for single institution (vs. research) Assessment Learning NASPA & ACCA Good Practice in Student Affairs : (Chickering & Gamson) * Engages students in active learning; helps students dev coherent values and ethical standards; sets and communicates high expectations for student learning; uses systematic inquiry to improve student and institutional performance; uses resources effectively to achieve institutional missions and goals; forges educational partnerships that advance student learning; builds supportive and inclusive communities.

Assessment and Student Affairs While a significant amount of student learning occurs outside of the classroom (Pascarella & Terenzini, 2005) and student affairs professionals often help create and coordinate these out-of-class experiences (Cuyjet & Weitz, 2009), the student affairs perspective and experience too often is absent in campus-wide discussions of student learning assessment.” (Schuh and Gansemer-Topf, 2010, p. 5) “Student affairs assessment activities, unfortunately, are often not well integrated with other campus assessment activities—in part, because academic affairs still conducts much of the work on student learning outcomes. “ “Student affairs practitioners cannot afford to ignore, obfuscate or refuse to be engaged in assessment activities….that …. lead to enhanced student learning” (Schuh, 2015) As ASSESSMENT evolved, there was a lot of debate about how to do it, whether to do it, or sort of duck and take cover until the storm passed. … Good news: Evaluation of Student Affairs programs has a well-established history – mid 1950’s…. + Schuh&Upcraft pp.9-13 QUESTION: How aspirational is this? How integrated is your institution’s Student Affairs assessment with learning outcomes?.... Then click for 2nd comment… HOW TO insert Student Affairs into the Topics of Learning?

“Good enough” (vs. perfect) assessments are the norm (Rossi & Freeman, 1993) due to: Resource limitations Time limitations Organizational contexts Implementation limitations Assessment Winnecott’s “good enough mother”….. “The perfect is the enemy of the good” Voltaire….NASPA & ACCA Good Practice in Student Affairs : (Chickering & Gamson) * Engages students in active learning; helps students dev coherent values and ethical standards; sets and communicates high expectations for student learning; uses systematic inquiry to improve student and institutional performance; uses resources effectively to achieve institutional missions and goals; forges educational partnerships that advance student learning; builds supportive and inclusive communities. LOOK at WHAT DATA is already being collected?

Counselor Assessment (2016-2018) was conducted over a 2 year period with assessment status reports along the way. Began with the outcomes Reviewed 2015 CCSSE data: focused on Item 9b disaggregated by race & gender Planned intervention - Green Bag Campaign (to promote academic and emotional fitness) Qualtrics survey results from 2016-17 (showed GBC interventions to be effective. Wanted to expand outreach, so included GBC Skillshops) CCSSE data on Item 9b from 2017: showed neg. slide of Male responses across race but other CCSSE items (9d, 9f, 21, 36) showed that students largely find the college supportive Qualtrics survey results from 2017-18: showed GBC to be effective on CCSSE Item 9b across all groups by race and gender Future Assessments: Could stay with GBC and look to expand contact, esp to Males --- or change direction With AD of Learning Assessment Office, reviewed about a dozen survey items (9, 10, 12 & 13), disaggregated all sorts of ways, thousands of lines….. To distill b4 presenting to college-wide Counselors considering…. Where is our work, what to measure as part of our assessment?

2015 Male responses on “Very Much” were lower than females White Male & Female “Very Much” responses lower Similar groupings (but not the same) when examining “Very Much” & “Quite a bit” responses together…

GBC Qualtrics Survey 2016-17 and 2017-18 Q1, Q2, & Q3 sent to GBC Express & GBC Skillshop participants

GBC Express Qualtrics Survey 2016-2017 Total Responses Total Invites Response Rate East 3 79 4% Osceola 66 59 112% West 6 92 7% Winter Park 18 138 13% Totals 93 368 25% Need to increase survey response rates overall!! + discussed how to do that, 10% is acceptable Response Rate for satisfaction surveys 25% is Very Good & consistent with survey Response Rates of Academic Divisions Want to increase survey response rates in 2017-18!

GBC Qualtrics Survey 2016-2017 96% Q3 - How much does this college emphasize providing the support you need to succeed at this college? GBC Qualtrics Survey 2016-2017 96% Students who participate in GBC activities/events have very strong responses

CCSSE 2017: Item 9b Valencia outperformed the 2017 CCSSE cohort (75% Quite a bit and Very Much) and other Top Performing Extra-Large Colleges (77% Quite a bit and Very Much). Our own data showed that males, across all ethnicities, reported a decrease between 2015 and 2017. …The changes are significant. It was disappointing to see the decreases across all males (including Hispanic Females)!..... So Institutional Assessment helped look deeper and wider….. LARGER CONTEXT of Males struggling, less enrollment in HE….fluctuations…. ++ We wanted to look at other data to see how big of an issue this was. CCSSE Item 9b Quite a bit & Very much 2017 2015 African American males 82.6% 87.5% Caucasian males 82.4% 82.8% Hispanic males 84.3% 86.0% ALL males 83.3% 85.1% CCSSE Item 9b Quite a bit & Very much 2017 2015 African American females 83.9% 82.6% Caucasian females 82.9% 77.8% Hispanic females 86.0% 87.2% ALL females 84.6% 83.4% 2017 2015 All Students 83.9%

Findings and magnitude: Do students feel supported? Valencia provides strong support for students, but students who are struggling may not feel supported. Colleges across the nation are doing more to reach out to those students. 9b. College emphasizes providing the support needed to succeed (Quite a Bit or Very Much) 9d. College emphasizes helping students cope with non-academic responsibilities (Quite a Bit or Very Much) 9f. College emphasizes providing the financial support needed (Quite a Bit or Very Much) 21. Someone contacts students who are struggling with their studies (Yes) 36. Overall experience at this college (Good or Excellent) Valencia base sample (N=1,369) 77.4% 32.3% 51.5% 11.6% 88.8% Valencia base and oversample (N=3,863) 83.9% (0.0 point from 2015) 37.0% (+1.6 points from 2015) 56.5% (-3.3 points from 2015) 12.4% (-0.6 points from 2015) (+0.9 points from 2015 5 top-performing extra-large community colleges (N=5,851) 77.0% 32.9% 55.5% 18.6% 87.5% CCSSE 2017 cohort (N=177,771) 75% 29.1% 51.6% 20.7% 86.2% Difference between Valencia base sample and CCSSE cohort (+2.4 points) (+3.2 points) (-.1 points) (-9.1 points) (+2.6 points) CCSSE survey Item 21 Asks if contacted when struggling “with my studies”?…… Per 9b, Valencia performing well, but what is bigger picture? Part of larger male alienation phenomenon?

Counselor Assessment – Two-year cycle 2016-18 Total responses 2016-17 [GBC Express only] – 93 Total responses 2017-18 [GBC Express only] – 115 Total responses 2017-18 [GBC Skillshops only] – 305 Total responses 2017-18 [GBC Express and GBC Skillshops] – 420 How much did the event you participated in support you to succeed academically? How much did the event you participated in support you to become or remain emotionally fit? How much does this college emphasize providing the support you need to succeed at this college? [CCSSE Item 9b] GBC Express 2017-18 very much or quite a bit academically 76% emotionally fit 79% succeed at this college 97% (96% in 2016-17) Total respondents 115 GBC Skillshops 2017-18 very much or quite a bit academically 83% emotionally fit 84% succeed at this college 92% Total respondents 305

Counselor Assessment – disaggregated by Male/ethnicity African American male GBC Express 2017-18 very much or quite a bit Academically 86% emotionally fit 43% succeed at this college 100% Total respondents 7 African American male GBC Skillshops 2017-18 very much or quite a bit academically 84% emotionally fit 85% succeed at this college 86% Total respondents 13 Caucasian male GBC Express 2017-18 very much or quite a bit academically 67% emotionally fit 100% succeed at this college Total respondents 3 Caucasian male GBC Skillshops 2017-18 very much or quite a bit academically 100% emotionally fit 86% succeed at this college Total respondents 14 Hispanic male GBC Express 2017-18 very much or quite a bit academically 71% emotionally fit 82% succeed at this college 88% Total respondents 17 Hispanic male GBC Skillshops 2017-18 very much or quite a bit academically 88% emotionally fit 91% succeed at this college 85% Total respondents 33

Counselor Assessment – disaggregated by Female/ethnicity African American female GBC Express 2017-18 very much or quite a bit Academically 85% emotionally fit 69% succeed at this college 100% Total respondents 13 African American female GBC Skillshops 2017-18 very much or quite a bit academically 82% emotionally fit 89% succeed at this college 97% Total respondents 27 Caucasian female GBC Express 2017-18 very much or quite a bit academically 73% emotionally fit 82% succeed at this college 100% Total respondents 11 Caucasian female GBC Skillshops 2017-18 very much or quite a bit academically 76% emotionally fit succeed at this college 96% Total respondents 25 Hispanic female GBC Express 2017-18 very much or quite a bit academically 78% emotionally fit 80% succeed at this college 100% Total respondents 41 Hispanic female GBC Skillshops 2017-18 very much or quite a bit academically 89% emotionally fit 87% succeed at this college 92% Total respondents 78

Counselor Assessment 2016-2018 -- What did we learn? --Per Valencia’s 5 Year Impact Plan for Equity and Diversity, Counselors identified a “performance gap” between students on CCSSE Item 9b and worked to close it --GBC, with both Express and Skillshops, is an effective means of helping students report the college provides the support they need to succeed at this college (more effective than oversample from NSE on Item 9b) --Future Assessments: Could stay with GBC curriculum and look to expand contact, esp to Males --- or change the assessment tools and approach