Kudos for Starfish! Pilot Review and Plans for 2016-17 2016 UC Advising Conference

Overview Early Alert vs. Connect Pilot (AY 15-16) AY 16-17 Going forward

Starfish Solutions Early Warning & Student Tracking System Student concerns can be manually flagged when observed or through periodic surveys. Automatic flags are raised based on online grade book data. Recommendations can be communicated to the student. Appropriate personnel are informed of flags raised and cleared. Close the loop with faculty when progress is made. Educational Support Networking System Each student has personal My Success Network of instructors and campus staff. Online scheduling for in-person meetings – both individual and group appointments. Appointment confirmation by email. Documentation of contact with students through private and shared notes. Service catalog of available campus support resources.

Pilot Participants (AY 15-16) UC Blue Ash All Courses Lead Contact: Mark Miller Progress Surveys Biology Biology 1082 Professor: Dr. Brian Kinkle System Flags Math Math 1026 (Pre-Calculus) Math 1044,1045 (Applied Calc I & II) Math 1046 (Business Calc) Math 1060 (Calc I with Pre-Calc) Math 1061, 1062 (Calculus I & II) Chemistry Chem1041 Professor: Dr. Dan Waddell System Flags / Zoom-In Psychology Psychology 1001 Professor: Dr. Kenneth King

Early Alert Starfish Workflow Faculty Raises Tracking Item Student Receives E-mail Advisor is copied

Flags for Fall 2015 Total Unique Students With a Flag or Kudos 4964 Average Number of Flags Per Student 2.61 Average Number of Kudos Per Student 1.63 Total Items Raised Breakdown of items based on method raised Item Name Total Raised Manually Raised Survey Raised System Raised FLAGS 9411 587 4704 4120 (ALL) Attendance Feedback 200 (ALL) Homework/Assignment Feedback 153 (ALL) Online Participation Feedback 21 (ALL) Quiz/Test Score Feedback 90 (MATH) Attendance Alert 1 97 (MATH) Overall Average Alert (60) 165 56 109 (MATH) Overall Average Alert (70) 101 9 92 (MATH) Running Score (60) 982 (MATH) Running Score (70) 990 (MATH) Test Score (60) 1315 (MATH) Test Score (70) 833 (UCBA) Academic Progress Concern (Final) 1342 49 1293 (UCBA) Attendance Alert 1 566 2 564 (UCBA) Attendance Alert 2 668 1 667 (UCBA) Homework/Assignment/Quiz Alert 2 860 3 857 (UCBA) Online Participation Alert 1 182 179 (UCBA) Online Participation Alert 2 163 (UCBA) Test Score Alert 2 683   KUDOS 3084 291 2793 (ALL) Kudos GRAND TOTAL 12495 878 7497

Improvements from Fall 15 to Spring 16 Fewer and consistent tracking items and templates for usage across all ‘clients’ Significant edits to e-mail templates based on feedback (sent ‘on behalf’ of FACULTY name). Reduced redundancy and confusion with multiple ‘similar’ tracking items Added “Academic Concern” Flag (with required faculty comments), and “Showing Improvement” kudo at faculty request Three more gateway courses were added for Clifton Campus – Psychology, Chemistry, Biology Successfully implemented “Zoom-In” functionality to raise flags Updated website, addition of list-serv and weekly communication with end-users Worked with Blackboard support to find solutions to system flag setup

Flags for Spring 2016 Total Unique Students With a Flag or Kudos 4288 Average Number of Flags Per Student 2.56 Average Number of Kudos Per Student 1.76 Total Items Raised Breakdown of items based on method raised Item Name Total Raised Manually Raised Survey Raised System Raised FLAGS 6612 919 3194 2499 Academic Progress Concern* 884 169 715 Assignment / Quiz Concern* 141 Attendance Concern* 1182 193 989 Cannot Progress in Sequence* 263 139 124 In Danger of Failing* 1328 90 1238 Online Participation Alert* 155 27 128 Running Total (Failing) 532 Running Total (less than C-) 688 Six or More Flags 91 Test Score Alert* 677 158 519 Three or More Flags 669   KUDOS 3971 725 3246 Kudos (Keep Up the Good Work)* 3580 717 2863 Kudos (Showing Improvement) 390 7 383 GRAND TOTAL 10552 1641 6440 2471

16 Fall Participants UC Blue Ash Biology Math Chemistry Business All Courses Progress Surveys Biology Biol1081, 1082 (Biology I & II) Dr. Brian Kinkle Biol2001c,2002c (A&P I & II) Dr. Paul Nodzak Zoom-in Math Math0039 (Algebra for College Math) Math1021, 1022 (Algebra, Trig) Math1023, 1024 (STEM Prep I & II) Math1026 (Pre-Calculus) Math1044,1045 (Applied Calc I & II) Math1046 (Business Calc) Math1060 (Calc I with Pre-Calc) Math1061, 1062 (Calculus I & II) Progress Surveys/Zoom-In Chemistry Chem1041 (Gen Chem) Zoom-In Business BA1041 (Bus. Pathways) Econ1001, 1002 (Macro, Micro)

New for Fall 2016 Full-time, on-campus support More gateway courses were added for Clifton Campus System flags are not being used A&S check-in kiosk Clermont Campus integration CECH pilot for Connect (Appointment Scheduling) Request Help functionality for students

Fall 2016 (So Far…) Total Unique Students With a Flag or Kudos 2178 Average Number of Flags Per Student 1.71 Average Number of Kudos Per Student 1.31 Total Items Raised Breakdown of items based on method raised Item Name Total Raised Manually Raised Survey Raised System Raised FLAGS 2067 202 1730 135 Assignment / Quiz Concern 32 Attendance Concern 879 66 813 General Academic Concern 633 25 608 I have questions about my class 7 In Danger of Failing 9 Low Test Score Alert Online Participation Alert 321 12 309 Open Template Flag 3 Request help from a UC expert 16 Six or More Flags 17 Three or More Flags 118   KUDOS 1493 433 1060 Kudos (Keep Up the Good Work) 1490 430 Kudos (Showing Improvement) GRAND TOTAL 3561 636 2790

Requests for help: How to schedule advising appointments Cannot find advisor on Starfish How to schedule appointments with instructor Homework/project help needed in classes Specific advising questions

Of Students Surveyed: Students with Kudos: Positive Feedback Responses: Was encouraging 78.5% 215 Boosted my confidence to succeed in class 62.4% 171 Motivated me to reach out to campus resources 26.6% 73 Had a positive impact on my performance in class 59.1% 162 Had a negative impact on my performance in class 1.1% 3 Had no any effect on me at all 12.8% 35 36% indicated that they reached out to their advisor and/or instructor after receiving this feedback. 54% indicated that the feedback was helpful. 41% said it was okay. 5% said it was not helpful. 87% indicated that the feedback helped them make informed decisions about their academic success. 76% indicated that they would like to continue using Starfish in the future Common themes included: Likes: appointment scheduling, ease of instructor communication, convenient, straightforward, not confusing, closer relationships with professors, encouraging, helpful, boosts confidence Dislikes: generic, insincere, confusing, complicated, didn’t work, not enough advisors/instructors using it

Of Students Surveyed: Students with Flags: Inspired me to take action 40.0% 118 Encouraged me to speak to my professor 43.1% 127 Encouraged me to reach out to my academic advisor 30.5% 90 Motivated me to reach out to campus resources 22.7% 67 Made me feel like I could still rectify the situation 26.8% 79 Embarrassed me / was discouraging 28.1% 83 Had a positive impact on my performance in class 24.1% 71 Had a negative impact on my performance in class 8.1% 24 Had no any effect on me at all 23.4% 69 62% indicated that they reached out to their advisors and/or instructor after receiving this feedback. 75% indicated that the feedback helped them make informed decisions about their academic success. Common themes included: Likes: It works, helps me keep track of my progress, helpful, user friendly, keeps me organized Dislikes: I don’t know what Starfish is, what is it supposed to be doing?, I couldn’t figure it out, confusing, messages generic – repetitive and impersonal, confusing, auto-generated message – not from instructor. Redundant

Faculty Testimony I really think that Starfish is something that all faculty should be strongly encouraged to use, especially those in large and/or gateway courses. It not only benefits the students but also actually ends up saving faculty time once they get used to. Students can auto-schedule appointments within faculty-selected blocks of office hours. This saves all the back and forth conversations, emails, and phone calls to schedule appointments. It's very efficient. The students also like it as it gives the clear message to them that their professor is available and wants to help them. - Brian Kinkle – Biology

Faculty Testimony Dan Waddell - Chemistry In general, I have been very pleased with Starfish. There are two main aspects of Starfish that I have utilized. One is that I set up my office hours so students can book a time with me in advance. The students have really enjoyed this. I have probably at least doubled the amount of students that have stopped by my office and I believe it is because they feel more comfortable dropping in if they have a scheduled appointment. I have always given students updates about their grades but it might have been too easy for them to stretch the truth of the situation when talking to advisors - now the advisors have all the information in front of them as well. I also like that Learning Community leaders now see students with concern or kudos as well. Peer to peer advising might be one of the more powerful tools that we have at our disposal. Again, overall, I'm very happy with Starfish and I see myself continuing to use this service.   Dan Waddell - Chemistry

Advisor Testimony I would say this has allowed me personally to get out ahead of a few scenarios where a student wasn’t a) aware of the possibility or deadline to withdraw, and b) was of the impression they couldn’t withdraw, but really could, c) didn’t realize they could actually get clarification from their instructor on their grade.   I’ve had a few students each semester email to thank me for “looking out” for them, so a bit of rapport building as well. Lastly, it’s allowed me to be prompted to be aware of students having difficulty in a class = helping with meeting prep. -Stephen Sovilla, Center for Exploratory Studies

Moving Forward with Starfish Identify methods for consistent and personalized intrusive advisor response to student flags Central support for monitoring student tracking through the newly created Center for Pathways Advising and Student Success Standardized protocol and expectations for assigned advisor response Deeper connections with other campus support networks for continued flag outreach Build stronger connections between Connect and Early Alert (referral, to-do lists, etc) Clean-Up individual student success networks within Starfish (advising, campus resources, etc) Identify solution to bogus Blackboard student e-mails Create Faculty, Advisor, and Student Starfish advisory committee to evaluate tracking items, e-mail templates, and protocol

Appointments (AY 15-16) Total: 26633

Going Forward Optimization of features in Starfish More assessment Success plan To-do assignments Success Network clean-up More assessment Addition of more courses More appointment scheduling options Catalyst & Starfish Integration

Thank You You can find additional Starfish resources at www.uc.edu/starfish. If you have any additional questions, please contact denise.ellis@uc.edu or call 513-556-2192