Institutional Course Evaluation Solution Faculty Senate Executive Committee September 12, 2012 Carol VanZile-Tamsen, Ph.D.; Associate Director, Office of University Accreditation and Assessment Joseph Mantione, Assistant Vice Provost, Institutional Analysis

Currently at UB: Five different course evaluation solutions are used across the 12 units Additional systems used by academic support units delivering instruction

Currently at UB: In some areas, preparation of evaluation forms for course offerings is an administrative nightmare Students enrolled in courses offered by different units do not have a single place in which they evaluate all of their courses In many areas, response rates are too low for any meaningful conclusions to be drawn

Currently at UB:

ConnectEdu in the College of Arts and Sciences UBCATS - “University at Buffalo Course and Teacher Survey” - A web-based software application that allows students to anonymously evaluate courses and instructors online.   Data Upload Process Upload of Course, Enrollment, Faculty, Student, TA, and Supplemental Instructor tables to the site Fall 2011 71,658 Student Enrollment Records 17,095 Student Import Records 2,350 Course Import Records 1,212 Faculty / Department Records 396 Teaching Assistants Records Departmental Access Process (Dept. Chair, Assistants) Set up the survey – keep it open for a month – close it two days after the last day of final exams.

ConnectEdu in the College of Arts and Sciences Communication Process Place student announcement and survey link on myUB Send Emails (17,078 initial emails) to Students to: Inform them that the evaluation site is open Invite them to log in and do the evaluation). Send Emails (1,137 initial emails) to Faculty to: Inform them that the survey is open Ask them to monitor response rates (real time) Remind their students to do the evaluation Send follow-up weekly emails/reminders to students with incomplete surveys (15,719, 14,072, 12,652, 11,733 emails in Fall’11) Send final Email to Faculty at the close of the survey to let them know that they can log in and access the evaluation results the day after web grading closes for the semester. Reporting Process Distribute our own custom Summary Reports to Dept. Chair and Director of Graduate Studies.

ConnectEdu in the College of Arts and Sciences UBCATS Challenges 1 Data upload- creation of new HUB querying tools 2. Undeliverable Emails – students not using/checking their UB email 3. Lack of Summary Reports 4. Departmental buy in – lack of ownership 5. Limited technical support 6. Confusion among students – not all UB courses are part of UBCATS 7. I can’t log in! 8. Reports are not accessible to students 9. Response rate 10. Survey fatigue   To some degree, the other 11 schools, Athletics, MFC and EOC all do the same thing and incur the same PSR costs. They also have the same challenges.

The Benefit of A Single Solution: Uniformity in Administration leading to greater efficiency and more reliable results Greater transparency for students and one-stop evaluations may encourage greater response rates Availability of Data and Reports for Assessment Purposes: Faculty review and development Program Review Accreditation Reporting

Course Evaluation Items: For comparability of ratings across courses and instructors, a core of common questions could be useful: 3-5 questions These would be developed jointly by Faculty Senate Executive Committee and Associate Deans Council Beyond that, we would promote the greatest flexibility possible for units in selecting questions

Vendors already on campus Campuslabs http://www.campuslabs.com/ (cost proposal submitted) ConnectEdu http://www.connectedu.com/postsecondary-solutions/college-evaluation-suite/college-course-evaluation.html (cost proposal submitted) Digital Measures http://www.digitalmeasures.com/course-evaluation-software/ E*Value https://www.e-value.net/ SmartEvals http://service.onlinecourseevaluations.com/ *Blackboard http://www.blackboard.com/ (no course eval solution at present)

Conclusions Regardless of platform chosen, there are many good reasons to have an institutional course evaluation system We hope you will consider data use and reporting reasons, as well as fiscal reasons, as you consider this issue.