Online Course Evaluations Report from Ad hoc Committee

Overview The history at CofC An online evaluation system History Advantages Concerns Challenges Current status

History - Summary In 2001, an ad hoc committee was formed to investigate a transition from a paper-based evaluation system to an online version. In 2005, the FETC was charged with studying the feasibility of implementing an online course evaluation system at the College The FETC brought a motion to adopt an online evaluation system to the Faculty Senate; the motion was not approved. A pilot study was conducted in to determine the feasibility of implementing such a system. Online course evaluations were adopted and fully implemented in Fall 2010.

History Paper evaluations - manual multi-step delivery system involving multiple parties. Issues: Lack of security Confidentiality and privacy may be compromised – hand written student comments Data are difficult to analyze. Results printed in paper format – not easily extracted. Slow feedback - 15 weeks required to prepare, deliver, return and tabulate the paper forms. Static and inflexible system.

Issues with paper system - Labor Pre-evaluation labor (IT) Total: ~ 205 hours + Administrative Assistants time in each department (~8 hrs) Forms still had to be returned for additional processing/scanning by IT and AAPA once they had been completed. They also had to be copied at the departmental level.

Issues with the paper system – Financial considerations The process was expensive (paper and printer) ~ 50,000 – 70,000 evaluation forms per semester not including paper copies issued later to faculty. Printing: ~$7500 Purchasing forms: $3000 / year Printing forms: $3600 / year Errors in printing: $800 / year Hardware: $10,500 Grand Total = ~ $18,000 + labor costsGrand Total = ~ $18,000 + labor costs

Online System – Advantages Rapid feedback. Student comments are returned immediately after the semester ends for formative use before the next semester starts. Anonymity. Student comments are typed. Analysis. Results are returned electronically in a form more suitable for data analysis. Enhanced security. Less expensive to administer. No unusable forms (double-bubbling). forms

Concerns with an online system Response Rate The literature shows that student response rates decreases following implementation (although response rates of over % have been reported using online course evaluation systems). Response rate – paper-and-pencil forms is 67%. Response rates generally recover over a period of time, generally 3-4 years. Response rate can be encouraged with positive reinforcement incentives.

Pilot Program Vendor selected was Digital Measures. The FETC investigated 25 other institutions at the time of the pilot program. Many of the institutions surveyed (our competitors) had moved to an online system at that time.

Challenges The student body must be convinced that their information is of value. Incentives must be built into the system to encourage student participation. Faculty must buy into this system and sell this system in their classes. The system must be marketed / advertised intensively to make it work.

Ad Hoc Committee developed following implementation – Fall 2011 Deanna Caveny-Noecker (Academic Affairs; Mathematics; Co-chair) Bethany Goodier (Communication; Co-chair) Mark Hurd (Psychology) Claire Curtis (Political Science) Martin Jones (Mathematics)

Current Status Vendor – Blue Portal One metric – Response rates (RR) Spring 2013 – 37% Fall 2012 – 34% Spring 2012: 37% Fall 2011: 32% Spring 2011: 32% Fall 2010: 41% Mean = 36%

What have we done to improve RR? Shorten the form: Removed Student Course Information Publication (SCIP) questions to reduce form length (with student input). Blue Portal integrator purchased: Mobile app implemented. Working on better integration with OAKS for reminders to students.

What more can be done? No silver bullet to fix issues and improve RR. Improve marketing - use of social media Reduce the length of the form Other faculty committees and Senate must be involved. One option: Use of mobile app in class – students can use smart phone, tablet or computer to complete evaluations as they did with paper evaluations.