Changing the Course Evaluation Process at by Jean-Pierre R. Bayard, PhD

ONE Roadmap… From Paper to Online Evaluations One of the largest academic IT changes for Sacramento State. Significant complexities: Campus culture/tradition Collective bargaining Promotion and tenure awarded primarily on the basis on teaching

Notes on Choice of Scantron Class Climate Not an advertising for Scantron Scantron Class Climate was the correct decision for Sacramento State.

Sac State’s Background ~28,000 students/FTES, and offers ~5000 course sections/semester. Spring 2013, we delivered ~2500 course evaluations online, an increase of 800 in one semester. We used Flashlight Online before; now, Scantron Class Climate. The CSU had required 2 courses be evaluated/year for each faculty member, but now ALL courses are.

The Problem Our paper evaluation process took months to give the reports back to departments. The workload was substantial, and a San Francisco study reported reduced staff workload with moving to online. There was no way for us to have data analyzed across programs, let alone colleges.

Why NOT go ALL Online?... Return rates for online evaluations are typically lower than for paper. Departments return rates average 24– 65%, with several reminders to students. These average rates include all enrolled students, even those who have stop attending classes.

…Why NOT go ALL Online? Part-time faculty’s employment relies heavily on course evaluation scores. Over half of Sac State’s faculty are part-time. If the University mandated online course evaluations, litigation with CFA on behalf of their part-time faculty clients would be likely.

The Strategy… An opt-in policy… Each instructor chooses between an online or paper course evaluation. Success is in the numbers Only 2 faculty have ever chosen to go back to paper evaluations

A Bump in the Road Ahead Now we are also improving paper evaluations using Scantron Class Climate (SCC). Will this detract from the growth in online evaluation numbers? Return Rate Improvement: SCC allows students to generate a certificate of completion, and identifies students who did. Select an improvement strategy that aligns with campus culture.

The Data Opportunity… To take the next important steps, we need campus leadership. While course evaluation data is student polling data, it is course data from the students’ perspective. In order for data to be meaningful, age-old evaluation instruments need to be redesigned with shared values.

…The Data Opportunity How data is used and by whom, is another difficult governance discussion. We now have access to large course data to use for a variety of purposes, including WASC, student satisfaction in groups of courses (i.e., GE, eLearning, lower/ upper division, and graduate courses).

Caution/Suggestion Select a tool from a company with long-term course evaluations experience. Tools available in LMS, are okay for cost & convenience, but not quality. Decouple some academic IT processes from the LMS.

Our Next Step Scantron has been very responsive, and innovative in coming up with solutions. Their survey interface needs a 21st Century redesign, but it works reliably. Our next step is to implement the data bridge, to connect our evaluation system to CMS and our data warehouse.

My thanks to… Kimo Ah Yun (Former CTL Director) Jing Wang (Institutional Research) Kent Porter (Human Resources)