Implementing an Online Course Evaluation Solution: Navigating Faculty Governance and Endorsement Jeffrey Cepull, Philadelphia University Brian Hopewell, Academic Management Systems Cpyright J. Cepull and B. Hopewell, 2009. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors.

Philadelphia University Undergraduate and graduate programs Six schools 3,400 total student head count 2,850 total undergrads 1,250 resident Numerous online student & faculty services

Course Evaluation - History Paper-based Coordinated by one individual (adjunct faculty) No University-wide office Six schools, six question sets Delayed processing each semester Complicated pre-survey set-up Enormous time spent on post-survey handling

Early stages - Fall 2006 Extensive literature search Influence of the UMinn EDUCAUSE presentation Efficacy at <100% response rates Identify a group of early adopters (faculty) Arrange a trial period for testing (AMS) Small-scale pilot at the end of 2006

Comprehensive Pilot - Spring 2007 Online evaluation pilot discussed at full faculty meeting Faculty comprised of champions, curious observers, and skeptics Proposed a larger, more comprehensive pilot, focused on undergraduates Partner with Academic Affairs Call for volunteers and target certain faculty Faculty balloting tool

Faculty Governance and Endorsement - Fall 2007 Approached the Faculty Affairs and Development Committee Presented data Emphasized rapid access to course feedback Ease of adding summary data to faculty dossiers Sought endorsement for the “option” for faculty to use the online solution

Seeking Additional Endorsement - Spring 2008 One-on-one meetings with Deans One-on-one meetings with faculty skeptics One Dean mandates the solution Participation increases Faculty Committee is tasked with revising the six instruments OIR staff provides extensive support Student balloting tool

Gaining Committee Support - Fall 2008 Targeting tenure and promotion committees Continue working with the Deans A second Dean mandates the online solution Participation increases

Online Evaluations – Total Sections Semester

Response Rates Percentage Semester

Placing Emphasis Faculty focus Eliminates the use of class time Immediate access to results Easy to read student comments Summary materials can easily be integrated into a dossier Efficacy of the online approach

Additional Benefits Course evaluations are preloaded Evaluation period is three weeks Immediate access for deans Eliminated post survey processing Students received automated reminders to complete surveys Faculty updates on student participation

Major Factors for Success Extensive literature search Aggressive sideline position Look for a success story Small-scale pilot with early adopter faculty Larger pilot focused on faculty volunteers Always refer to the “pilot” process Partner with Academic Affairs Engage the Deans

More Factors for Success Engage the skeptics Discuss at faculty and committee meetings Ballots and surveys Create pathways to adoption (Help Desk) Work toward small victories (option for faculty) Accommodate reasonable faculty requests Be flexible with the online solution Again, always refer to the “pilot” process

CoursEval Academic Management Systems “founded by deans & for deans” 15 years in market 150+ customers in USA, Canada, etc. CoursEval customers Looking to save time and money Deeply concerned about “process” (“evaluation is the 3rd rail of academic admin.”) Academic Management Systems

Email reminders Academic Management Systems

MyCoursEval Remote Portal Plug-In works with Blackboard, WebCT, Moodle, Sakai and other campus portal systems. Alerts students to pending evaluations. Academic Management Systems

Secure, private access Campus logo or program logo here Log-in info and message for students and/or faculty Several authentication choices are available Administrator feedback via quick e-mail Academic Management Systems

Use your own survey instruments Sample student survey customized with campus-specific questions, course and teacher ID, special instructions, multiple faculty, and many other options (including question formats faculty photos and required answers) Flexible answer formats Non-course surveys Long-answer windows Academic Management Systems

Participant report: real-time lists of compliant and non-compliant evaluators By individual compliance within class. Remind delinquents w/email Academic Management Systems

Sample faculty report Campus, date,and course ID Faculty report has many format, display, and data options Chart with selected ratings, counts, comparisons, and response rate Question category Easy to read ratings and comparisons (many options) Collected written comments Academic Management Systems

Cross-institutional reporting An ‘At-A-Glance’ view of ratings by department (questions selected by viewer) Academic Management Systems

Department level reporting An ‘At-A-Glance’ view of ratings by course and instructor (questions selected by viewer) Academic Management Systems

Tenure and Promotion Summary Report All evaluations in archive for professor Academic Management Systems

Export results to .xls, .csv, .dbf files Academic Management Systems

Faculty evaluation of students Customize evaluation criteria using your own rubrics. Academic Management Systems

References & Resouces Cohen, P.,1980, “Student ratings of instruction and student achievement: a metaanalysis of multisection validity studies,” Review of Education Research, v 51, pp. 281-309 Seldin, P., ed., 2006 Evaluating Faculty Performance, Anker, Theall, M. and Franklin J. (eds.) 1990, Student Ratings of Instruction: Issues for Improving Practice, Jossey-Bass, Brigham Young University: Trav Johnson http://onset.byu.edu/onset/index.php?title=Main_Page Academic Management Systems

What did you think? Your input is important to us! Click on “Evaluate This Session” on the Mid-Atlantic Regional program page. Contact AMS for information on no-cost “pilot.”

Questions…. Contact information: Jeffrey C. Cepull Vice President for Information Resources and CIO Philadelphia University 215-951-2516 Cepullj@philau.edu Brian R. Hopewell Director of Business Development - Higher Education Academic Management Systems 716.867.8434 bhopewell@academicmanagement.com www.academicmanagement.com