Enterprise Course Evaluation at Georgetown: The Long Road from an All Paper to a Totally Online System CE3 User Conference September 29, 2011 Boston Charles F. Leonhardt

History Enterprise paper-driven system custom developed in ADABAS/Natural in 1980 Mainframe-based Integrated with SIS+ for Faculty, Course, and Student information Contains 30+ years of longitudinal survey response data Printed 90K+ evaluation forms each semester to be hand delivered All Faculty, Departmental, Decanal, and Provost reports printed and hand delivered

The Problem Space Migrate paper evaluation forms to a web environment Maintain student privacy and absolute confidentiality Migrate all reports (which include 30+ years of longitudinal data) to an online environment Maintain or increase Student response rates Enhance fixed surveys with optional questions from faculty members who desire this option Support new types of online surveys Build a data warehouse for enhanced reporting, data mining, and research

Other Virtues Go Green Reduce or eliminate paper Reduce or eliminate deliveries Retire Mainframe Eliminate High Speed/High Cost Printer Eliminate need to replace/maintain obsolete scanner Go Mobile Deliver a 21 st century service that students expect

The Obstacles Rigorously document our enterprise-wide requirements Reviewing likely online candidate systems against them Student information system replacement project Funding Creating a strategy to replace the deliverables that a COTS will not provide Faculty resistance to change Legitimate fear and concern regarding significantly lower response rates

High Level Requirements Exact duplication of the paper form NetID enabled Interfaces with legacy reporting system until that system is replaced Optional additional questions from Faculty (from a question bank) Completely anonymous Mobile Faculty control survey start and end dates/times Paper request process replaced Paper reporting system replaced

Use CoursEval3 for Online Surveys Use CoursEval3 and paper forms Continue with paper forms exclusively Solutions: Surveys

Solutions: Historical Data Build Data Warehouse for longitudinal data Migrate longitudinal data to CoursEval3

Solutions: Reporting Use CoursEval3 standard reports Build custom reports from the data warehouse

The High Level Design

Timeline 1980 Custom paper based system developed in house One survey after another exactly the same as the one before… 2003All campuses begin to discuss CE replacement 2003Law Center moves to school-based home grown system 2004School of Medicine moves to CE3 for online

Timeline Main campus develops requirements seriously SIS+ replaced by Banner (a major diversion) 2008Scholarly Systems Group assumes responsibility for CE from Student Systems Group 2009 Online candidates evaluated against requirements CoursEval3 chosen for Main Campus online pilots 2010Spring Pilot for 150 Courses successful

Timeline 2010Replacement project funding 2011Project Manager hired into SSG to manage CE Replacement Project 2011Faculty vote to make CE3 the default survey option for Fall 2011 Faculty vote to accept online evaluations exclusively for Fall 2012 and beyond under 3 conditions: 1) Faculty Control survey time 2) Mobile service is available 3) Response rates are monitored

Where We Are Now Fall 2011 CE3 in production as the default option Paper forms available on request Faculty control survey date/time beginning and end Interfaces built among Banner, data mart, and CE3 (to load course data into both environments) Some architectural details on reporting need discussion and resolution Spring 2012 Report reconstruction from data mart Interface built between CE3 and data mart (to load survey results for reporting) Fall 2012 Fully online !