1. Automating Post-Instruction Student Feedback with eXplorance Blue Pathways from IR to IE November 6, 2014 Bloomington, MN Daniel Glover, M.S. Office of Measurement Services University of Minnesota

2. Agenda History Scope Policy What we have been using What we are doing with Blue

3. History of Student Course Feedback at University of Minnesota Pre 1993: Voluntary Student Opinion Survey (SOS) Fall 1993: Student Evaluation of Teaching (SET) Fall 1994: First unit customized SET form c. 1997: Some course identification sheets barcoded Spring 1998: Student Release (SR) items added Fall 1999: University databases centralized Spring 2003: Web based data collection started Fall 2007: 54 SET customizations for units; 67 instructor individualized web customizations

4. Student Course Feedback History (& Future) Spring 2008: Student Rating of Teaching (SRT) implemented Fall 2009: Crookston campus added to SRT web collection Spring 2010: Morris campus added to paper collection Fall 2013: eXplorance Blue pilot (CEHD) Spring 2014: Blue rollout to “current” users Summer 2014: Retire legacy in-house web collection system Fall 2014: Start to incorporate Medical School and other units Spring 2015: SRT form revision implemented Spring 2015: Major central database revision (PS 9.x) Fall 2015: ???

5. What is the Univ. of MN? Multi-campus Land and Sea Grant public research university Undergraduate (including approx. 300 high school class sections per term), graduate and professional studies Twin Cities, Duluth, Morris, Crookston, Rochester campuses Founded 1851 (Minnesota became a state in 1858)

6. Course Feedback by Students Governance/Policy Student Release Photographs SRT Policy Update PeopleSoft Update Decentralization to Centralization

7. What Preceded Blue? Homegrown web collection system built by central Information Technology with focus on distance education courses. Homegrown data collection and reporting system built by OMS (mySQL database with web interface for operations and links to central database). These two systems did not talk to each other directly.

8. What’s In Blue Mobile compatibility Web based delivery Accessibility compliance Moodle integration PeopleSoft integration Multi-instructor SRTs University policy and data privacy regulation compliance Integration with University Data Infrastructure

9. How is Blue Helping IE? Distributes decision making closer to where they are made Integrated with central database Manages open collection times for short courses Helps with response rates

10. Blue Gets Units Involved All Dept. courses are visible to Dept. Administrators Paper/Online collection options will be chosen by Dept. Administrators Links sent to Dept. Chair and other report readers for SRT Reports

11. How is Blue Helping IE? Distributes decision making closer to where they are made Integrated with central database Manages open collection times for short courses Helps with response rates

12. What’s in Blue? Anything you can feed to it from another system.

13. Employee info has been kept minimal until after database migration.

14. Extending Blue Wanted to link as much as possible to our central database. (Course, instructor, instructor classification, unit and course classifications, cross-listed course info, other university course info (e.g., SLO)) However, not everything we need is available in our central database.

15. Developed Website to Guide Data Feed. Manage database provided information Manage other semi-dynamic data streams

16. Limit terms pulled from central system.

17. May be used for simplifying departmental updates.

18. Used to filter which courses are pulled into the system.

19. Can we manage the unmanageable? Keeping track of supervisory functions across units. Much more work remains. Planning on leveraging other BI processes.

20. How is Blue Helping IE? Distributes decision making closer to where they are made Integrated with central database Manages open collection times for short courses Helps with response rates

21. Dynamic Days Managing Process for Short Duration Courses (Traditionally Under Reported)

25. How is Blue Helping IE? Distributes decision making closer to where they are made Integrated with central database Manages open collection times for short courses Helps with response rates

26. What is BPI? Integrated with our Moodle platform for both Student “Form Fill Out” and “Subject View Management” process for instructors/departmental administrators management of the paper/online format selection.

27. Students with any type of Moodle exposure see all course with web/online data collection in their feed.

28. Response Rates (Institutional Policy Prohibits Incentives) Vendor provided additional handling during Fall 2013 pilot for web data collection. Moodle site went down right when Summer 2014 data collection was occurring. OnlinePaper CoursesInstructorsEnrolledRespondedResp %CoursesInstructorsEnrolledRespondedResp % Spring 201311511286299241441448.17%4980544416649712344574.14% Summer 20131621702918128844.14%56663111941983582.36% Fall 20139721118271801318748.52%4778521617207912881274.86% Fall 2013 Blue3393898653485156.06%39840710625897884.50% Spring 201411701602394761970149.91% Summer 20143113315230188035.95%4655129702858588.49%

29. What Can Instructors Do? Instructors are able to view response rates, but not responses nor respondents during open period and can encourage participation beyond the usual emails.

30. Some Issues We’ve Encountered Size (we are large, larger than other eXplorance educational clients) Data Structure (internal University data is organized around matriculation rather than completion)

31. Questions?

32. Thank you!