1. Supporting Online Engineering Education: Lessons Learned and Best Practices Sheila Warren Doctoral Candidate, Instructional Technology Stephanie Moore, Ph.D. Director of Engineering Instructional Design

2. Sheila Warren Curry School of Education sw2xe@virginia.edu Stephanie Moore Director of Engineering Instructional Design University of Virginia stephanie.moore@virginia.edu

3. Overview Two separate distance programs  Engineers PRODUCED in Virginia Undergraduate UVA students  Commonwealth Graduate Engineering Program (CGEP)  Five Virginia universities offering cooperating to offer graduate engineering degrees via distance education  Some students are enrolled in other universities so we cannot levy specific hardware requirements Each program has different support needs  Number of instructors  Frequency of teaching distance students

4. Technology Used Synchronous classes in Blackboard Collaborate Electronic inking through tablet or Smart Board Video feed of instructor Microphones for instructor (and classroom) Students have microphones

5. Our Goals and Responsibilities Assure that the distance students receive the best quality education we can provide Provide instructors with appropriate support to teach all their students without unnecessary burden caused by technology We provide the training and support for the technology platform that delivers the online classes Administrative need: to have data basis for informed decisions on technology selection and investment

6. Challenges Approximately twenty separate classes taught each term Rotating roster of instructors Rotating roster of graduate students Classroom environment

7. Framework for Practice Based on the idea of performance dashboards (use of technology to improve organizational performance) as part of a performance support system (PSS) within an educational setting (Villachica & Stone, 1998) Recognize that solutions will be different for other contexts because of varying needs, constraints, etc.

8. Support Structure Technology recommendations for distance students  Higher interaction with undergraduates choosing tablet computers and software Training instructors and teaching assistants in the classroom technology platform Support available for every class  Student workers monitor the classes and are ready to troubleshoot and offer assistance if technical difficulties arise  Collect and submit the raw data from each class

9. Data Collection Guides the team in determining what areas need attention, where to focus resources, how to resolve and prevent recurrence Supports administrative decisions and information at the university level

10. Types of Data Collected Issues, glitches, problems, or concerns during a class period such as  Software: Blackboard Collaborate, PowerPoint, etc.  Hardware: Microphones, in-room computers, tablet, Smart boards, camera  Infrastructure: University-wide internet, building-specific internet, university hosted applications  Human Error: Training issue, performance issue, simple mistake Responsible entity  Our Team, the instructor, a student, university services, mother nature Level of Impact  Minor annoyance through class cancellation

11. Lessons Learned Programmatic  Sound quality is a major issue  Better training for instructors  Contingency plans for audio, video, software malfunction Support Services  Be ready for anything Data Collection  Try to avoid changes to data collection process after start of semester  More dropdown menus and checkboxes will result in less descriptive qualitative data  There is a cutoff to the data people will consistently provide  If the data is perceived as too cumbersome or unimportant, it will be unreliable

12. Next Steps in Data Collection and Analysis Automate data processing Improve communication and visualization of data  Present a basic set of charts  Allow user to define additional parameters for specific, detailed reporting Review videos to check for accuracy of reporting Attempt to train for inter-rater reliability

13. Proposed Web-based Form

14. Current Google Docs Form

15. Semester Totals

16. Sample Data Reports

17. Thank You! Contact information: Sheila Warren sw2xe@virginia.edu sw2xe@virginia.edu Stephanie Moore, Ph.D. stephanie.moore@virginia.edu stephanie.moore@virginia.edu