David Joyner April 5th, 2018 Hasso Plattner Institut Understanding the Relationship between MOOC Viewing Behavior and Completion Rates David Joyner April 5th, 2018 Hasso Plattner Institut

Talk Outline Personal Background Georgia Tech Research Community Research Aggregated Guidelines Questions

Personal Background About David: Senior Research Associate in Georgia Tech’s College of Computing Associate Director of Student Experience for the OMSCS Program Director, LucyLabs Creator and Instructor, CS6460: Educational Technology and CS6750: Human-Computer Interaction Creator and Instructor, CS1301: Introduction to Computing Course Co-Instructor: CSE6242: Data & Visual Analytics and CS7637: Knowledge-Based AI

Talk Outline Personal Background Georgia Tech Research Community Research Aggregated Guidelines Questions

Study #1: Predicting OMSCS Success By Question Findings Guidelines Nate Payne, Daniel Hegberg, and David Joyner Can video-watching data help predict success rates? Course success can be predicted in part by video-watching: pro-active, on-pace watchers get better grades. Authentically incentivize keeping up with video lectures.

Study #2: Video-viewer Typology By Question Findings Guidelines Georgia Tech VIP Team In what patterns do students watch videos and complete exercises? Student approaches can be distilled down to a small number of types: linear- watchers, exercise-jumpers, video- skippers, performance-optimizers, “slackers”. Ensure assessments are comprehensive.

Study #3: mOOCs vs. For-Credit By Question Findings Guidelines Georgia Tech College of Computing How do completion rates differ between for-credit online classes and not-for-credit MOOCs? Not-for-credit MOOCs have completion rates of 1% to 10%. For-credit online classes have completion rates of 80% to 85%. Investigate and understand student goals.

Talk Outline Personal Background Georgia Tech Research Community Research Aggregated Guidelines Questions

Study #4: Early Course Engagement By Question Findings Guidelines Suhang Jiang, Adrienne Williams, Katerina Schenke, Mark Warschauer, Diane O’Dowd Can early-course behaviors and performance predict success? Week 1 viewership and performance patterns predict course success; however, deliberate investment in week 1 experience may not work. Use early course performance data for rapid revision.

Study #5: Videos & Engagement By Question Findings Guidelines Philip Guo, Juho Kim, Rob Rubin How do specific video production decisions affect student engagement? Students engage more with short, headshot, hand-drawn, custom-produced, or informal videos. High production values have relatively little effect on engagement. Keep videos short and personal. Produce custom online material. Understand the role of production value.

Study #6: In-Video Dropout rates By Question Findings Guidelines Juho Kim, Philip Guo, Daniel Seaton, Piotr Mitros, Krzysztof Gajos, Robert Miller What predicts peak moments of student video interaction? Interaction peaks occur with first-time viewership, missed content, tutorial steps, visual transitions, and audio explanations. Vary delivery modalities for the same content.

Study #7: Information Processing By Question Findings Guidelines Tanmay Sinha, Patrick Jermann, Nan Li, Pierre Dillenbourg Can principles of information processing theory help predict attrition rates? Clickstream interactions can be abstracted into information-processing behaviors. Deliberative information processing predicts course completion. Evaluate clickstream data to find the rate of deliberate investigation.

Study #8: Fine-Grained Video Behavior By Question Findings Guidelines Nan Li, Lukasz Kidziński, Patrick Jermann, and Pierre Dillenbourg What do fine-grained video interactions (fat-forward, rewind, change speed, pause) Pausing, slowing, and repeating are indicative of difficult videos. Speeding up or skipping are indicative of easy videos. Pay attention to changes in video interaction patterns. Provide quick access to review videos.

Study #9: Overall MOOC Attrition By Question Findings Guidelines Hanan Khalil & Martin Ebner What predicts MOOC attrition rates? MOOC attrition is best explained by student-specific criteria: student time, student motivation, student feelings of isolation, lack of interactivity, and lack of correct background. It’s not just about video.

Study #10: Completion Rates By Question Findings Guidelines Andrew Ho, Justin Reich, Sergiy Nesterko, Daniel Seaton, Tommy Mullaney, Jim Waldo, Isaac Chung Are completion rates the right measure for MOOC success? Probably not. Stratify drop-out points. When there is no barrier to entry, there is no barrier to exit.

Talk Outline Personal Background Georgia Tech Research Community Research Aggregated Guidelines Questions

Talk Outline Authentically incentivize keeping up with video lectures. Ensure assessments are comprehensive. Investigate and understand student goals. Use early course performance data for rapid revision. Keep videos short and personal. Produce custom online material. Understand the role of production value. Vary delivery modalities for the same content. Evaluate clickstream data to find the rate of deliberate investigation. Pay attention to changes in video interaction patterns. Provide quick access to review videos. It’s not just about video. Stratify drop-out points. When there is no barrier to entry, there is no barrier to exit.

Questions? For more: omscs.gatech.edu www.DavidJoyner.net LucyLabs.gatech.edu Contact me: david.joyner@gatech.edu