1. Collaboration and Education Group Jonathan Grudin Microsoft Research jgrudin@microsoft.com

2. Collaboration and Education Group Formed about 12 months ago Mission:  To explore novel technologies and applications that enhance collaboration and education / training  Current work focuses on streaming media Research model Evaluation: Laboratory and Field Studies Build Prototype Evaluation / Publication Refine Prototype Product Impact

3. Technology and Education Two broad facets:  Technology for improved content  deep models of subject matter and student  active exploration of subject (simulations)  relate to students context/environment (situated learning) MOSTLY DOMAIN DEPENDENT  Technology infrastructure for:  course and student management  content creation  delivery / distribution  collaboration MOSTLY DOMAIN INDEPENDENT Both aspects are important and complementary

4. Technology Adoption Phases Phase-1:  digital version of non-digital process Phase-2:  value-added features appear in digital version Phase-3:  process and technology re-design

5. Why Consider Multimedia? Network, processor, memory capability changing quickly  Reasoning about exponential growth Simultaneous emergence of live and on-demand capability Shift in the definition of scholarship

6. Ongoing Projects MSTE and MURL: Online Seminars Time Compression, Skimming, Indexing, Browsing MRAS: Multimedia Annotations and Authoring Flatland: Telepresentations

9. MSTE Presentations Logs of ~10,000 sessions by over 2000 users Some results:  On-demand audience about 40% of live audience  60% < 5 minutes  Viewers jump around video  Initial portions much more likely to be watched Presentations will be designed differently in future  Present key messages early in talk  Present key messages early in slide  Use meaningful slide titles  Reveal talk structure in slide titles  Consider post-processing talk for on-line viewers

10. Analysis of Online Presentation Viewing Logs of ~10,000 sessions by over 2000 users Some results:  On-demand audience about 40% of live audience  60% < 5 minutes  Viewers jump around video  Initial portions much more likely to be watched Presentations will be designed differently in future  Present key messages early in talk  Present key messages early in slide  Use meaningful slide titles  Reveal talk structure in slide titles  Consider post-processing talk for on-line viewers

11. Ongoing Projects MSTE and MURL: Online Seminars Time Compression, Skimming, Indexing, Browsing MRAS: Multimedia Annotations and Authoring Flatland: Telepresentations

12. Time Compression, Skimming, Indexing While text documents are easy to skim, that is not true for audio-video Ability to skim can be a key advantage of web-video  time-compression: up to ~2-fold; nothing thrown away  skimming: > 2-fold; some content thrown away  indexing: adding navigable structure Also useful in “live” broadcast scenarios  e.g., late joiners can catch up to live talk

13. Time Compression: Synchronized Audio and Video To preserve pitch: throw away portion of each 100ms chunk, then stitch together Basic signal processing well known, but several systems issues Results of lab studies:  People choose ~1.4 speed, don’t adjust much  They like it  “I think it will become a necessity… Once people have experienced it they will never want to go back. Makes viewing long videos much, much easier.”  Comprehension may go up

14. Time-Compression Demo

15. Skimming: Compression Goes Nonlinear To beat 2x speedup, must throw away content Sources of information  audio: pauses, intonation, speech-to-text and NLP  video: scene changes  other: slide-changes, previous viewers’ patterns Lab studies of 4x-5x speedup  Viewers learn from automatic summaries  Viewers like and learn more when author-edited  Perception of quality increases over time Mixed-initiative summarization is promising

16. Indexing Vanilla video provides no structure for navigation Indexing provides navigable structure; examples:  textual table of contents (slide titles)  video shots / scenes  speech-to-text => NLP => topic detection

20. Ongoing Projects MSTE and MURL: Online Seminars Time Compression, Skimming, Indexing, Browsing MRAS: Multimedia Annotations and Authoring Flatland: Telepresentations

21. Multimedia / Temporal Annotations Motivating scenarios:  a virtual university  all students are remote, asynchronously watching lecture videos  a standard university  making better use of in-class time Temporal annotations:  annotations associated with streaming media  each annotation is linked to the media time-line  annotations stored separately from the media files

22. Ability to annotate can add significant value  shared notes for asynchronous collaboration  question-answers linked to a streaming-video lecture  archived feedback for the instructor  personal notes on audio-video found on the web  personal/shared table of contents; summarizations  annotations may be computer generated  use speech-to-text providing search and seek ability  captured strokes from electronic white-board  captured questions, slide-flips, from “live” broadcast ...

26. Results from Preliminary User Studies Personal note-taking study (MRAS vs. Paper)  similar # of notes (~1 / minute)  positioning: none in paper; ~10-15s later in MRAS  all subjects preferred MRAS (although more time), and thought more useful for future reference Shared notes study  text preferred to audio  14/18 stated more participation than in “live” session  auto-tracking particularly useful

27. Currrent Work MSTE class to use MRAS and recorded lectures  Can we increase instructor productivity?  Can we emulate live-classroom discussion / community formation in an asynchronous environment using MRAS?

28. Ongoing Projects MSTE and MURL: Online Seminars Time Compression, Skimming, Indexing, Browsing MRAS: Multimedia Annotations and Authoring Flatland: Telepresentations

29. Flatland Tele-presentation System Joint project with the Virtual Worlds Group  Flexible architecture for distributed collaborative applications Target scenarios:  presentations to remote audience  online conferences  distributed tutored-video-instruction ...

30. The Flatland Project

31. Do We Need to Sacrifice Quality? The goal is to improve it Stanford Tutored Video Instruction (TVI)  Process:  video tapes of un-rehearsed live lectures  small group of students watch along with a para- professional tutor Results from 1978-86  All MSEE: 1800 students, avg. GPA 3.40  TVI-MSEE: 89 students, avg. GPA 3.62 Similar observations recently for D-TVI version

32. Stanford TVI Experiments: 10/73 - 3/74 remote TVI students with tutor do best it helped “at-risk” students even more Source: J.F. Gibbons, et al. Science, Vol. 195, No. 4283, 18 March 1977

33. Flatland Experiences Initial use in 3 multi-session MSTE classes  Presentations from desktop to remote audience Students:  Liked the convenience  Liked ability to multitask  Did not think learning suffered Instructors:  Missed familiar sources of feedback  Comfort level rose over time for 2 of 3 Overall: Lack of awareness of others a key problem

34. Issues Being Explored Creating presence and awareness  representing attendees; gaze; activity level;... Providing for interactivity; protocols for online talks  types of widgets; floor control; multiple back channels Complexity of interface for speaker / audience  use of channels over time; different physical contexts; … Capture and replay of tele-presentations  capture “all” activity; time-compression; annotations

39. Activity Surrounding Teaching/Learning Pre-authoring  Slides, web notes, reference material, exercises, … Content delivery  Synchronous delivery to local/remote audience  Archived for on-demand audience and review On-demand access by students  Watch content; personal notes; TOC; index; … Discussion around content  Synchronous: small group; one-on-one  Asynchronous Post-lecture work by instructor / tutor  Answer questions; discussions; feedback & redesign; …  Student evaluation …

40. Concluding Remarks Key drivers of change  market needs  technology Key new directions  learner-centric  asynchronous; small-group synchronous Key challenges  concrete studies to indicate effectiveness  technology/products taking value beyond cost  business model and bootstrapping issues

41. For More Information: http://www.research.microsoft.com

42. Watching Behavior Within a Session 0 10 20 30 40 50 60 70 0102030405060708090 Nth minute into the talk User count