2. Novel Visualization and Interaction for Small to Jumbo Displays Mary Czerwinski Microsoft Research Mary Czerwinski Microsoft Research

3. Who Contributed: VIBE Team

4. Large Display Surfaces Are Here Workstation in the real world

5. Overview Initial large display research Prototypes around usability issues observed Visualization and interaction New user experiences have to scale the wide continuum of displays Future directions Initial large display research Prototypes around usability issues observed Visualization and interaction New user experiences have to scale the wide continuum of displays Future directions

6. Harris Poll Responses (7/02, N=1197)

7. no multimon 30% plan to use multimon 38% use multimon 32% Peddie Research, 2001 (N>6000) Multimon Usage Trends

8. Why A Larger Display Surface? Productivity benefits 10- 30% (despite sw usability issues) Users prefer more display surface Prices dropping Footprints getting smaller Productivity benefits 10- 30% (despite sw usability issues) Users prefer more display surface Prices dropping Footprints getting smaller

9. 1 st Prototype--dSharp Display Triple projection matrox parhelia card 3028 x764 resolution 42 in. across Slightly curved 120 degree FOV Triple projection matrox parhelia card 3028 x764 resolution 42 in. across Slightly curved 120 degree FOV

10. Task Times – Significant

11. User Satisfaction - Significant

12. Windows Layout - Significant

13. Tenets--Large Display UX User studies show large display surfaces fundamentally change user interaction Designed tools to better understand/ complement how work practice changes Large display surfaces provide non-linear productivity increases Additional space has different utility e.g. focal/peripheral displays provide different cues User studies show large display surfaces fundamentally change user interaction Designed tools to better understand/ complement how work practice changes Large display surfaces provide non-linear productivity increases Additional space has different utility e.g. focal/peripheral displays provide different cues

14. But…Usability Issues Why click to bring a clearly visible window into focus? caused many errors Where is my cursor? Where is my start button? Where is my taskbar? Where are my dialogs? The software doesn’t know where the bezel is… Why click to bring a clearly visible window into focus? caused many errors Where is my cursor? Where is my start button? Where is my taskbar? Where are my dialogs? The software doesn’t know where the bezel is…

15. Vibelogger 1st activity repository for studying windows usage in aggregate can profile users based on display size can be extended to visualize workflow and capture context single user: capture task contexts to surface pertinent ui or provide reminders 1st activity repository for studying windows usage in aggregate can profile users based on display size can be extended to visualize workflow and capture context single user: capture task contexts to surface pertinent ui or provide reminders

16. Multitask Visualization colored block for each time point and app amount of shading indicates percentage of visibility of the window tasks subtasks

17. Task Switching Visualization switching tasks (red to blue) how are email windows arranged and used? compare to... switching tasks (red to blue) how are email windows arranged and used? compare to...

18. Windows and Task Management Issues Emerge Larger displays = more open windows Multimon users arrange windows spatially Taskbar does not scale: aggregation model not task-based users can’t operate on groups of related windows Larger displays = more open windows Multimon users arrange windows spatially Taskbar does not scale: aggregation model not task-based users can’t operate on groups of related windows

19. Changes in Window Access Patterns

20. Input: Drag-and-Pop problem large displays create long distance mouse movement touch & pen input has problems moving between screen units solution drag-and-pop brings proxies of targets to the user from across display surfaces the user can complete drag interactions locally—no need to deal with distances or to cross display borders problem large displays create long distance mouse movement touch & pen input has problems moving between screen units solution drag-and-pop brings proxies of targets to the user from across display surfaces the user can complete drag interactions locally—no need to deal with distances or to cross display borders

21. Table Cloth Problem: User wants to access content physically far away Solution: Pan the desktop to user Compress content to the right of focus Grab content you need and snap back Problem: User wants to access content physically far away Solution: Pan the desktop to user Compress content to the right of focus Grab content you need and snap back

22. Multitasking Support Projectbar, layoutbar, groupbar Scalable fabric Task flasher Secret passwords Table cloth Wincuts Projectbar, layoutbar, groupbar Scalable fabric Task flasher Secret passwords Table cloth Wincuts

23. Task Management: Groupbar Taskbar for lightweight grouping of windows Allows for multiple bars, spatial placement of bars Desktop snapshotting; task snapshots Taskbar for lightweight grouping of windows Allows for multiple bars, spatial placement of bars Desktop snapshotting; task snapshots

24. Task management: Scalable Fabric Configurable central focus + periphery Easy task switch from periphery to focus Leverages human spatial memory Configurable central focus + periphery Easy task switch from periphery to focus Leverages human spatial memory

25. Task Flasher A more visual alt + tab Uses 3d scaling and selection animation Windows stay on the monitor on which they are positioned A more visual alt + tab Uses 3d scaling and selection animation Windows stay on the monitor on which they are positioned

26. Secret Passwords Problem: many touchscreen systems (eg. SmartBoard, TabletPC) have no keyboard Use software virtual keyboard Hard to hide password on a virtual keyboard Previous research showed users think anything on large display is considered public…they watch (Tan et al, CHI ‘03) Problem: many touchscreen systems (eg. SmartBoard, TabletPC) have no keyboard Use software virtual keyboard Hard to hide password on a virtual keyboard Previous research showed users think anything on large display is considered public…they watch (Tan et al, CHI ‘03)

27. Solution: Secret Passwords

28. Meeting Support: WincutsWincuts Meeting Support: WincutsWincuts What about co-located collaborative work? People bring different expertise and information on personal devices to meetings and need to share today’s model is broken—only one person gets to display entire desktop at a time (for better or worse) or, must share applications and files with others what about private information? What about UI that should be scaled for the task at hand? What about co-located collaborative work? People bring different expertise and information on personal devices to meetings and need to share today’s model is broken—only one person gets to display entire desktop at a time (for better or worse) or, must share applications and files with others what about private information? What about UI that should be scaled for the task at hand?

29. Visualization Research DateLens with Ben Bederson Facetmap Team Tracks (SW Vis) DateLens with Ben Bederson Facetmap Team Tracks (SW Vis)

30. Datelens with Ben Bederson Fisheye representation of dates Compact overviews User control Integrated search Integrated with outlook Pen-enabled Fisheye representation of dates Compact overviews User control Integrated search Integrated with outlook Pen-enabled

31. FacetMap

32. Scalable Visualization for “all of your stuff” Queries MyLifeBits SQL database interactively Domain includes web pages, digital images, documents, email, SenseCam physical activity data, VibeLog PC activity data Heterogeneous data, metadata, annotations, and relationships organized into “facets” Browsing and searching accomplished by iterative selection from among available item attributes (and/or full text search) Uses Piccolo.Net (on top of GDI+) for graphics & animation Scalable Visualization for “all of your stuff” Queries MyLifeBits SQL database interactively Domain includes web pages, digital images, documents, email, SenseCam physical activity data, VibeLog PC activity data Heterogeneous data, metadata, annotations, and relationships organized into “facets” Browsing and searching accomplished by iterative selection from among available item attributes (and/or full text search) Uses Piccolo.Net (on top of GDI+) for graphics & animation

33. Scalability Scales in three ways: Space: Recursive, space-filling algorithm generates useful views for any size screen Items: Large numbers of items aggregated/abstracted into groups and counts at multiple levels Facets: New item attributes can be added to the visualization dynamically Fixed minimum node size to guarantee readability Consistent information density Greater levels of detail are collapsed until they are feasible to present, given display constraints Scales in three ways: Space: Recursive, space-filling algorithm generates useful views for any size screen Items: Large numbers of items aggregated/abstracted into groups and counts at multiple levels Facets: New item attributes can be added to the visualization dynamically Fixed minimum node size to guarantee readability Consistent information density Greater levels of detail are collapsed until they are feasible to present, given display constraints

34. Medium Size (No Filters)

35. Medium Size (Filters Active)

36. Small Size

37. Large Size (Wall Display)(Wall Display) Large Size (Wall Display)(Wall Display)

38. FacetMap for Mobile Phones Quick searching through structured data Scalable UI Half visual and half text list Prototyping stage (desktop) Multiple visualizations (pick the best for a given results set) Quick searching through structured data Scalable UI Half visual and half text list Prototyping stage (desktop) Multiple visualizations (pick the best for a given results set)

39. SW Visualization: Team Tracks Goal: help developers new to a code base familiarize themselves quickly Based on logs of teams using the code base Assumption: most frequently visited areas of the code are more important We empirically verified this Show developers most related areas of code based on where they currently are Also give them previews of that code Goal: help developers new to a code base familiarize themselves quickly Based on logs of teams using the code base Assumption: most frequently visited areas of the code are more important We empirically verified this Show developers most related areas of code based on where they currently are Also give them previews of that code

40. Screenshot of Team Tracks

41. Evaluation Results Better task completion rates 9 / 9 completed tasks 1 and 2 (same) 3 / 9 completed task 3 (versus 1 / 7) dominated by algorithmic detail 7 / 9 completed task 4 (versus 1 / 7) dominated by finding relevant code fragments Better quiz scores (t(16)=-2.04, p<.03, one-tailed) Same importance ratings (r=0.45, p=.02) Better task completion rates 9 / 9 completed tasks 1 and 2 (same) 3 / 9 completed task 3 (versus 1 / 7) dominated by algorithmic detail 7 / 9 completed task 4 (versus 1 / 7) dominated by finding relevant code fragments Better quiz scores (t(16)=-2.04, p<.03, one-tailed) Same importance ratings (r=0.45, p=.02)

42. Future VIBE Directions Novel interaction and visualization techniques that scale from small to very large displays Continued evaluation and iteration of designs from a user-centered perspective Automatic task identification For more information: http://research.microsoft.com/research/vibe http://research.microsoft.com/research/vibe Novel interaction and visualization techniques that scale from small to very large displays Continued evaluation and iteration of designs from a user-centered perspective Automatic task identification For more information: http://research.microsoft.com/research/vibe http://research.microsoft.com/research/vibe

43. Thank You! http://research.microsoft.com /research/vibe http://research.microsoft.com /research/vibe Thank You! http://research.microsoft.com /research/vibe http://research.microsoft.com /research/vibe