1. making sense on small screens patrick baudisch microsoft research adaptive systems and interaction interaction focus

2. small screens store content off-screen halo store content in another layer multiblending squeezing it in fishnet make the unreadable readable summary thumbnails manually collapse irrelevant material collapse-to-zoom (making two small screens into one big stitching, mouse ether) collapse expand return

3. scenarios… commuting between Seattle & Redmond mobile user interface supports recall: bookmarked weather report site execute: navigation system the real work I do on my desktop PC  cell phone sync cable

4. but not for all… PCs: 300 million cell phones: 1.5 billion

5. …the “others” rural computing, such as India phone users, never had access to PC phone = first access to the digital world

6. problems 1. phone-only users need to plan and understand maps and routes explore the web interactively analyze and compare data (sensemaking) but almost all apps designed for phone+PC combo users 2. screen too small most content designed for desktop PC [rooms, card et al 86]

7. research agenda long term goal how much and what functionality can/should we offer phone only users? short term goal overcome limitations of small screen size

8. oh, and, does it matter? PCs market at 300 million PCs fairly saturated cell phones already 5 times bigger growing at high speed

9. related work skip

10. related work: zooming [Xie etc. al, www’04]: tap to zoom into a tile

11. related work: overview+detail [O’Hara et. at CHI 99]: readable text on hover

12. related work: multiple foci [Wobbrock et. al UIST’02]: Web thumb

13. related work: popouts popout prism [suh, et al., chi’02] enhanced thumbnails

14. related work: fisheyes document lens [robertson, uist’93] unifying presentation space [carpendale, uist’01] focus+context sketching on a Pocket PC [lank, chi’04] fishnetfishnet, collapse, s-thumb collapse s-thumb

15. related work: peep hole Ka-Ping Yee [CHI’03]

16. collapse expand add off screen squish context make readable collapse irrelevant add a layer halo

17. store content in periphery

18. the problem +

19. halo

20. cinematography 1.entry and exit points 2.point of view  arrow-based techniques 3.partially out of the frame  halo rings are familiar, graceful degradation

21. streetlamps aura visible from distance aura is round overlapping auras aggregate fading of aura indicates distance what we changed smooth transition  sharp edge disks  rings dark background  light background

22. title: user study user study

23. interfaces  arc/arrow fading off  scale 110-300m/cm  map as backdrop  readability ok  same selectable size  hypothesis: halo faster halo ring distance from display border legend

24. tasks

25. procedure 12 participants within subject design, counterbalanced four training maps per interface/task, then eight timed maps questionnaire

26. task completion time TaskArrow interfaceHalo interface Locate20.1 (7.3)16.8 (6.7) Closest9.9 (10.1)6.6 (5.3) Traverse20.6 (14.1)16.8 (8.7) Avoid9.2 (4.7)7.7 (5.8) 33% 16%

27. error rate TaskArrow interfaceHalo interface Locate23.5 pixels (21.6)28.4 pixels (33.8) Closest22% (42%)21% (41%) Traverse97.4 pixels (94.7)81.0 pixels (96.7) Avoid15% (35%)14% (34%)  participants underestimated distances by 26%  participants saw ovals (gestalt laws?)  we can compensate for that: width += 35%

28. subjective preference

29. results halo 16%-33% faster than arrows no split attention distortion-free space scale independent no need to annotate distance perceive all rings at once [treisman & gormican] limitation: max number or rings > back to large displays

30. teaser: multiblending application UI competes with content for screen space where to put it? > back to large displays

31. multiblending collapse expand add off screen squish context make readable collapse irrelevant add a layer conclude

32. multiblending: summary solution multiblending: glass palette distinguished from background photo & background more recognizable problem with traditional alpha blending: is this bush in palette or background?

33. goal of (alpha)-blending goal: get twice the benefit from limited screen space applications of alpha palettes everquest but not photoshop example overview: explain detail view photo: context for editing

34. limitations of alpha-blending limitations loss of contrast in both layers colors are not “true” anymore interference makes it unclear which layer a feature belongs to all features are affected reason each channel (rgb) = weighted sum of respective channel in source images

35. multiblending preserve the most relevant features  allow different weight for each features class avoid visual ambiguity  to preserve feature in one layer eliminate that feature in other layers  use perception-oriented feature classes (CIE Lab color + edges)

36. glass palette: steps 1-2 1. desaturate2. emboss…

37. glass palette: step 3 3. blur background… problems with noisy background

38. glass palette: step 4 4. remove unused areas unused decoration

39. step 5 & limitation 5. …re-map features collision: layers with same requirements

40. opaque alpha blending complete example multi blending

41. implementation recent graphics cards (with pixel shader 2.0) does computation on the fly 145 frames/sec

42. general procedure glass palettes is just one possible example other application scenarios may favor different palette styles general procedure: for each feature class decide which layer benefits more from it eliminate feature from other layer or map it to a different feature class

43. user studies compare multiblending with alpha blending at several levels of opacity background visibility palette visibility better technique

44. background visibility

46. palette visibility

48. results multiblending is a generalization of alpha blending provides both background and foreground visibility simultaneously (no single alpha value can do this) avoids interference allows preserving the most relevant features is widely applicable also for desktop: chameleon skin, games (Everquest)…

49. ok, back to large documents: web pages

51. fishnet collapse expand add off screen make readable collapse irrelevant add a layer conclude squish context

52. fishnet squish context

53. summary: highlighted search US Blackout PGA Microsoft found! don’t know highlights tell me “found” …but cannot tell me “not there”  fishnet not there! found! benefit: judge relevance of web page at a glance Microso

54. design

55. scrolling, context highlighting

56. space filling use parallel projection manhattan lenszoomscapescentral projectionparallel projection readability bottlenecks content disrupted tall objects won’t shrink two tasks…thus two resolutions reading: render as much as possible at full size skimming: preserve readability of context as well as possible homogenous context resolution preserves alignment space filling

57. context area: popouts ensure that search terms are readable preserves font attribute, such as underline shows anchor

58. extended highlights implementation: place color-full table cell behind popout prism: popouts interfere (need to turn off to read) fishnet: similar saliency with extended highlights

59. anchoring focus area context top context bottom context anchor points popout

60. user study

61. goal: when to use fisheye view; when not to use it  4 tasks 3 interfaces with comparable functionality linear: standard internet explorer overview: left 25% of screen = overview fisheye: top & bottom 25% = context size & aspect ration: same amount of screen space fit in professional web pages, 800 pixels wide  with overview/context: different aspect ratios 13 participants (7 males, 6 females) 

62. task 1: In this page outdated? babydisneyvideosale

63. task 2: product choice 512 MB DVD XP Home

64. task 3: co-occurrence of and MarinersMets

65. task 4: text analysis Clinton

66. … and hypotheses distinguishing columns immaterial … …necessary distinguishing rows immaterial favors overview-fisheyefavors fisheye … necessary favors overviewfavors linear view outdated task product comparison analysis co-occurrence task

67. 0 2 4 6 8 10 12 14 16 18 20 product choice co-occur- rence is this page out-dated? analysis task completion time (in sec) LinearOverviewFisheye results  confirmed hypotheses

68. subjective preference per interface + task consistent with task time exception: overview preferred for the co-occurrence task however overall 10 participants preferred overview, only 3 preferred fisheye, none preferred linear fisheye view less familiar, more cognitive effort? 0 2 4 6 8 10 12 14 16 18 20 Out-datedProduct choiceCo-occurrenceAnalysis Time (in sec) 0 1 2 3 4 5 6 7 8 Out-datedProduct choiceCo-occurrenceAnalysis LinearOverviewFisheye task completion time satisfaction

69. discussion: when to use what? fisheye views + search term highlighting results in useful interface there is no black and white wrt fisheye views use fisheye view if grouping in rows does not matter use overview if grouping in columns does not matter consider fisheye & overview on demand unlike overview, fisheye no page reflow

70. but wait: web pages are too wide for PDA

71. summary thumbnails collapse expand add off screen squish context make readable collapse irrelevant add a layer conclude

72. unreadable readable

73. any screen size any font size

77. demo

79. implementation scale font up to 23pt 2 crop text to repair 3 render page at small scale 4 mars mission mars m bitmap html load in web browser control for each mshtml element in document object model (DOM) count lines increase font size reduce text to preserve #lines client does scaling

80. text reduction current cut off at the end of paragraph OR remove common words future use something smarter (but what we have works surprisingly well) however goal is to show users where to zoom in may not be a summarizer

83. related work

85. user study design first set of “pilots” (20 students) go to history, pick fifth page, tell us URL and task second set of “pilots” (2 internal) pilots to generate balanced sets, remove 1 outlier page actual participants (12 externally recruited) the actual study

86. results ThumbnailSummary Thumbnail Single Column # of participants 0 2 4 6 8 10 task time preference

87. more results 0 10 20 30 TNSTSCDT Total vertical scroll 0 10 20 30 TNSTSCDT # vert scroll dir change a c 0 2 4 6 8 10 12 14 ThumbnailSummary Thumbnail # of zooming events # zooming scrolling

88. discussion summary thumbnails subsume traditional thumbnails simple conversion using a proxy server any platform  product

89. collapse-to-zoom collapse expand add off screen squish context make readable add a layer collapse irrelevant conclude

90. collapse- to-zoom narrow down manually collapse expand

91. short demo (there will be a second, slightly longer demo at the end of this talk)

92. ? limitation of thumbnails irrelevant page content costs valuable screen space when used with traditional thumbnails page content can become unreadable… related work

93. approaches device-specific authoring multiple-device authoring automatic re-authoring and client-side navigation thumbnail views collapse-to-zoom

94. collapse-to-zoom allow users to use their knowledge about relevant areas  zoom in (arbitrary rectangular regions).

95. collapse-to-zoom allow users to use their knowledge about relevant areas  zoom in …but also allow leveraging their knowledge about what is not relevant  collapse

96. collapse-to-zoom always show full page width use freed space to grow remaining (relevant) content

97. collapse-to-zoom provide visual context at all times: placeholders

98. collapse-to-zoom allow bookmarking collapsed state

99. marquee menu: there are 4 ways to select today: no distinction between the four ways of opening rectangular selection

100. marquee menu: direction selects 1 of 4 commands collapse-cell expand-cell collapse-column expand&zoom expand collapse an main com ds m

101. the name of the game photoshop: “ marquee selection” “marking menu ”: selecting commands with a pen stroke combine both  marquee menu

102. transfer to smartphone generic label cells with numeric codes and let users enter cell label [Paek et al. CSCW 2004] simple: reduce to 1D collapse column and expand column  webTV

103. implementation runs on desktop / tabletPC page-splitting based on DOM limitation: can’t start drag over link  user study

104. collapse expand add off screen squish context make readable collapse irrelevant add a layer next steps

106. integrating it all collapse-to-zoom for interaction summary thumbnails fits page to screen width ++ fishnet fits page to screen height

107. conclusions new visualizations techniques are the foundation for viewing large documents, such as web pages

108. next steps support users’ sensemaking activities on small screen devices how to pick the best insurance on a PDA? how to author a web page on a phone? how to do a tax return on a phone?

109. read more collapse expand add off screen squish context make readable collapse irrelevant add a layer CHI 2003 CHI 2004 UIST 2004 CHI 2005 AVI 2004 patrickbaudisch.com/projects

110. thanks! thanks to all the coauthors ruth rosenholtz carl gutwin bongshin lee heidi lam xing xie and VIBE

111. END

113. related work: fisheyes works well to reduce page length applying it to page width works not so well forces users to scroll for each line fishnet [Baudisch, AVI 2004] back to collapse-to-zoom or summary thumbnailscollapse-to-zoomsummary thumbnails

114. next visit of that page: page is already pre-collapsed which leads to fully readable detail view User uses expand-and-zoom gesture… User can follow links directly from within overview Content area expands more and is now readable …make page content grown. Now collapses “menu” column user collapses “archive” column to… collapse- to-zoom: walk through thumbnail view: unreadably small

115. next: search

116. 1. locate task click at expected location of off-screen targets  had to simulate on PC

117. 2. closest task click arrow/arc or off-screen location closest to car

118. 3. traverse task

119. 4. avoid task click on hospital farthest away from traffic jams

120. background visibility 50 % gp25 % 10 %

121. palette visibility

122. 50 % gp25 % 10 %

123. background visibility 50 % gp25 % 10 %

124. why even consider fisheye? web pages contain little spatial information limit distortion to what is absolutely necessary enhance periphery with something useful: search term popouts research question for what types of pages does highlighted search benefit from fisheye view?

125. different from [hornbæck & frokjær, chi’01] fishnet shows entire page integrated navigation search term highlighting add utility to context areas

126. independent variables… distinguishing columns immaterial … …necessary distinguishing rows immaterial task 1task 2 … necessarytask 3task 4