making sense on small screens patrick baudisch microsoft research adaptive systems and interaction interaction focus

small screens store content off-screen halo store content in another layer multiblending squeezing it in fishnet make the unreadable readable summary thumbnails manually collapse irrelevant material collapse-to-zoom (making two small screens into one big stitching, mouse ether) collapse expand return

scenarios… commuting between Seattle & Redmond mobile user interface supports recall: bookmarked weather report site execute: navigation system the real work I do on my desktop PC  cell phone sync cable

but not for all… PCs: 300 million cell phones: 1.5 billion

…the “others” rural computing, such as India phone users, never had access to PC phone = first access to the digital world

problems 1. phone-only users need to plan and understand maps and routes explore the web interactively analyze and compare data (sensemaking) but almost all apps designed for phone+PC combo users 2. screen too small most content designed for desktop PC [rooms, card et al 86]

research agenda long term goal how much and what functionality can/should we offer phone only users? short term goal overcome limitations of small screen size

oh, and, does it matter? PCs market at 300 million PCs fairly saturated cell phones already 5 times bigger growing at high speed

related work skip

related work: zooming [Xie etc. al, www’04]: tap to zoom into a tile

related work: overview+detail [O’Hara et. at CHI 99]: readable text on hover

related work: multiple foci [Wobbrock et. al UIST’02]: Web thumb

related work: popouts popout prism [suh, et al., chi’02] enhanced thumbnails

related work: fisheyes document lens [robertson, uist’93] unifying presentation space [carpendale, uist’01] focus+context sketching on a Pocket PC [lank, chi’04] fishnetfishnet, collapse, s-thumb collapse s-thumb

related work: peep hole Ka-Ping Yee [CHI’03]

collapse expand add off screen squish context make readable collapse irrelevant add a layer halo

store content in periphery

the problem +

halo

cinematography 1.entry and exit points 2.point of view  arrow-based techniques 3.partially out of the frame  halo rings are familiar, graceful degradation

streetlamps aura visible from distance aura is round overlapping auras aggregate fading of aura indicates distance what we changed smooth transition  sharp edge disks  rings dark background  light background

title: user study user study

interfaces  arc/arrow fading off  scale m/cm  map as backdrop  readability ok  same selectable size  hypothesis: halo faster halo ring distance from display border legend

tasks

procedure 12 participants within subject design, counterbalanced four training maps per interface/task, then eight timed maps questionnaire

task completion time TaskArrow interfaceHalo interface Locate20.1 (7.3)16.8 (6.7) Closest9.9 (10.1)6.6 (5.3) Traverse20.6 (14.1)16.8 (8.7) Avoid9.2 (4.7)7.7 (5.8) 33% 16%

error rate TaskArrow interfaceHalo interface Locate23.5 pixels (21.6)28.4 pixels (33.8) Closest22% (42%)21% (41%) Traverse97.4 pixels (94.7)81.0 pixels (96.7) Avoid15% (35%)14% (34%)  participants underestimated distances by 26%  participants saw ovals (gestalt laws?)  we can compensate for that: width += 35%

subjective preference

results halo 16%-33% faster than arrows no split attention distortion-free space scale independent no need to annotate distance perceive all rings at once [treisman & gormican] limitation: max number or rings > back to large displays

teaser: multiblending application UI competes with content for screen space where to put it? > back to large displays

multiblending collapse expand add off screen squish context make readable collapse irrelevant add a layer conclude

multiblending: summary solution multiblending: glass palette distinguished from background photo & background more recognizable problem with traditional alpha blending: is this bush in palette or background?

goal of (alpha)-blending goal: get twice the benefit from limited screen space applications of alpha palettes everquest but not photoshop example overview: explain detail view photo: context for editing

limitations of alpha-blending limitations loss of contrast in both layers colors are not “true” anymore interference makes it unclear which layer a feature belongs to all features are affected reason each channel (rgb) = weighted sum of respective channel in source images

multiblending preserve the most relevant features  allow different weight for each features class avoid visual ambiguity  to preserve feature in one layer eliminate that feature in other layers  use perception-oriented feature classes (CIE Lab color + edges)

glass palette: steps desaturate2. emboss…

glass palette: step 3 3. blur background… problems with noisy background

glass palette: step 4 4. remove unused areas unused decoration

step 5 & limitation 5. …re-map features collision: layers with same requirements

opaque alpha blending complete example multi blending

implementation recent graphics cards (with pixel shader 2.0) does computation on the fly 145 frames/sec

general procedure glass palettes is just one possible example other application scenarios may favor different palette styles general procedure: for each feature class decide which layer benefits more from it eliminate feature from other layer or map it to a different feature class

user studies compare multiblending with alpha blending at several levels of opacity background visibility palette visibility better technique

background visibility

palette visibility

results multiblending is a generalization of alpha blending provides both background and foreground visibility simultaneously (no single alpha value can do this) avoids interference allows preserving the most relevant features is widely applicable also for desktop: chameleon skin, games (Everquest)…

ok, back to large documents: web pages

fishnet collapse expand add off screen make readable collapse irrelevant add a layer conclude squish context

fishnet squish context

summary: highlighted search US Blackout PGA Microsoft found! don’t know highlights tell me “found” …but cannot tell me “not there”  fishnet not there! found! benefit: judge relevance of web page at a glance Microso

design

scrolling, context highlighting

space filling use parallel projection manhattan lenszoomscapescentral projectionparallel projection readability bottlenecks content disrupted tall objects won’t shrink two tasks…thus two resolutions reading: render as much as possible at full size skimming: preserve readability of context as well as possible homogenous context resolution preserves alignment space filling

context area: popouts ensure that search terms are readable preserves font attribute, such as underline shows anchor

extended highlights implementation: place color-full table cell behind popout prism: popouts interfere (need to turn off to read) fishnet: similar saliency with extended highlights

anchoring focus area context top context bottom context anchor points popout

user study

goal: when to use fisheye view; when not to use it  4 tasks 3 interfaces with comparable functionality linear: standard internet explorer overview: left 25% of screen = overview fisheye: top & bottom 25% = context size & aspect ration: same amount of screen space fit in professional web pages, 800 pixels wide  with overview/context: different aspect ratios 13 participants (7 males, 6 females) 

task 1: In this page outdated? babydisneyvideosale

task 2: product choice 512 MB DVD XP Home

task 3: co-occurrence of and MarinersMets

task 4: text analysis Clinton

… and hypotheses distinguishing columns immaterial … …necessary distinguishing rows immaterial favors overview-fisheyefavors fisheye … necessary favors overviewfavors linear view outdated task product comparison analysis co-occurrence task

product choice co-occur- rence is this page out-dated? analysis task completion time (in sec) LinearOverviewFisheye results  confirmed hypotheses

subjective preference per interface + task consistent with task time exception: overview preferred for the co-occurrence task however overall 10 participants preferred overview, only 3 preferred fisheye, none preferred linear fisheye view less familiar, more cognitive effort? Out-datedProduct choiceCo-occurrenceAnalysis Time (in sec) Out-datedProduct choiceCo-occurrenceAnalysis LinearOverviewFisheye task completion time satisfaction

discussion: when to use what? fisheye views + search term highlighting results in useful interface there is no black and white wrt fisheye views use fisheye view if grouping in rows does not matter use overview if grouping in columns does not matter consider fisheye & overview on demand unlike overview, fisheye no page reflow

but wait: web pages are too wide for PDA

summary thumbnails collapse expand add off screen squish context make readable collapse irrelevant add a layer conclude

unreadable readable

any screen size any font size

demo

implementation scale font up to 23pt 2 crop text to repair 3 render page at small scale 4 mars mission mars m bitmap html load in web browser control for each mshtml element in document object model (DOM) count lines increase font size reduce text to preserve #lines client does scaling

text reduction current cut off at the end of paragraph OR remove common words future use something smarter (but what we have works surprisingly well) however goal is to show users where to zoom in may not be a summarizer

related work

user study design first set of “pilots” (20 students) go to history, pick fifth page, tell us URL and task second set of “pilots” (2 internal) pilots to generate balanced sets, remove 1 outlier page actual participants (12 externally recruited) the actual study

results ThumbnailSummary Thumbnail Single Column # of participants task time preference

more results TNSTSCDT Total vertical scroll TNSTSCDT # vert scroll dir change a c ThumbnailSummary Thumbnail # of zooming events # zooming scrolling

discussion summary thumbnails subsume traditional thumbnails simple conversion using a proxy server any platform  product

collapse-to-zoom collapse expand add off screen squish context make readable add a layer collapse irrelevant conclude

collapse- to-zoom narrow down manually collapse expand

short demo (there will be a second, slightly longer demo at the end of this talk)

? limitation of thumbnails irrelevant page content costs valuable screen space when used with traditional thumbnails page content can become unreadable… related work

approaches device-specific authoring multiple-device authoring automatic re-authoring and client-side navigation thumbnail views collapse-to-zoom

collapse-to-zoom allow users to use their knowledge about relevant areas  zoom in (arbitrary rectangular regions).

collapse-to-zoom allow users to use their knowledge about relevant areas  zoom in …but also allow leveraging their knowledge about what is not relevant  collapse

collapse-to-zoom always show full page width use freed space to grow remaining (relevant) content

collapse-to-zoom provide visual context at all times: placeholders

collapse-to-zoom allow bookmarking collapsed state

marquee menu: there are 4 ways to select today: no distinction between the four ways of opening rectangular selection

marquee menu: direction selects 1 of 4 commands collapse-cell expand-cell collapse-column expand&zoom expand collapse an main com ds m

the name of the game photoshop: “ marquee selection” “marking menu ”: selecting commands with a pen stroke combine both  marquee menu

transfer to smartphone generic label cells with numeric codes and let users enter cell label [Paek et al. CSCW 2004] simple: reduce to 1D collapse column and expand column  webTV

implementation runs on desktop / tabletPC page-splitting based on DOM limitation: can’t start drag over link  user study

collapse expand add off screen squish context make readable collapse irrelevant add a layer next steps

integrating it all collapse-to-zoom for interaction summary thumbnails fits page to screen width ++ fishnet fits page to screen height

conclusions new visualizations techniques are the foundation for viewing large documents, such as web pages

next steps support users’ sensemaking activities on small screen devices how to pick the best insurance on a PDA? how to author a web page on a phone? how to do a tax return on a phone?

read more collapse expand add off screen squish context make readable collapse irrelevant add a layer CHI 2003 CHI 2004 UIST 2004 CHI 2005 AVI 2004 patrickbaudisch.com/projects

thanks! thanks to all the coauthors ruth rosenholtz carl gutwin bongshin lee heidi lam xing xie and VIBE

END

related work: fisheyes works well to reduce page length applying it to page width works not so well forces users to scroll for each line fishnet [Baudisch, AVI 2004] back to collapse-to-zoom or summary thumbnailscollapse-to-zoomsummary thumbnails

next visit of that page: page is already pre-collapsed which leads to fully readable detail view User uses expand-and-zoom gesture… User can follow links directly from within overview Content area expands more and is now readable …make page content grown. Now collapses “menu” column user collapses “archive” column to… collapse- to-zoom: walk through thumbnail view: unreadably small

next: search

1. locate task click at expected location of off-screen targets  had to simulate on PC

2. closest task click arrow/arc or off-screen location closest to car

3. traverse task

4. avoid task click on hospital farthest away from traffic jams

background visibility 50 % gp25 % 10 %

palette visibility

50 % gp25 % 10 %

background visibility 50 % gp25 % 10 %

why even consider fisheye? web pages contain little spatial information limit distortion to what is absolutely necessary enhance periphery with something useful: search term popouts research question for what types of pages does highlighted search benefit from fisheye view?

different from [hornbæck & frokjær, chi’01] fishnet shows entire page integrated navigation search term highlighting add utility to context areas

independent variables… distinguishing columns immaterial … …necessary distinguishing rows immaterial task 1task 2 … necessarytask 3task 4