drag-and-pop a technique for accessing remote screen content on touch- and pen-operated systems patrick baudisch & ed cutrell, dan robbins, mary czerwinski, peter tandler, ben bederson, and alex zierlinger microsoft research visualization and interaction research

goals mixing touch screens/pen-input with large screens creates interesting new interface challenges bringing target icons to the user (“drag-and-pop”) allows users to complete drag interactions faster general theme: limiting interaction space to the display space at the user’s location can solve problems

contents goals scenario: touch/pen input meets large screens and multimon problems: dragging across bezels and long distances drag-and-pop related work design, design, and more design user study drag-and-pop makes users faster conclusions

touch/pen input breaks touch/pen-input + multimon touch and pen input renaissance PDAs Tablets Liveboards / Smartboards multi-display systems DynaWall, iRoom Smartboard wall connect tablet to external screen …

scenario 1: tables + screen tablet users scribble with pen… but filing icons into folder on external monitor requires mouse

scenario 2: dragging + bezels dragging across bezels in display wall is no problem for the mouse… …but a big problem when using pen/touch input

scenario 3: long distances dragging is designed for small screens… … but becomes time-consuming on large screens

drag-and-pop users starts dragging icon towards a distant folder or application icons of compatible type come towards mouse cursor user drops icon with minimal motion targets retract drag-and-pop works across bezels

demo… bringing target icons to the user allows users to complete drag interaction at the user’s current locations

scenario 1: tablet + screen filing icons into folder on external monitor

scenario 2: dragging + bezels dragging across bezels in display wall

scenario 3: long distances dragging on large screens

video

related work techniques for transferring information drag-and-drop avoids hidden clipboard (e.g. Xerox Star) hyper­dragging (Rekimoto, 1999) pick-and-drop (Rekimoto, 1997)+ take-and-put (Streitz et al., 2001) overcome large distances magic pointing (Zhai et al., 1999) requires an indirect input device gesture input techniques (Rubine, 1991) throwing (Geißler, 1998) for reliable target acquisition? laser pointers to acquire targets on a Smartboard (Myers et al. 2002) mouse-based interaction techniques lodestones and lay lines (Jul, 2002) flick (Dulberg et al., 1999) sticky icons (Swaminathan and Sato 1997)

design

selecting candidates initialize all icons are candidates filter eliminate icons with non-matching file types eliminate icons that are too close eliminate icons outside target angle if necessary, restrict to some hard limit

preserving layout snap to grid eliminate empty rows and columns translate back place center of bounding box in front of user closer for experts

the rubber band animation did not work “frozen” motion blur narrow midriff suggests elasticity clue for distance simplified version

getting it out of the way to rearrange icons on the desktop (overloaded): any mouse motion moving away from the “popped-up” icons de-activates drag-and-pop introduce flick gesture into mouse motion

study

pre-study 3 layouts for study: sparse (11), frame (28), cluttered (35) 15 single, 6 dual, and 4 triple monitor users overall resolutions 800,000 pixels to 3,900,000 pixels (= 66% more than the display wall used in the experiment).

user study participants: 2 female, 5 male dynaWall 3 Smartboard 15’ long (4.5m) 3 x 1024x768 pixels native code not stable enough à Macromedia Flash version task: drag icons into matching folder highlighting disappeared when started each desktop: icons + 10 icons to be filed

results faster with drag-and-pop error rate higher with drag-and-pop most of the effect caused by the bezels Control Drag-and-pop 3.7 times speedup Control Drag-and-pop

subjective satisfaction > 6 (out of 7) “I liked using drag-and-pop” “I always understood what was happening when drag-and- pop was on”, “I would use drag-and-pop for large displays.” < 3 for “It took a long time to get used to drag-and-pop” “It was hard to control what the targets did when drag-and- pop was on.” drag-and-pop interface causes less manual stress and fatigue than the control interface

lesson learned the shortest connection between two points on a display wall is not a straight line (we fixed this by opening target sector towards top of display)

the larger picture

general theme WIMP metaphor can break on large screens with pen/touch input drag-and-pop generalizes direct manipulation bring content to the user let the user interact with it send content back  interaction space is not the same as display space anymore

drag-and-pick problem launch app or open file drag-and-pick user drags “background” all icons in that direction move to the cursor user drags % releases mouse over it target is activated

what about dialog boxes?

inside applications… drag-and-pop works even if target is occluded clipped closed (folder) use the concept to file s?

goals revisited - conclusions mixing touch screens/pen-input with large screens creates interesting new interface challenges bringing target icons to the user (“drag-and-pop”) allows users to complete drag interactions faster general theme: limiting interaction space to the display space at the user’s location can solve problems

thank you! try it out: Google drag-and-pop thanks to: the VIBE research group (mary czerwinski, george robertson…) diane kelly, dieter boecker, lance good, amanda williams,