1. drag-and-pop a technique for accessing remote screen content on touch- and pen-operated systems patrick baudisch & ed cutrell, dan robbins, mary czerwinski, peter tandler, ben bederson, and alex zierlinger microsoft research visualization and interaction research

2. goals mixing touch screens/pen-input with large screens creates interesting new interface challenges bringing target icons to the user (“drag-and-pop”) allows users to complete drag interactions faster general theme: limiting interaction space to the display space at the user’s location can solve problems

3. contents goals scenario: touch/pen input meets large screens and multimon problems: dragging across bezels and long distances drag-and-pop related work design, design, and more design user study drag-and-pop makes users faster conclusions

4. touch/pen input breaks touch/pen-input + multimon touch and pen input renaissance PDAs Tablets Liveboards / Smartboards multi-display systems DynaWall, iRoom Smartboard wall connect tablet to external screen …

5. scenario 1: tables + screen tablet users scribble with pen… but filing icons into folder on external monitor requires mouse

6. scenario 2: dragging + bezels dragging across bezels in display wall is no problem for the mouse… …but a big problem when using pen/touch input

7. scenario 3: long distances dragging is designed for small screens… … but becomes time-consuming on large screens

8. drag-and-pop users starts dragging icon towards a distant folder or application icons of compatible type come towards mouse cursor user drops icon with minimal motion targets retract drag-and-pop works across bezels

9. demo… bringing target icons to the user allows users to complete drag interaction at the user’s current locations

10. scenario 1: tablet + screen filing icons into folder on external monitor

11. scenario 2: dragging + bezels dragging across bezels in display wall

12. scenario 3: long distances dragging on large screens

13. video

14. related work techniques for transferring information drag-and-drop avoids hidden clipboard (e.g. Xerox Star) hyper­dragging (Rekimoto, 1999) pick-and-drop (Rekimoto, 1997)+ take-and-put (Streitz et al., 2001) overcome large distances magic pointing (Zhai et al., 1999) requires an indirect input device gesture input techniques (Rubine, 1991) throwing (Geißler, 1998) for reliable target acquisition? laser pointers to acquire targets on a Smartboard (Myers et al. 2002) mouse-based interaction techniques lodestones and lay lines (Jul, 2002) flick (Dulberg et al., 1999) sticky icons (Swaminathan and Sato 1997)

15. design

16. selecting candidates initialize all icons are candidates filter eliminate icons with non-matching file types eliminate icons that are too close eliminate icons outside target angle if necessary, restrict to some hard limit

17. preserving layout snap to grid eliminate empty rows and columns translate back place center of bounding box in front of user closer for experts

18. the rubber band animation did not work “frozen” motion blur narrow midriff suggests elasticity clue for distance simplified version

19. getting it out of the way to rearrange icons on the desktop (overloaded): any mouse motion moving away from the “popped-up” icons de-activates drag-and-pop introduce flick gesture into mouse motion

20. study

21. pre-study 3 layouts for study: sparse (11), frame (28), cluttered (35) 15 single, 6 dual, and 4 triple monitor users overall resolutions 800,000 pixels to 3,900,000 pixels (= 66% more than the display wall used in the experiment).

22. user study participants: 2 female, 5 male dynaWall 3 Smartboard 15’ long (4.5m) 3 x 1024x768 pixels native code not stable enough à Macromedia Flash version task: drag icons into matching folder highlighting disappeared when started each desktop: 11-35 icons + 10 icons to be filed

23. results faster with drag-and-pop error rate higher with drag-and-pop most of the effect caused by the bezels Control Drag-and-pop 3.7 times speedup Control Drag-and-pop

24. subjective satisfaction > 6 (out of 7) “I liked using drag-and-pop” “I always understood what was happening when drag-and- pop was on”, “I would use drag-and-pop for large displays.” < 3 for “It took a long time to get used to drag-and-pop” “It was hard to control what the targets did when drag-and- pop was on.” drag-and-pop interface causes less manual stress and fatigue than the control interface

25. lesson learned the shortest connection between two points on a display wall is not a straight line (we fixed this by opening target sector towards top of display)

26. the larger picture

27. general theme WIMP metaphor can break on large screens with pen/touch input drag-and-pop generalizes direct manipulation bring content to the user let the user interact with it send content back  interaction space is not the same as display space anymore

28. drag-and-pick problem launch app or open file drag-and-pick user drags “background” all icons in that direction move to the cursor user drags % releases mouse over it target is activated

29. what about dialog boxes?

30. inside applications… drag-and-pop works even if target is occluded clipped closed (folder) use the concept to file emails?

31. goals revisited - conclusions mixing touch screens/pen-input with large screens creates interesting new interface challenges bringing target icons to the user (“drag-and-pop”) allows users to complete drag interactions faster general theme: limiting interaction space to the display space at the user’s location can solve problems

32. thank you! try it out: Google drag-and-pop thanks to: the VIBE research group (mary czerwinski, george robertson…) diane kelly, dieter boecker, lance good, amanda williams,