1. Precise Selection Techniques for Multi-Touch Screens Hrvoje Benko Andy D. Wilson Patrick Baudisch Columbia University and Microsoft Research CHI 2006

2. 2 Selecting a small target is very HARD!

3. CHI 20063 Small target size comparison Average finger ~ 15 mm wide Target UI element Width (abstract screen) Width 17” screen 1024x768 Width 30” screen 1024x768 Close button 18 pixels 6 mm (40% of finger) 10.8 mm (66% of finger) Resize handle 4 pixels 1.34 mm (9% of finger) 2.4 mm (16% of finger)

4. CHI 20064 Touchscreen Issues 1. Finger >>> Target 2. Finger occludes the target 3. Fingers/hands shake and jitter 4. Tracking can be noisy (e.g. video) 5. No hover state (hover == drag)

5. CHI 20065 Previous Work Solutions based on single touch interfaces and complex on-screen widgets: Albinsson, P. A. and Zhai, S. “High Precision Touch Screen Interaction.” (CHI ’03) Sears, A. and Shneiderman, B. “High Precision Touchscreens: Design Strategies and Comparisons with a Mouse.” (’91)

6. CHI 20066 Dual Finger Selections Multi-touch techniques Single fluid interaction no lifting/repositioning of fingers Design guidelines: Keep simple things simple. Provide an offset to the cursor when so desired. Enable user controlled control-display ratio.

7. CHI 20067 Simulating Hover State Extension of the “area==pressure” idea (MacKenzie and Oniszczak, CHI 1998) Problem: LARGE area difference  reliable clicking SMALL movement (i.e. SMALL area difference)  precise and accurate clicking

8. CHI 20068 SimPress (Simulated Pressure) Clicking gesture – “finger rocking” Goal: Maximize ∆ touch area Minimize ∆ cursor location

9. CHI 20069 Top Middle Cursor Large ∆ touch area Small ∆ cursor loc. Center-of-Mass Cursor Large ∆ touch area Large ∆ cursor loc. SimPress Cursor Placement

10. CHI 200610 SimPress in Action

11. CHI 200611 Dual Finger Selections 1. Offset 2. Midpoint 3. Stretch 4. X-Menu 5. Slider Primary finger  cursor position & click Secondary finger  cursor speed or C/D

12. CHI 200612 Dual Finger Offset Fixed offset WRT finger Ambidextrous control

13. CHI 200613 Dual Finger Midpoint Cursor  ½ distance between fingers Variable speed control Max speed reduction is 2x Dead spots on screen!

14. CHI 200614 Dual Finger Stretch Inspired by ZoomPointing (Albinsson & Zhai,‘03) Primary finger  anchor Secondary finger defines the zooming area scales the area in all directions away from the anchor

15. CHI 200615 Dual Finger Stretch Offset is preserved after selection!

16. CHI 200616 Zooming Comparison Bounding Box Zoom Fingers placed OFF target Target distance increases w/ zoom “Stretch” Zoom Primary finger placed ON target Same motion = 2x zoom

17. CHI 200617 Dual Finger X-Menu Crossing Menu (no buttons/no clicks) 4 speed modes 2 helper modes Cursor notification widget Eyes-free interaction Freezing cursor Quick offset setup Eliminate errors in noisy conditions Helpers: Snap – Remove offset Magnification Lens

18. CHI 200618 Dual Finger X-Menu

19. CHI 200619 Dual Finger X-Menu with Magnification Lens

20. CHI 200620 Dual Finger Slider Normal Slow 4X Slow 10X Freeze Snap

21. CHI 200621 Dual Finger Slider

22. CHI 200622 Multi-Touch Table Prototype Back projected diffuse screen IR vision-based tracking Similar to TouchLight (Wilson, ICMI’04)

23. CHI 200623 User Experiments Measure the impact of a particular technique on the reduction of error rate while clicking 2 parts: Evaluation of SimPress clicking Comparison of Four Dual Finger Techniques Task: Reciprocal target selection Varying the square target width Fixed distance (100 pixels) 12 paid participants (9 male,3 female, ages 20– 40), frequent computer users, various levels of touchscreen use

24. CHI 200624 Part 1: SimPress Evaluation Within subjects repeated measures design 5 target widths: 1,2,4,8,16 pxls Hypothesis: only 16 pxls targets are reliably selectable Results: 8 pixel targets still have ~10% error rate F (4,44) =62.598, p<0.001

25. CHI 200625 Part 2: Comparison of 4 Dual Finger Selection Techniques Compare: Offset, Stretch, X-Menu, Slider Varying noise conditions Inserted Gaussian noise: σ=0, 0.5, 2 Within subjects repeated measures design: 3 noise levels x 4 techniques x 4 target widths (1,2,4,8 pxls) 6 repetitions  288 trials per user Hypotheses: Techniques that control the C/D will reduce the impact of noise Slider should outperform X-Menu

26. CHI 200626 Part 2: Error Rate Analysis Interaction of Noise x Technique F (6,66) = 8.025, p<0.001

27. CHI 200627 Part 2: Error Rate Analysis Interaction of Width x Technique F (9,99) =29.473, p<0.001

28. CHI 200628 Part 2: Movement Time Analysis Analysis on median times Stretch is ~ 1s faster than Slider/X-Menu (t(11)=5.011, p<0.001) Slider similar performance to X- Menu Missing

29. CHI 200629 Subjective Evaluation Post-experiment questionnaire (5 pt Likert scale) Most mental effort: X-Menu (~2.88) Hardest to learn: X-Menu ( ~2.09) Most enjoyable: Stretch (~4.12), Slider (~4.08) No significant differences WRT fatigue

30. CHI 200630 Conclusions and Future Work Top performer & most preferred: Stretch Slider/X-Menu Comparable error rates to Stretch No distortion of user interface Cost: ~1s extra Freezing the cursor (positive feedback) Like “are you sure?” dialog for clicking… Possible future SimPress extensions: Detect user position/orientation Stabilization of the cursor

31. Questions

32. CHI 200632 Multi-Touch Tabletops MERL DiamondTouch (Dietz & Lehigh, ’01) SmartSkin (Rekimoto, ’02) PlayAnywhere and TouchLight (Wilson, ’04, ’05)

33. CHI 200633 ANOVA Table SourcedfFp Noise (N)(2,22)20.24<0.001 Technique (T)(3,33)169.14<0.001 Width (W)(3,33)150.40<0.001 N x T(6,66)8.03<0.001 T x W(9,99)29.47<0.001 N x W N x T x W