Precise Selection Techniques for Multi-Touch Screens Hrvoje Benko Andy D. Wilson Patrick Baudisch Columbia University and Microsoft Research CHI 2006

2 Selecting a small target is very HARD!

CHI Small target size comparison Average finger ~ 15 mm wide Target UI element Width (abstract screen) Width 17” screen 1024x768 Width 30” screen 1024x768 Close button 18 pixels 6 mm (40% of finger) 10.8 mm (66% of finger) Resize handle 4 pixels 1.34 mm (9% of finger) 2.4 mm (16% of finger)

CHI Touchscreen Issues 1. Finger >>> Target 2. Finger occludes the target 3. Fingers/hands shake and jitter 4. Tracking can be noisy (e.g. video) 5. No hover state (hover == drag)

CHI Previous Work Solutions based on single touch interfaces and complex on-screen widgets: Albinsson, P. A. and Zhai, S. “High Precision Touch Screen Interaction.” (CHI ’03) Sears, A. and Shneiderman, B. “High Precision Touchscreens: Design Strategies and Comparisons with a Mouse.” (’91)

CHI Dual Finger Selections Multi-touch techniques Single fluid interaction no lifting/repositioning of fingers Design guidelines: Keep simple things simple. Provide an offset to the cursor when so desired. Enable user controlled control-display ratio.

CHI Simulating Hover State Extension of the “area==pressure” idea (MacKenzie and Oniszczak, CHI 1998) Problem: LARGE area difference  reliable clicking SMALL movement (i.e. SMALL area difference)  precise and accurate clicking

CHI SimPress (Simulated Pressure) Clicking gesture – “finger rocking” Goal: Maximize ∆ touch area Minimize ∆ cursor location

CHI Top Middle Cursor Large ∆ touch area Small ∆ cursor loc. Center-of-Mass Cursor Large ∆ touch area Large ∆ cursor loc. SimPress Cursor Placement

CHI SimPress in Action

CHI Dual Finger Selections 1. Offset 2. Midpoint 3. Stretch 4. X-Menu 5. Slider Primary finger  cursor position & click Secondary finger  cursor speed or C/D

CHI Dual Finger Offset Fixed offset WRT finger Ambidextrous control

CHI Dual Finger Midpoint Cursor  ½ distance between fingers Variable speed control Max speed reduction is 2x Dead spots on screen!

CHI Dual Finger Stretch Inspired by ZoomPointing (Albinsson & Zhai,‘03) Primary finger  anchor Secondary finger defines the zooming area scales the area in all directions away from the anchor

CHI Dual Finger Stretch Offset is preserved after selection!

CHI Zooming Comparison Bounding Box Zoom Fingers placed OFF target Target distance increases w/ zoom “Stretch” Zoom Primary finger placed ON target Same motion = 2x zoom

CHI Dual Finger X-Menu Crossing Menu (no buttons/no clicks) 4 speed modes 2 helper modes Cursor notification widget Eyes-free interaction Freezing cursor Quick offset setup Eliminate errors in noisy conditions Helpers: Snap – Remove offset Magnification Lens

CHI Dual Finger X-Menu

CHI Dual Finger X-Menu with Magnification Lens

CHI Dual Finger Slider Normal Slow 4X Slow 10X Freeze Snap

CHI Dual Finger Slider

CHI Multi-Touch Table Prototype Back projected diffuse screen IR vision-based tracking Similar to TouchLight (Wilson, ICMI’04)

CHI User Experiments Measure the impact of a particular technique on the reduction of error rate while clicking 2 parts: Evaluation of SimPress clicking Comparison of Four Dual Finger Techniques Task: Reciprocal target selection Varying the square target width Fixed distance (100 pixels) 12 paid participants (9 male,3 female, ages 20– 40), frequent computer users, various levels of touchscreen use

CHI Part 1: SimPress Evaluation Within subjects repeated measures design 5 target widths: 1,2,4,8,16 pxls Hypothesis: only 16 pxls targets are reliably selectable Results: 8 pixel targets still have ~10% error rate F (4,44) =62.598, p<0.001

CHI Part 2: Comparison of 4 Dual Finger Selection Techniques Compare: Offset, Stretch, X-Menu, Slider Varying noise conditions Inserted Gaussian noise: σ=0, 0.5, 2 Within subjects repeated measures design: 3 noise levels x 4 techniques x 4 target widths (1,2,4,8 pxls) 6 repetitions  288 trials per user Hypotheses: Techniques that control the C/D will reduce the impact of noise Slider should outperform X-Menu

CHI Part 2: Error Rate Analysis Interaction of Noise x Technique F (6,66) = 8.025, p<0.001

CHI Part 2: Error Rate Analysis Interaction of Width x Technique F (9,99) =29.473, p<0.001

CHI Part 2: Movement Time Analysis Analysis on median times Stretch is ~ 1s faster than Slider/X-Menu (t(11)=5.011, p<0.001) Slider similar performance to X- Menu Missing

CHI Subjective Evaluation Post-experiment questionnaire (5 pt Likert scale) Most mental effort: X-Menu (~2.88) Hardest to learn: X-Menu ( ~2.09) Most enjoyable: Stretch (~4.12), Slider (~4.08) No significant differences WRT fatigue

CHI Conclusions and Future Work Top performer & most preferred: Stretch Slider/X-Menu Comparable error rates to Stretch No distortion of user interface Cost: ~1s extra Freezing the cursor (positive feedback) Like “are you sure?” dialog for clicking… Possible future SimPress extensions: Detect user position/orientation Stabilization of the cursor

Questions

CHI Multi-Touch Tabletops MERL DiamondTouch (Dietz & Lehigh, ’01) SmartSkin (Rekimoto, ’02) PlayAnywhere and TouchLight (Wilson, ’04, ’05)

CHI ANOVA Table SourcedfFp Noise (N)(2,22)20.24<0.001 Technique (T)(3,33)169.14<0.001 Width (W)(3,33)150.40<0.001 N x T(6,66)8.03<0.001 T x W(9,99)29.47<0.001 N x W N x T x W