1. Shift: A Technique for Operating Pen-Based Interfaces Using Touch Daniel Vogel University of Toronto Patrick Baudisch Microsoft Research

2. Motivation

5. Small Targets

7. Advantages of the Pen PenFinger unique contact point remove hand from screen ambiguous contact point finger occludes target

8. Possible Solutions …

9. Offset Cursor (Potter et al. 1988) PenOffset Cursor

10. Offset Cursor (Potter et al. 1988)

11. Disadvantages 1.no visual feedback until contact, need to estimate offset 2.makes some display areas inaccessible 3.unexpected offset affects walk-up-and-use scenarios Offset Cursor (Potter et al. 1988)

12. Shift

13. Benefit 1: Aim for the Target Users expect to click on the target itself.  allows switching between pen and touch  walk-up and use with kiosk

14. Users expect to click on the target itself.  allows switching between pen and touch  walk-up and use with kiosk Benefit 1: Aim for the Target

15. Benefit 2: All Areas Accessible Callout is relative to finger, so it can go anywhere.  no edge problems

16. Callout is relative to finger, so it can go anywhere.  no edge problems Benefit 2: All Areas Accessible

17. Callout only used when necessary  same speed as unaided touch screen for large targets Benefit 3: Fast For Large Targets

18. Design Iterations

19. Performance Model Model

20. First Prototype

21. Revision and Visuals

22. Based on selection ambiguity with fallback to hesitation. S T = Target Size, S F = Finger occlusion threshold  S T << S F  high selection ambiguity  no delay  S T >> S F  no selection ambiguity  long delay  S T ≈ S F  “ambiguous selection ambiguity”  short delay Escalation

23. Based on selection ambiguity with fallback to hesitation. S T = Target Size, S F = Finger occlusion threshold  S T << S F  high selection ambiguity  no delay  S T >> S F  no selection ambiguity  long delay  S T ≈ S F  “ambiguous selection ambiguity”  short delay

24. user’s view hardware’s view Perceived Input Point Correction Users expect selection point to be higher. Iterative estimate for a correction vector V using difference between initial contact point P 1 and final lift off point P 2  V t+1 = V t + w(P 2 - P 1 )

25. Experiment

27. Experimental Design 3 techniques (Shift, Touch, Offset Cursor) x 2 finger styles (nail, tip) x 3 blocks x 6 target sizes (6, 12, 18, 24, 48, 96px) x 4 target directions (NW, NE, SW, SE)

28. Error

29. Time

31. Corrective Movements

33. Discussion Able to select small targets reliably (like Offset Cursor) Fast for large targets (like unaided Touch Screen) However, biggest benefit may be simpler mental model:  “Just aim for the target”

34. High Accuracy Enhancements Added Zooming and CD-Ratio Manipulation

35. High Accuracy Enhancements

36. Thanks to members of the ASI and VIBE groups at MSR, special thanks to Raman Sarin, Ed Cutrell, and David Thiel.

37. Appendix

38. Estimating Occlusion Threshold Don’t know actual finger size, so estimate it over time  when S T ≈ S F  short delay … means user can choose to use escalation by hesitating or not  if they hesitate and use escalation  make S F larger  if they just click without escalation  make S F smaller

39. Prototypes