1. Ahmed Sabbir Arif York University, Toronto, Canada a.s.arif@gmail.com
Text Entry Nomadicity Ambient Awareness, Handedness, and Error Adaptation
Ahmed Sabbir Arif
York University, Toronto, Canada

2. Character-Based Text Entry
One character at a time:
Non-ambiguous: Qwerty, ...
Ambiguous: Multi-tap, ...
We also have:
Word-based text entry: handwriting, ...
Phrase-based text entry: predictive, ...

3. Techniques: Timeline 1714 Mechanical Transcribing Machine
1830 Stenotype Machine
1870s
Qwerty
1880s Typewriters
1936
Dvorak
Personal Computers
1970s
Mobile Keypads
1990s
1993 handwriting Apple Newton

4. Typewriters & Computers
Qwerty
Dvorak
Qwerty vs. Dvorak:
Path dependence?

5. Chording Keyboards Chording keyboards: Twiddler, ...
Chorded keyers: Septambic Keyer, ...
Was never widely accepted
Stenotype machine was built by Karl Drais in 1830.

6. 12-Key Mobile Keypad Multi-tap: T9 Text Entry Other techniques:
Time-out, kill button
T9 Text Entry
Predictive
Other techniques:
TiltText, LetterWise, ...
Other keypads:
Less-Tap, Reduced Qwerty, ...

7. Reduced Sized Keyboards
Mini-Qwerty or thumb keyboards
Virtual or soft keyboards and keypads:
Usually soft versions of physical keyboards
A few use different methods: RollPad, ...

8. Projection Keyboards The concept immerged from IBM in 1995
Failed to get its anticipated attention
Business decision rather than usability issues

9. Touchscreens Touchscreen devices are in demand
Many replace physical keyboards
Difficult to input text with virtual keyboards:
No synthetic tactile feedback: vibration, ...
More error prone:
Error prevention techniques: character replacements, key-target resizing, ...

10. Nomadic Text Entry Non-stationary text entry Facts:
Walking, driving, or commuting
Facts:
Slower and more erroneous [Hillman][Lin][Mustonen]
Perfect task-parallelism is not possible [Meyer]
Involves a limited peripheral resource – our eyes
Creates competition for the attention between the device and the ambient environment
from gettyimages.ca

11. Nomadic Text Entry Techniques
Eyes-free:
Gesture – performs well only when guided by auditory feedback [Brewster][Lumsden]
Voice – error prone, heavyweight, performance drops when noisy, not realistic [Mankoff] [Brewster]
Other:
Chorded – takes time to master [Yatani]
Wearable – not convenient, erroneous [Chamberlain]
Synthetic tactile feedback improves touchscreen performance [Hoggan]

12. Nomadic Text Entry
Ambient Awareness

13. Our Approach Reduce the competition for focus: Real-time, because
Increase users’ awareness of ambient environment
By providing real-time feedback on their surroundings
Users already swap focus regularly between the text entry area and the keyboard [Arif]
Real-time, because
Users mostly occupied with instant spatial factors
Human navigation system is a dynamic, egocentric representation [Wang]

14. Four Feedback Techniques
Textual
Visual
Textual & visual
Textual & visual via translucent keyboard

15. Textual Feedback In textual or written form
Like turn-by-turn directional information by a GPS
We used the WOz method during the experiment
pre-set list containing messages, i.e. go straight, left turn ahead, etc.

16. (Textual &) Visual Feedback
Live video using the embedded camera
Textual feedback:
Translucent (alpha = 0.5) in textual & visual
Visual feedback area is not compromised
Background doesn’t obscure text
Users hold devices in 10–40° angles:
Shows the next few metres of the path
Allows short-term navigation
Highly beneficial

17. Textual & Visual via Translucent KB
A translucent virtual KB (alpha = 0.35) to show the visual feedback behind the keys
Less focus swap within the interface
Solid textual feedback area
Background doesn’t obscure text

18. Similar Techniques No empirical study
Video feedback obscures text input and the keys
Input background keeps changing
Causes confusion and irritation [Pilot study]
Road SMS Type n Walk Walk and Text

19. User Study Apple iPhone 4 Textual feedback simulated by the Wizard
Inputted the presented text phrases [Soukoreff]
Textual feedback simulated by the Wizard
Sent directly to the iPhone using a web app
Initial walking speed
Text entry
Stationary and nomadic

20. Obstacle Path Mimics realistic walking environments:
Forces users’ attention to the obstacles placed along the path
Similar to [Barnard]
Approx. 7.5×6 metres
One lap 24 metres
13 turns, 3 intersections

21. Design 12 participants + Initial text entry and walking performance
* 5 techniques
* 15 phrases
= 900 phrases
+ Initial text entry and walking performance
Record:
WPM, Total ER, Tfix automatic
Lap time, total laps, wrong turns, bumps manual
Wizard’s mistakes manual

22. Results: WPM, Total ER Significant effect
No Significant effect
Significant effect
Textual, textual & translucent significantly faster

23. Discussion: WPM, Total ER
Improved entry speed:
Textual 14%
Visual %
Textual & visual, 6%
Textual & translucent 11%
compared to the baseline
Textual & Textual & translucent significantly faster

24. Results: Walking Speed
159% more time to finish a lap while nomadic
No significant effect
Considered walking a secondary task

25. Results: Wrong Turns, Collisions
No Significant effect
Significant effect

26. User Feedback No significant effect Most felt “neutral”
Wanted to use in challenging scenarios i.e. busy street
Wanted to acquire the textual feedback system

27. Overall Performance
Textual and textual & visual via translucent keyboard had better overall performance
Improved entry speed by 14% and 11%
Reduced error rates by 13%
Textual – fastest walking speed (51.10 sec. per lap)
Collision count was high (8 in total)
Translucent – Low collision count (4 in total)
Highest lap time (57.11 sec. per lap)

28. Nomadic Text Entry
Handedness

29. Research Questions Do users input text while nomadic?
[YES] do they use
Both hands,
The dominant hand, or
The non-dominant hand?
While nomadic & only one hand is available?
[YES] do they use:

30. Survey Design Online – forums, e-mailing lists, ...
Voluntary sampling method – users self-selected
Screened for:
Adult – 18+
Owns a handheld device
Fluent in English
English is their primary mobile OS language

31. User Demographics 133 users after pre-screening
From 20 countries (4 continents)
46% female
71% touch-typists
Avg. usage – 4hrs/day
Avg. texts – 26/day
Handedness:
90% right-handed
7% left-handed
3% ambidextrous

32. Devices & Keyboards 89% owned a smartphone
All of them use physical or virtual Qwerty keyboard
11% use regular mobile devices
Use physical or virtual 12-key keypad

33. While Walking 48% input text [almost everyday]
Gender – no significant effect
Male 51%
Female 49%
Age – significant effect
18– %
26– %
36– % %

34. While Walking: Handedness
Mobile handedness
Handedness – no significant effect
Both %
Dominant 36.0%
Non-dominant 9.3%
Gender – no significant effect
Age – no significant effect

35. While Walking: Hand Availability
88.7% input text
Gender – no significant effect
Age – significant effect
18-25 years old younger users are more committed
Mobile handedness
Handedness – no significant effect
78.7% dominant & 21.3% non-dominant
Gender – significant effect
65% male & 92% female users prefer using dominant hand
Age – no significant effect

36. While Commuting 90% input text [significantly higher]
Gender – no significant effect
Age – no significant effect
Mobile handedness
Handedness – no significant effect
53.5% both, 46.5% dominant, & 0% non-dominant
Age – no significant effect

37. While Commuting: Hand Availability
85.8% input text [significantly higher]
Gender – no significant effect
Age – no significant effect
Mobile handedness
Handedness – no significant effect
85.9% dominant & 14.1% non-dominant
Gender – significant effect
77% male & 95% female users prefer using dominant hand

38. While Driving 75% drive: 58% input text [more than walking!]
Gender – no significant effect
Age – no significant effect
Mobile handedness
Handedness – no significant effect
91.7% dominant, & 8.3% non-dominant
Dominant hand use [significantly higher than walking/commuting]
Age – no significant effect

39. Key Findings A large number of users input text while nomadic
Commuting – significantly more
Walking & driving – similar
96% drivers texts while driving: while doing such is illegal!!
Age and gender do not influence the decision of texting + commuting or driving
Age influence the decision of texting + walking
No effect of handedness, age, or gender on mobile handedness
~50% use both & ~50% the dominant hand

40. Key Findings
Almost all users continue typing while while the other hand is occupied
[Commute] no effect of age or gender on this choice
[Walk] usually younger users
Most female users prefer using the dominant hand
Most drivers prefer using the dominant hand

41. Conclusion & Recommendations
Nomadic techniques must be properly investigated
In order to meet users’ need
Stationary handedness do not apply
Mobile handedness change based on:
Whether walking/commuting or driving
Hand availability
Gender
Must develop/explore driver-safe techniques

42. Adapting to a Faulty Text Entry Technique
Adaptation

43. Research Questions Do users adapt to a faulty system?
Do system errors influence adaptation process?
Is there a threshold for identification as error-prone?
How do users adapt to an error-prone system?

44. Selection of the 7 Letters
To guarantee uniformity all letters must appear the same number of times
Humans can remember 7 chunks ±2 in short-term memory tasks
Require relatively similar effort to draw with Graffiti and Unistrokes

45. Procedure
Input letters with pen on tablet » Primary method: Graffiti (large above) » Alternate method: Unistrokes (small above) » Suggested usage of alternate if unreliable
$1 Recognizer
Injected 10%, 30%, 50% errors
» Into three out of seven random Graffiti letters
» Different letters for each session

46. Results of the Pilot Studies
Pilot Study 1
I) No reliable switching behaviour for < 10% errors
Pilot Study 2
II) Error-prone letters not reliably identified
Instead: Global switch to alternate method
III) Extra care if error-prone letters were identified

47. Final Study Within-subject, 12 participants
Initial session × 1 block × 280 letters +
Final session × 3 blocks × 280 letters
= 1120 letters/user, total letters

48. The Usage of Alternate Method
Used Unistrokes instead of Graffiti - significant

49. Extra Care while Inputting
More time than average to draw a letter - significant

50. For Both Measures
Significant effect for extra care per letter with more time than average to draw a letter error rates
Significant learning effect

51. Summary Users do gradually adapt to a faulty system
Adaptation is proportional to error rate
Error rate has to be >10% to be perceived as error-prone
Users learn to avoid frequently occurring errors faster

52. Thank You!
Questions?