1. Making Touchscreen Keyboards Adaptive to Keys, Hand Postures, and Individuals – A Hierarchical Spatial Backoff Model Approach Ying Yin 1,2, Tom Ouyang 1, Kurt Partridge 1, and Shumin Zhai 1 1 Google Logo here 2 MIT Logo here

2. Foundations to current methods Language modeling – vocabulary – 1-gram, 2-gram … N gram frequencies Spatial models – converting input touch points into probabilities of letters Edit distance correction – assigning cost to insertion, deletion, and other spelling errors User and posture independent

3. Research questions One promising area for improvement is by making them adapt to the user – What types of adaption are possible? – How do they affect performance?

4. Contributions A novel hierarchical adaptive model Show benefits of posture and user adaptation Online posture classification method 13.2% reduction in character error rate – compared to base model – without language model

5. Types of adaptation Individual differences (cf. Findlater & Wobbrock, 2012) Furthermore, people use different hand postures to type (cf. Azenkot and Zhai, 2012)

6. Different typing postures: two thumbs, one finger, or one thumb

7. Types of adaptation Different postures  different touch patterns Touch patterns also depend on letter keys (Azenkot & Zhai, 2012) Need adaptation

8. Challenges of adaptation Complexity – three adaptive factors: key, posture, individual – large number of submodels – need sufficient data to build each submodel Model selection – wrong selection may hurt keyboard quality – uncertainty in posture classification

9. Hierarchical spatial backoff model (SBM)

10. Combinatorial and fine grained adaptation Conservative Does not require an extra training phase Updates the model continuously online

11. Research method “Pepper” dataset (Azenkot & Zhai, 2012) – 30 right-handed participants – given random phrases to type – between-subject: each person uses one posture – 84,292 touch points in total 10-fold cross validation

12. Comparison of spatial models

13. Two-thumb One-finger Effective key areas

14. Posture classification SVM-based classifier Based on correlation between time and distance between consecutive touch points – no additional sensors required – speed independent 86.4% accuracy Real-time

15. Posture adaptation

16. Individual adaptation

17. Prototype implementation of SBM 13.2% reduction in character error rate – compared to base model – without language model Integrated with real keyboard – combined with language model – runs on Android phone in real-time

18. Future work Weighted average of submodels instead of making binary decisions More data: real-use logging and game playing User studies – validate the accuracy and speed improvement – how users adapt their behavior to SBM Combine spatial and language models

19. Contributions A novel hierarchical adaptive model Show benefits of posture and user adaptation Online posture classification method Opens up many more interesting HCI questions

20. Q & A

22. Prototype implementation of SBM Posture & key adaptation models – supervised and batch learning Individual adaptation models – unsupervised and online learning Backs-off to more basic models when – posture estimation is uncertain (conf. < 0.94) – there is insufficient user data (< 50 data points)