Text Input: Techniques and Research Tools Poika Isokoski at NIT Background: A Collage of images scanned from: Albertine Gaur. A history of writing. The British Library, London, UK, 2 edition, Tampere University Computer Human Interaction Group

Text Input: Techniques and Research Tools Contents Introduction Historical Notes Text Input Methods Keyboards Text Recognition Pointing Temporal Measuring Performance More Info 1

Text Input: Techniques and Research Tools Introduction For some time in the past text input was not a very interesting research topic Desktop keyboard is so good that it cannot be easily beaten Additional Inferior text input methods have not been needed Mobile computing has changed the situation Keyboards are difficult to fit in a mobile phone or a PDA. Handwriting recognition is difficult and writing is slow Speech recognition is even more difficult => There is no obvious solution. Is this there a real and lasting need for a new writing system(s)? 2

Text Input: Techniques and Research Tools Historical Notes Interplay of culture and writing Culture chooses a writing system that best suits it –need to communicate –need for information storage –available technology and materials –needs of the individuals in power Good inventions are meaningless if there is no need for them 3

Text Input: Techniques and Research Tools Historical Notes 4

Text Input: Techniques and Research Tools Historical Notes Separation of interfaces For a long time there was only one interface Written on paper (or whatever material the culture preferred) Stored on paper Read from paper Pen movement was connected to the form of the written character which again was tightly coupled with the form of the character to be read. 5

Text Input: Techniques and Research Tools Historical Notes After Gutenberg Written in printing press (or with a typewriter) Stored on paper Read from paper Writing motion is not necessarily the same as the form of the characters 6

Text Input: Techniques and Research Tools Historical Notes Today Written with text input methods Stored as bits (magnetic fields, dots on glass, etc.) Read from text output system. None of the tasks are mechanically connected. There is software in between. Having separate system for each phase: Gives more freedom for optimization of each task Requires more skills from the user of texts 7

Text Input: Techniques and Research Tools Historical Notes 8

Text Input: Techniques and Research Tools Text Input Methods 9

Text Input: Techniques and Research Tools Keyboards Context free QWERTY 12-key multi-press Chord – GKOS as an example ( ) Contextual Instant Text Microsoft Excel 12

Text Input: Techniques and Research Tools Keyboards How to measure performance Speed and error rate (more on these later) In desktop use: physical stress (dvorak-QWERTY-debate) How to model/predict performance No good solution for multi-finger operation For one-finger typing use same stuff as with soft-keyboards (discussed later) With multi-press/contextual methods consider the number of key presses, finger travel, and the need for visual feedback. 13

Text Input: Techniques and Research Tools Text Recognition Machine readable Bar-codes Human Readable OCR On-line handwriting recognition Off-line recognition with information on writing dynamics images from:

Text Input: Techniques and Research Tools Text Recognition (2) Unistrokes Explicit segmentation by lifting the pen Character level: Unistrokes, Graffiti Word level: octave ( (not available since 2002 )) 15

Text Input: Techniques and Research Tools Text Recognition How to measure performance Speed Human error rate Recognition error rate Need for training (user or algorithms) How to model (handwriting) Models are complicated Steering law Models for post-mortem analysis/synthesis (non-predictive) Model for unistroke writing (simple, but not very accurate) All these models require some empirical data on the task, therefore they cannot be used in pure prediction. 16

Text Input: Techniques and Research Tools Pointing Continuous gesturing (session level unistrokes) Dasher (web demo) ( ) Quikwriting (web demo) ( ) 17

Text Input: Techniques and Research Tools Pointing Direct Soft-keyboards: qwerty, fitaly, OPTI ( ) Menu hybrids MessageEase ( ) Indirect FOCL ( ) QFUMCKZ OHT WXAERSB IND F1YLGVPJ space QFU M C KZ O H T W X A E R SBI N D YL GV P J 18

Text Input: Techniques and Research Tools Pointing How to measure performance Speed and error rate How to model Direct: Fitts’ law + statistics on the text to be written Indirect: number of keypresses (independent KSPC). 19

Text Input: Techniques and Research Tools Temporal input Morse code How to measure performance Speed and accuracy How to model KSPC 20

Text Input: Techniques and Research Tools Fitts’ Law Fitts’ law TTime for pointing task a,bdetermined empirically Adistance to target Wwidth of the target More on Fitts’ law at: A W 21

Text Input: Techniques and Research Tools Fitts’ Law Steering Law T C Time for steering task C a,bempirically determined constants W(s)width of the steering tunnel at point s strajectory being modeled Straight tube: Circle: More on Steering law: Johnny Accott and Shumin Zhai, Performance evaluation of Input Devices in Trajectory-based Tasks: An Application of The Steering Law, Proceedings of CHI’99, ACM. W A R W 22

Text Input: Techniques and Research Tools Measuring Performance Measuring speed What speed? –Walk-up or expert or something in between? –Error free or errors included and corrections included? –Pure writing or in task context? –The users, are they young, old, blind, one-handed? –The list is endless. Measure under conditions that represent actual use or are comparable with other studies. 23

Text Input: Techniques and Research Tools Measuring Performance Measuring error rate What is an error? –A character in wrong position?abba abbbaabbba –How about corrections and corrections withing corrections? The best practice: –Compute string distance (levenshtein’s algorithm) –Compute input/character (dependent KSPC) –Edit distance gives the number of errors –KSPC is a measure of the efficiency of the writing method including the effort needed for corrections to achieve the measured error rate. More at: (CHI2001 Extended Abstracts) (NordiCHI) 24

Text Input: Techniques and Research Tools Tradeoffs SpeedAccuracy Minimum input per characterFreedom of input Novice SpeedExpert speed Optimal device useDevice Independence 25

Text Input: Techniques and Research Tools More Info My text input research page: Links to other sites Bibliography Papers 26