Text Input to Handheld Devices for People with Physical Disabilities Brad A. Myers and Jacob O. Wobbrock Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA Brad A. Myers and Jacob O. Wobbrock Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA Human Computer Interaction Institute School of Computer Science Carnegie Mellon University

Brad MyersCarnegie Mellon2 Motivation l Provide text entry for people with physical disabilities l For handheld devices like Palm Pilots l Also called Personal Digital Assistants (PDAs) l Also use Palm Pilots as interface to desktop computers l Provide text entry for people with physical disabilities l For handheld devices like Palm Pilots l Also called Personal Digital Assistants (PDAs) l Also use Palm Pilots as interface to desktop computers

Brad MyersCarnegie Mellon3 Part of the Pebbles Project l Investigate use of handheld devices like Palm Pilots and PocketPCs, and Smartphones at the same time as PCs and other devices PEBBLESPEBBLES PEBBLESPEBBLES DAs for ntry of oth ytes and ocations from xternalources. DAs for ntry of oth ytes and ocations from xternalources.

Brad MyersCarnegie Mellon4 First Steps l Support people with Muscular Dystrophy l Low strength, limited motion, but high accuracy l Tiny keyboard on handheld screens are good l Support people with Muscular Dystrophy l Low strength, limited motion, but high accuracy l Tiny keyboard on handheld screens are good

Brad MyersCarnegie Mellon5 Use Our Remote Commander l Palm Pilot application to control PC using the Palm l Extend tiny keyboard with other keys l Word completion l Other adaptations: no auto-repeat, no auto-off l Mouse control on same screen l Wireless or wired l Palm Pilot application to control PC using the Palm l Extend tiny keyboard with other keys l Word completion l Other adaptations: no auto-repeat, no auto-off l Mouse control on same screen l Wireless or wired

Brad MyersCarnegie Mellon6 But what about other disabilities? l Cerebral Palsy, Parkinson’s, Spinal Injury, and many others l Loose accuracy as well as strength l Tiny buttons are exactly wrong l Still can’t do Graffiti, etc. l Cerebral Palsy, Parkinson’s, Spinal Injury, and many others l Loose accuracy as well as strength l Tiny buttons are exactly wrong l Still can’t do Graffiti, etc.

Brad MyersCarnegie Mellon7 New Design: EdgeWrite™ l Added custom template with a square hole to input area l Gain stability from edges l Travel along edges and into corners l Designed alphabet using gestures inside of hole, along edges and into corners l Added custom template with a square hole to input area l Gain stability from edges l Travel along edges and into corners l Designed alphabet using gestures inside of hole, along edges and into corners

Brad MyersCarnegie Mellon8 Alphabet Properties l Recognizes based on sequence of corners l Users can wiggle during diagonals l Not speed dependent l Easy to customize and adapt letter forms l Even user-driven l Letters “feel” normal l Use multiple strokes along edge l Presentation (only) uses arcs l No capitalization mode l Capitals end in upper left l No special mode for numbers I on left, 1 on right l Recognizes based on sequence of corners l Users can wiggle during diagonals l Not speed dependent l Easy to customize and adapt letter forms l Even user-driven l Letters “feel” normal l Use multiple strokes along edge l Presentation (only) uses arcs l No capitalization mode l Capitals end in upper left l No special mode for numbers I on left, 1 on right

Brad MyersCarnegie Mellon9 Actual Alphabet from User Studies l Wanted high guessability l Correlates with initial usability l Designed a technique to let users guess the gestures l Told users the rules l Then resolve conflicts if same shape used twice l Left multiple forms for each letter l Result: guessability improved from 51% to 80% l Wanted high guessability l Correlates with initial usability l Designed a technique to let users guess the gestures l Told users the rules l Then resolve conflicts if same shape used twice l Left multiple forms for each letter l Result: guessability improved from 51% to 80%

Brad MyersCarnegie Mellon10 Current Alphabet

Brad MyersCarnegie Mellon11 EdgeWrite on Palm l Novice able-bodied users l Same speed with Graffiti and EdgeWrite l But EdgeWrite 18% more accurate l Handicapped users l Too few for statistical significance, but dramatic l E.g., accuracy: 31% with Graffiti vs. 94% EdgeWrite l For example: l “The dog is going fast” vs: “The g i gbsiangu% fast” l Novice able-bodied users l Same speed with Graffiti and EdgeWrite l But EdgeWrite 18% more accurate l Handicapped users l Too few for statistical significance, but dramatic l E.g., accuracy: 31% with Graffiti vs. 94% EdgeWrite l For example: l “The dog is going fast” vs: “The g i gbsiangu% fast”

Brad MyersCarnegie Mellon12 EdgeWrite on Game Controllers l Some joysticks use a square hole l Current techniques are very awkward: l On-screen keyboard l “Date-stamp” cycling letters at each spot l Some joysticks use a square hole l Current techniques are very awkward: l On-screen keyboard l “Date-stamp” cycling letters at each spot

Brad MyersCarnegie Mellon13 EdgeWrite on Wheelchair l Look at people who use power wheelchairs l Custom hardware and software to interface to commercial joystick l Informally evaluated with 7 disabled power- wheelchair users l 6 with Cerebral Palsy, 1 with Multiple Sclerosis l Look at people who use power wheelchairs l Custom hardware and software to interface to commercial joystick l Informally evaluated with 7 disabled power- wheelchair users l 6 with Cerebral Palsy, 1 with Multiple Sclerosis

Brad MyersCarnegie Mellon14 Touchpad EdgeWrite l Adaptation that uses elevated edges around a Synaptics touchpad l May be easier for people with motor impairments l Adaptation that uses elevated edges around a Synaptics touchpad l May be easier for people with motor impairments

Brad MyersCarnegie Mellon15 Results l Touchpad worked better than wheelchair joystick l Still pretty slow l Touchpad worked better than wheelchair joystick l Still pretty slow

Brad MyersCarnegie Mellon16 EdgeWrite on 4 Buttons l Only need four corners l Implementation for any 4 keys l Four keys on the Logitech steering wheel l Not necessarily for text entry while driving l Text entry especially for navigation system l Only need four corners l Implementation for any 4 keys l Four keys on the Logitech steering wheel l Not necessarily for text entry while driving l Text entry especially for navigation system

Brad MyersCarnegie Mellon17 EdgeWrite with Trackball & Pointing Stick l Newest version l Many disabled users prefer trackball for desktop pointing l EdgeWrite for desktop l Optimized for trackball and pointing stick l Also works with regular mouse l Highly parameterized l Input window can be transparent when in use l Has help, etc. l Up to 20 wpm l Newest version l Many disabled users prefer trackball for desktop pointing l EdgeWrite for desktop l Optimized for trackball and pointing stick l Also works with regular mouse l Highly parameterized l Input window can be transparent when in use l Has help, etc. l Up to 20 wpm

Brad MyersCarnegie Mellon18 Initial Reaction to Trackball Version l One user in long-term trial l Spinal injury, quadriplegic, with some mobility of one arm l Given up on-screen keyboard in favor of EdgeWrite l Still uses speech entry when possible l With practice, EdgeWrite is faster “With an on-screen keyboard, there is just too much visual scanning and concentration. In EdgeWrite, if you know the letter, you just bang it out by feel, and you can keep your eyes on your document.” l One user in long-term trial l Spinal injury, quadriplegic, with some mobility of one arm l Given up on-screen keyboard in favor of EdgeWrite l Still uses speech entry when possible l With practice, EdgeWrite is faster “With an on-screen keyboard, there is just too much visual scanning and concentration. In EdgeWrite, if you know the letter, you just bang it out by feel, and you can keep your eyes on your document.”

Brad MyersCarnegie Mellon19 Summary and Future Work l EdgeWrite works across a wide range of devices l With the same alphabet l Learn once l Use most comfortable device l Future work l Continue to refine l Add word completion l Further tests and deployment l New devices (phone?) l EdgeWrite works across a wide range of devices l With the same alphabet l Learn once l Use most comfortable device l Future work l Continue to refine l Add word completion l Further tests and deployment l New devices (phone?)

Brad MyersCarnegie Mellon20 Acknowledgements l Funded by grants from NSF, General Motors and The NEC Foundation of America l Equipment grants from: l Synaptics, Inc. l A.T. Sciences l Microsoft l Funded by grants from NSF, General Motors and The NEC Foundation of America l Equipment grants from: l Synaptics, Inc. l A.T. Sciences l Microsoft