2. Dialogue Design Speech, pen, and gestures

3. Speech Output  Tradeoffs in speed, naturalness and understandability  Male or female voice? Technical issues (freq. response of phone) User preference (depends on the application)  Rate of speech Technically up to 550 wpm! Depends on listener  Synthesized or Pre-recorded? Synthesized: Better coverage, flexibility Recorded: Better quality, acceptance

4. Speech Output  Synthesis Quality depends on software ($$) Influence of vocabulary and phrase choices http://www.research.att.com/~ttsweb/tts/demo.php  Recorded segments Store tones, then put them together The transitions are difficult (e.g., numbers)

5. Designing Speech Interaction  Constrain vocabulary Limit valid commands Structure questions wisely (Yes/No) Manage the interaction Examples?  Slow speech rate, but concise phrases  Design for failsafe error recovery  Visual record of input/output  Design for the user – Wizard of Oz

6. Speech Tools/Toolkits  Java Speech SDK FreeTTS 1.2 http://freetts.sourceforge.net/docs/index.php  IBM JavaBeans for speech  Microsoft speech SDK (Visual Basic, etc.)  OS capabilities (speech recognition and synthesis built in to OS) (TextEdit)  VoiceXML

7. General Issues – Speech/NL  Who is in control - user or computer  Initial training required  Learning time to become proficient  Speed of use  Generality/flexibility/power  Special skills - typing  Gulf of evaluation / gulf of execution  Screen space required  Computational resources required

8. Non-speech audio  Good for indicating changes, since we ignore continuous sounds Traditionally used for warnings, alarms or status information  Provides secondary representation Supports visual interface Provides information that helps reduce error  Tradeoff in using natural (real) sounds vs. synthesized noises.

9. Non-speech audio examples  Error ding  Info beep  Email arriving ding  Recycle  Battery critical  Logoff  Logon Others?

11. Pen & Mobile dialog  Stylus or finger Tradeoffs of each?  Pen as a standard mouse (doubleclick?)  Variety of platforms Desktop touch screens or input pads (Wacom) Tablet PCs Handheld and Mobile devices Electronic whiteboards  Platforms often involve variety of size and other constraints

12. Mobile devices  More common as more platforms available PDA Cell phone Ultra mobile tablets  Smaller display (160x160), (320x240)  Few buttons, different interactions Free-form ink Soft keyboard Numeric keyboard => text Stroke recognition Hand printing / writing recognition

13. http://www.oqo.com/ http://www.blackberry.com/ http://www.intel.com/design/mobile/platform/umpc.htm Ultra-Mobile PC (Samsung) Palm Z22 handheld http://www.palm.com

14. Soft Keyboards  Common on PDAs and mobile devices Tap on buttons on screen

15. Soft Keyboard  Presents a small diagram of keyboard  You click on buttons/keys with pen  QWERTY vs. alphabetical Tradeoffs? Alternatives?

16. Numeric Keypad -T9  Tegic Communications developed  You press out letters of your word, it matches the most likely word, then gives optional choices  Faster than multiple presses per key  Used in mobile phones  http://www.t9.com/

17. Quikwriting - Stroke Recogntion  Developed by Ken Perlin

18. Quikwriting Example pl http://mrl.nyu.edu/~perlin/demos/Quikwrite2_0.html e Said to be as fast as graffiti, but have to learn more

19. Hand Printing / Writing Recognition  Recognizing letters and numbers and special symbols  Lots of systems (commercial too)  English, kanji, etc.  Not perfect, but people aren’t either! People - 96% handprinted single characters Computer - >97% is really good  OCR (Optical Character Recognition)

20. Recognition Issues  Boxed vs. Free-Form input Sometimes encounter boxes on forms  Printed vs. Cursive Cursive is much more difficult  Letters vs. Words Cursive is easier to do in words vs individual letters, as words create more context Usually requires existence of a dictionary  Real-time vs. off-line

21. Special Alphabets  Graffiti - Unistroke alphabet on Palm PDA What are your experiences with Graffiti?  Other alphabets or purposes Gestures for commands

22. Pen Gesture Commands -Might mean delete -Insert -Paragraph Define a series of (hopefully) simple drawing gestures that mean different commands in a system

23. Pen Use Modes  Often, want a mix of free-form drawing and special commands  How does user switch modes? Mode icon on screen Button on pen Button on device

24. Error Correction  Having to correct errors can slow input tremendously  Strategies Erase and try again (repetition) When uncertain, system shows list of best guesses (n-best list) Others??

25. Free-form Ink  Ink is the data, take as is  Human is responsible for understanding and interpretation  Often time-stamped  Applications Signature verification Notetaking Electronic whiteboards Sketching

26. Electronic whiteboards  Smartboard and Mimio  Can integrate with projection  Large surface to interact with Issues? http://www.smarttech.com/ http://www.mimio.com/

27. Real paper  Anoto digital paper and pen technology (http://www.anoto.com/)  Issues? http://www.logitech.com/ Logitech io Digital Writing System

28. General Issues – Pen input  Who is in control - user or computer  Initial training required  Learning time to become proficient  Speed of use  Generality/flexibility/power  Special skills - typing  Gulf of evaluation / gulf of execution  Screen space required  Computational resources required

29. Other interesting interactions  Gesture input Specialized hardware, or tracking  3D interaction Stereoscopic displays Virtual reality  Immersive displays such as glasses, caves  Augmented reality Head trackers and vision based tracking

30. Exercise  Bus locator service Design a speech-based service that uses your cell phone Design a pen-based service that uses a PDA