1. TAUCHI – Tampere Unit for Computer-Human Interaction Presenting Information with Speech SUI/ASUI/SPI group since 1998 Experiences from several of our applications –Bussimies & Interact timetable systems –Ovimies (Doorman) –Mailman (Postimies) Questions remaining for all areas Jaakko Hakulinen (jh@cs.uta.fi)

2. TAUCHI – Tampere Unit for Computer-Human Interaction TOC ”How do users know what to say” help and guidance Tutoring Route descriptions –multimodality, language generation E-mail reading –multilinguality, e-mail parsing

3. TAUCHI – Tampere Unit for Computer-Human Interaction Help and Guidance ”How Do Users Know What To Say” ( Yankelovich in Interactions november + december 1996) –”functionality of application is hidden” (in speech only systems) –> prompt design is the key –implicit vs. explicit prompts –incremental prompts / tapering –hints how to tell (complete) functionality, i.e. create power users?

4. TAUCHI – Tampere Unit for Computer-Human Interaction Help Messages Don’t rely on help messages –comprehensive help requires a lot of text –speech is slow, way too slow Using examples is effective, problem is users ”getting stuck” in them In incremental help, rephrase further messages completely Postimies has context sensitive, reactive help with ”what next” command (~ in ”Universal Speech Interfaces” Rosenfeld, Olden, Rudnicky, Interactions november + december 2001 )

5. TAUCHI – Tampere Unit for Computer-Human Interaction Tutoring Something we have planned for the Postimies application the system tutors the user, user learns by doing requires a user model to guide what to teach questions: –should the tutor be a separate ”person”, different voice etc.? –will users try to speak to the tutor, should that be possible?

6. TAUCHI – Tampere Unit for Computer-Human Interaction Doorman Outside the user identifies the person (s)he is coming to see System opend door (lock) The system lets the user in (day hours, evening plays doorbell) Inside the system guides the user to the room where the host is Current version implemented by Prusi, more info and demo in the afternoon

7. TAUCHI – Tampere Unit for Computer-Human Interaction Route Descriptions complex thing to describe [those directions] –more visual than linguistic information multimodality, i.e. Ovimies has puppets with hand gestures –removes left/right hearer/speaker ambiguity

8. TAUCHI – Tampere Unit for Computer-Human Interaction Requirements for NLG Natural descriptions require some information about surroundings, landmarks etc. –names for places, rooms etc. –information, what is good landmark (uniqueness) –information about visibility

9. TAUCHI – Tampere Unit for Computer-Human Interaction route description generation system has 2d geometry with of premises ”semantic annotation” and place names ”unique” targets are defined first shortest route is searched route is constructed of legs and turns start and each leg is described (landmarks, distances, directions, place names) + gestures text is generated for tts, gestures timed

10. TAUCHI – Tampere Unit for Computer-Human Interaction route description questions what if we have several robots? –guide to the next robot or all the way –do users stop how to speak (how much)? –complex information -> slow presentation? –or keep it simple overall level of descriptions –back there on left side –turn left, forward next to meeting room...

11. TAUCHI – Tampere Unit for Computer-Human Interaction reading e-mails system reads e-mails in telephone problems of e-mail domain –e-mail is made to be viewer, not heard –large amount of e-mail hard to navigate –multilingual content Current Postimies version by Salonen & Helin, more info and demo in the afternoon

12. TAUCHI – Tampere Unit for Computer-Human Interaction Example problems Dear members, Here is your last issue including informations gathered in the last minutes while this issue was under preparation. Chris W. -- ________________________________________________________________________________ Professor Christian J. WellekensTel: +33 (0) 4 93 00 26 28 Dpt Multimedia CommunicationsSecr:+33 (0) 4 93 00 26 33 EURECOMFax: +33 (0) 4 93 00 26 27 2229 route des Cretes BP 193 F-06904 Sophia AntipolisE-mail:welleken@eurecom.fr FRANCEWeb: http://www.eurecom.fr ________________________________________________________________________________ dash (”-”) characters are skipped completely telephone numbers are parsed (”area code” etc.) web addresses are parsed (”world wide web”) visual columns are lost

13. TAUCHI – Tampere Unit for Computer-Human Interaction What do we do to help this e-mail a grouped to folders (if many of them) e-mail structured into xml –internet addresses –numbers (telephone, large digits etc.) –horisontal rulers –emoticons –language in paragraph level reading part decides how to read it all

14. TAUCHI – Tampere Unit for Computer-Human Interaction questions about e-mail reading how to inform listener, what is interpretation and what is straight text? –can we user different voices? –How about voice families (male/female, young/old...) navigation inside a mail –currently paragraph level, smaller units?