1 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Applications of distributed dialogue systems: The KTH Connector Jens Edlund Anna Hjalmarsson Aalborg, November 10th, 2005

2 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Why spoken interfaces? Speech is good because it is Hands free Eyes free Intuitive – already known Robust – e.g. redundancy, grounding Flexible Responsive Efficient …

3 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What do they do? Purpose —Problem solving —Information seeking —Transactions —Control —… Initiative —System —User —Mixed Modality —Multimodal (input and/or output) —Unimodal (input and/or output) …

4 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What else do they do? Transactions, e.g. ATIS Web forms (and sales reps) Call routing, e.g. HMIHY DTMF (and operators) Info services, e.g. weather Web forms, DTMF Control, e.g. smart rooms Buttons, remote controls Spoken interfaces often replace or complement existing automated interfaces, for example:

5 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Alternative interface systems Speech is an alternative or substitute Commonly built to be as good as or better than the corresponding system Symmetry often required – what can be done with the original system should be doable with speech and vice versa

6 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Why spoken interfaces? Hands free Eyes free Intuitive – already known Robust – e.g. redundancy, grounding Flexible Responsive Efficient … Hands free Eyes free Intuitive – already known Robust – e.g. redundancy, grounding Flexible Responsive Efficient … These aspects are often not exploited much in alternative interface systems

7 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson What metaphor to rely on? The voice as an input device? —”You may use your voice to order” From a travel booking instruction in Swedish —”It didn’t give me any alternatives” From a post interview with call routing user The computer as a human? —Problem: Turing test

8 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Summary Speech can be used successfully as an alternative or complementary interface to other interfaces, particularly when hands and/or eyes are occupied, disabled, or otherwise impratcical to use The advantages of speech promised by analogies to human-human communication may not be fully exploited in such domains

9 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson The KTH Connector Background Domain System

10 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Background: CHIL CHIL: Computers in the Human Interaction Loop ( EU funded, IP ) The dialogue system as an unobtrusive conversational partner in a group of humans

11 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson A telephony based secretary Wide range of complexity From answer phone…

12 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson A telephony based secretary …to meeting assistant

13 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Dialogue setup Multimodal, multiparty, system barge-in Multiparty telephony

14 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson System highlights System output is included in discourse model (incrementally) Prosody enhanced endpointing System may dial users

15 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Responsiveness When should we respond? Turn yielding & turn holding cues: —Prosody —Gaze —…

16 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Incrementality When can we respond? —After a ”long enough” silence? —At some semantic or syntactic completeness? —After any ”word”? —Anytime? What has actually happened? —System and user barge-in —Keep track of what we say, as well as what the user says

17 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Research highlight: Unobtrusiveness With what should we respond? —Long prompts can be annoying —Short ones may be insufficient How? —Efficiently or politely? —Speech, gesture, other? Do we have to ”take turn”? —Backchannels —Grounding

18 © 2005 CHIL KTH ASIDE 2005, Aalborg, Applications of distributed dialogue systems: The KTH Connector Jens Edlund & Anna Hjalmarsson Thank you for your attention.