Dialogue systems at KTH

The August project

Part of the Stockholm Cultural Capital of Europe '98 program Swedish spoken dialogue system with an animated agent Named after the 19 th century author August Strindberg Built with speech technology components developed at TMH Designed to be easily expanded and reconfigured Communicates using synthetic speech, facial expressions, maps, tables and a thought balloon Used to collect spontaneous speech data Key features of the August system

The topics of the August system Stockholm (events and general information) Yellow pages KTH and speech technology August Strindberg Greetings and social utterances Comments about the system capabilities and the discourse

A sample video of the system environment There are 24,000 islands in the Stockholm archipelago Can you see me? We are now in the center of Stockholm Hello AugustHello!How are you today?Excellent as always!What can I ask about? I can tell you about restaurants in Stockholm Can you recommend a restaurant in the neighborhood There are over 700 restaurants in Stockholm Is there one close by?You must specify a street Are there any restaurants on Blekingegatan? The information is shown on the map Where are we?But we are here!Where?I did not understand that!Where are we?But we are here!Are we in Kulturhuset? My life cannot be measured in terms of days and years! My life cannot be measured in terms of days and years!

The August system What is your name? I call myself Strindberg, but I dont really have a surname When were you born?Strindberg was born in 1849What do you do for a living? I can answer questions about Strindberg, KTH and Stockholm How many people live in Stockholm? Over a million people live in the Stockholm area Do you like it here? People who live in glass houses should not throw stones Yes, that was a smart thing to say! I come from the department of Speech, Music and Hearing The Royal Institute of Technology! The information is shown on the map Thank you!You are welcome!Good bye! Perhaps we will meet soon again! Yes, it might be that we will! Strindberg was married three times! Strindberg was married three times!

The AdApt system Multimodal conversational dialogue system System features an animated agent and an interactive map in an apartment domain Developed by researchers from Centre for Speech Technology (KTH) and Telia Research (Linda Bell, Johan Boye, Rolf Carlson, Jens Edlund, David House, Joakim Gustafson and Mats Wirén)

The Adapt user interface

Animated faces are appearing everywhereThey offer us advice on stock exchange and there are virtual news anchors None of them understand what we sayAt the Centre for Speech Technology at the Royal Institute of Technology researchers are developing animated agents that will be able to engage in dialogues The animated agent Urban will be able to pursue a freer sort of conversation My name is Urban. What can I do for you? Well, what can you do? I can tell you about available apartments in downtown Stockholm I'd like to look at two-room apartments on the south side of town with a balcony and a fireplace I have information about five such apartments, and I now show their locations on the map Thank you very muchThis is fun, but I could just as well access this information through a text-based interface What is the purpose of speaking computer?The same information can be accessed, but perhaps more efficiently The system can also compare different apartmentsI think speaking computers will be important in the futureYou don't need the keyboardDisabled people can use them and inexperienced computer users What are the difficulties involved?Our goal is to build a system that can handle a more complex sort of dialogue A lot of problems remain to be solved many things in the dialogue can go wrong misunderstandings occur We are now collecting data to see how people speak to such a computer...to try to improve the performance of future systemsDo people speak about anything just because it is a face they are talking to? We get a sense of trust, since we believe the animated agent has human capabilities This is something we have observed in our data - people like to socialize with Urban and they enjoy talking to him about this and that. And once they have been developed, what do you think their functions will be? They will be used for a wide range of thingsThey can be used as personal assistants and there is a commercial potential We will probably see them selling things. When can we meet Urban again Stockholm Technical museum, where he will be part of an exhibition shown this summer Urban will make a guest appearance thereAnd you can talk to himDo you know what the next part of the program will be about? Yes, about magnifying glasses that the Vikings used a thousand years ago The AdApt project presented in a popular science program broadcasted on Swedish national television, May 2000

Hello my name is Urban and I can tell you about nice apartments in downtown Stockholm Hi Urban..eh..are there any available apartments.. around here How many rooms do you wantI want a three room apartment..eh..it would be nice with an older one There are five apartments in the shaded areaWell this apartment on Nybrogatan, can you tell me a little bit more about it The blue apartment was built in It has a tiled stove and a parquette floor Alright and..eh..the yellow apartmentThe yellow building was constructed in 1883Does this apartment also have tiled stoves?Im sorry but I cant answer that question