1. 23-May-151 Multiparty Communication with a Tour Guide ECA Aleksandra Čereković HOTLab group Department of telecommunications Faculty of electrical engineering and computing Zagreb University http://hotlab.tel.fer.hr

2. Outline 23-May-152  Project overview  Background Information  eNTERFACE ‘08 system  features  scenario  configuration  Project members

3. Project Overview 23-May-153  Project target  System with an Embodied Conversational Agent (ECA)  ECA is a tour guide which describes the city of Dubrovnik to two visitors and interacts with them  We aim to explore issues of a multiparty support in ECA system with focus on:  Dialogue support  Participant roles  Interaction Management  Dialogue structure  Nonverbal behavior model of an ECA  Getting attention, model of gazing, maintaining a conversational flow

4. Background Information 23-May-154  During eNTERFACE ‘06 workshop we developed a Tour Guide system  System is based on common GECA framework  GECA Platform, GECA Protocol, GECA Plugs  Connects various system components  Tranfers messages between components in the real time  Enables simple system integration

5. eNTERFACE ‘08 system scenario 23-May-155  In our system scenario ECA has a role of narrator  ECA takes visitor through 5 different scenes in the city  Users can interrupt ECA  They can raise their hands to take a speaker’s role or just say something  User utterances are predefined: they can make comments, have requests or just express thier willingness to leave the session  Durring interaction ECA keeps initiative and gets attention

6. eNTERFACE ‘08 system design 23-May-156

7. Component functionalities (I) 23-May-157  Input  Captures speech and users’ position, size, openness of the eyes and the mouth, facial orientation (Okao’s Vision)  Deliberative phase components  Input understanding  Interprets raw messages coming from the system input  Detects speaker and his utterance (request, comment, leave the system)  Detects speech collision and mutual conversation between users  Decision Making Planner (Dialogue Management)  Generates transitions between dialogue states in a multiparty interaction (Information State, Nonverbal Event, Specifying addressee, Grounding judgment, Assuming the next speaker)  MIDIKI DM toolkit will be extended to manage multiparty interaction

8. Component functionalities (II) 23-May-158  Deliberative phase components  Behavior Planner  Generates appropriate response by an ECA  Dynamically changes ECA’s behaviors according the Information State  Output  Player  Generates ECA’s behaviors selected by Behavior Planner  Has a support to stop utterance and start a new one

9. Project Members 23-May-159  Project Researches  Takuya Furukawa* (Input, Input Understanding)  Huang Hsuang- Hung* (DM, Behavior Planner), Shinya Takeda** and Yuji Yamaoka** (DM)  Aleksandra Cerekovic*** (Output)  Project supervisors  Toyoaki Nishida*  Yukiko Nakano**  Igor S. Pandzic*** *University of Kyoto, Graduate School of Informatics, Nishida-Sumi Lab ** Seikei University (Tokyo), Faculty of Science and Technology ***University Of Zagreb, Faculty of Electrical Engineering and Computing, HOTLab group