1. Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams Spelman College

2. Motivation How should an intelligent agent incorporate communication? How does communication and behavior integrate within an agent model? How can ideas from many different dialogue models and conversation examples by incorporated? How can one validate the correctness of an agent conversational model?

3. Motivation Negotiation Persuasion Formal Argumentation Informal Argumentation Integrated Model Cooperative Planning Belief Grounding

4. Dialogue Agent Paradigm Embedded dialogue manager –Perception processing Embedded behavior model DIALOGUE CAPABLE AGENT ActionsPercepts Dialogue Manager Behavioral Model Conversation Internal API or Language

5. Goal A unified conversational architecture for intelligent agents and assistants –Representation of communication –Incorporation of communication within behavior –Incorporating a variety of models and ideas into a single integrated model –Validation of the conversational model –Independent of dialogue interpreter or agent

6. Focus CommunicationBehavior Agent Implementation Bring behavior to communication as much as possible

7. Focus CommunicationBehavior Agent Implementation Separate dialogue interpreter from agent –Parallel development of each –Interchangeable components

8. Human Interpreter

9. The Practical Communication Language (PCL) Hypothesis There exists a language between that of a human conversational participant and that of an intelligent agent. This language is capable of abstracting away the complexity of human language while yet maintaining the practical information of the conversation. Adding toThe Practical Dialogue Hypothesis and The Domain-independence Hypothesis stated in Allen 2000.

10. Current Utterance-Based Languages Application Programmer Interfaces (API) –Task Management Interface Specialized Languages –Artificial Discourse Language –Universal Communication Language (Interlingua) –Parameterized Action Representation Discourse and Speech Act Tags Agent Communication Languages –Foundation for Intelligent Physical Agents (FIPA-ACL) –Knowledge Query Manipulation Language (KQML)

11. Searching for the language… True PCL is ideal and volatile Ever expanding definition of ‘practical’ PCL should be abstracted * of 1.Region and dialect aspects of language. 2.Informal, Colloquial, Slang and Idiomatic expressions. 3.Modality (Spoken, Written, Gestural, GUI) *Translated or Incorporated not discarded.

12. Approach Task Communication Language (TCL) –Messages to/from Dialogue Interpreter Task Model Task Communication Model Interaction Model TCL Message –Set of integrated models

13. TCL Messages Header –Generator: Generated utterance or gesture –Addressee: Intended Receivers of message –Receiver: Participants who saw or heard –Uncertainty in all above fields –Interpretation Stack Information obtained at al levels of translation –Used by feedback mechanism for improving interpreters –Content Meaning-Action Concept

14. Conversational Paradigms HumanAgent Human Observation Manager / Assistant Teacher / Student Coach / Player Peer / Peer Agent Agent Communication Semantic Web

15. Conversational Paradigms Single AgentMultiple Agent Single Human Current Trend Simulation and Training Consumer Products Multiple Human Mediator Discussion Leader Team Coordinator Referee Semantic Web Marketplaces Teamwork

16. Conversational Paradigms Not just endpoint to endpoint Multiple segmentations –A conversation between people listening in on another conversation

17. Meaning-Action Concept Meaning of utterance or gesture Possible association with action. “propose( action: )” “propose( goal: )” “reject( goal: )” “counter-propose( action: )” “query( justification( action: ) )”

18. Meaning-Action Concepts (MAC) Defined in ontological format –Allows for rollback to known concepts –Manageable growth of concept space Proposal( ) Counter-Proposal( ) Commit( ) Commit( confidence:30 )

19. Task Communication Expression First-order logic expression of MAC. –Conjunction: Multiple Meanings –Disjunction: Ambiguity –Expressiveness and complexity

20. Focus CommunicationBehavior

21. Task Model Task Concepts: Objectives Recipes Actions Resources Situations States Constraints Beliefs Intentions Metrics Priorities

22. Task Model Task Operations: Adoption Selection Deferment Abandonment Release Identification Evaluation Modification

23. Task Model CommunicationBehavior Task Model

24. Task-Communication Model Integrate the task concepts and operators –Communication with a dialgue interpreter –Task manipulation of an intelligent agent Modeling can be language independent –CFSM, CPetriNet, Inference-Based, BDI...

25. Task-Communication Model Nested task operators Lower layers: –Persuasion, inquiry, deliberation, formal argumentation, informal argumentation, clarification, explanation… Higher layers: –Negotiation, cooperative planning, learning through orders, command and control…

26. Task-Communication Model CommunicationBehavior Task Model Task-Communication Model

27. Trivial Example: Communicative acts –TCL Messages Behavioral acts –Agent integration [IN]: Propose( Action A ) Evaluate( Action A ) [OUT]: Reject( Action A ) [OUT]: Accept( Action A )

28. Integration Model Generated automatically through tracing the Task-Communication Model Represents incoming and outgoing message sequences and possibilities

29. Task-Communication Model [IN]: Propose( Action A ) Evaluate( Action A ) [OUT]: Reject( Action A ) [OUT]: Accept( Action A )

30. Interaction Model Extraction of Input-Output Sequences [IN]: Propose( Action A )[OUT]: Accept( Action A ) [OUT]: Reject( Action A ) [OUT]: Refine( Action A ) [OUT]: Clarify( Goal G ) [OUT]: Counter( Action A )

31. Interaction Model CommunicationBehavior Task Model Interaction Model Task-Communication Model

32. Interaction Model Validated with known TCL sequences –If sequence is covered, path validated –If sequence is missing, update model Assists in integration of models Prove various properties –Turn Taking –Liveness –Sanity checks

33. Mixed-Initiative Control No longer in hands of dialogue interpreter –Also managed by ‘task communication model’ Task-communication model must –Initiate dialogue sequences –Manage turn-taking context tracking autonomy

34. Stratagus Open source real-time strategy engine –Multiple data sets for varying games Dynamically changing environment Real time resource management

36. Discussion