Dialogue Modeling and Dialogue Management Frameworks Svetlana Stoyanchev Seminar on SDS, Columbia 2/16/2015.

Slides:



Advertisements
Similar presentations
ARCHITECTURES FOR ARTIFICIAL INTELLIGENCE SYSTEMS
Advertisements

OpenDial Framework Svetlana Stoyanchev SDS seminar 3/23.
5/10/20151 Evaluating Spoken Dialogue Systems Julia Hirschberg CS 4706.
INTERNATIONAL CONFERENCE ON NATURAL LANGUAGE PROCESSING NLP-AI IIIT-Hyderabad CIIL, Mysore ICON DECEMBER, 2003.
Seminar on Spoken Dialogue Systems
Search Engines and Information Retrieval
U1, Speech in the interface:2. Dialogue Management1 Module u1: Speech in the Interface 2: Dialogue Management Jacques Terken HG room 2:40 tel. (247) 5254.
Shared Ontology for Knowledge Management Atanas Kiryakov, Borislav Popov, Ilian Kitchukov, and Krasimir Angelov Meher Shaikh.
Conversational Agent 1.Two layers: Dialogue manager and Conversational agent. 2.Rule-Based Translator (ELIZA and PARRY) 3. Layer one: Dialogue Manager.
Information, action and negotiation in dialogue systems Staffan Larsson Kings College, Jan 2001.
Goteborg University Dialogue Systems Lab GoDiS and TrindiKit MITRE workshop 27/10-03 Staffan Larsson Göteborg University Sweden.
CMPT 370: Information Systems Design Instructor: Curtis Cartmill, Simon Fraser University – Summer 2003 Lecture Topic: Layered Architecture Class Exercise:
1 Chapter 19: Dialogue and Conversational Agents Nadia Hamrouni and Ahmed Abbasi 12/5/2006.
1212 Management and Communication of Distributed Conceptual Design Knowledge in the Building and Construction Industry Dr.ir. Jos van Leeuwen Eindhoven.
ADL Slide 1 December 15, 2009 Evidence-Centered Design and Cisco’s Packet Tracer Simulation-Based Assessment Robert J. Mislevy Professor, Measurement &
The chapter will address the following questions:
Lecture 1, 7/21/2005Natural Language Processing1 CS60057 Speech &Natural Language Processing Autumn 2005 Lecture 1 21 July 2005.
AQUAINT Kickoff Meeting – December 2001 Integrating Robust Semantics, Event Detection, Information Fusion, and Summarization for Multimedia Question Answering.
Ashish Vaswani Speech acts for Dialogue agents, Coding schemes and dialogue act taxonomies.
Katanosh Morovat.   This concept is a formal approach for identifying the rules that encapsulate the structure, constraint, and control of the operation.
9/8/20151 Natural Language Processing Lecture Notes 1.
Search Engines and Information Retrieval Chapter 1.
Lecture 12: 22/6/1435 Natural language processing Lecturer/ Kawther Abas 363CS – Artificial Intelligence.
Some Thoughts on HPC in Natural Language Engineering Steven Bird University of Melbourne & University of Pennsylvania.
Interactive Dialogue Systems Professor Diane Litman Computer Science Department & Learning Research and Development Center University of Pittsburgh Pittsburgh,
EXCS Sept Knowledge Engineering Meets Software Engineering Hele-Mai Haav Institute of Cybernetics at TUT Software department.
Knowledge representation
Markup of Multimodal Emotion-Sensitive Corpora Berardina Nadja de Carolis, Univ. Bari Marc Schröder, DFKI.
Theories of Discourse and Dialogue. Discourse Any set of connected sentences This set of sentences gives context to the discourse Some language phenomena.
Spoken dialog for e-learning supported by domain ontologies Dario Bianchi, Monica Mordonini and Agostino Poggi Dipartimento di Ingegneria dell’Informazione.
1 Computational Linguistics Ling 200 Spring 2006.
Learning Science and Mathematics Concepts, Models, Representations and Talk Colleen Megowan.
EEL 5937 Agent communication EEL 5937 Multi Agent Systems Lecture 10, Feb. 6, 2003 Lotzi Bölöni.
Adaptive Hypermedia Tutorial System Based on AHA Jing Zhai Dublin City University.
Towards multimodal meaning representation Harry Bunt & Laurent Romary LREC Workshop on standards for language resources Las Palmas, May 2002.
 Copyright 2008 Digital Enterprise Research Institute. All rights reserved. Semantic on the Social Semantic Desktop.
Chapter 3 DECISION SUPPORT SYSTEMS CONCEPTS, METHODOLOGIES, AND TECHNOLOGIES: AN OVERVIEW Study sub-sections: , 3.12(p )
Towards A Context-Based Dialog Management Layer for Expert Systems Victor Hung, Avelino Gonzalez & Ronald DeMara Intelligent Systems Laboratory University.
An Ontological Framework for Web Service Processes By Claus Pahl and Ronan Barrett.
Issues in Multiparty Dialogues Ronak Patel. Current Trend  Only two-party case (a person and a Dialog system  Multi party (more than two persons Ex.
Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction John R. Lee Assistive Intelligence Inc. Andrew B. Williams.
ENTERFACE 08 Project 1 “MultiParty Communication with a Tour Guide ECA” Mid-term presentation August 19th, 2008.
A Common Ground for Virtual Humans: Using an Ontology in a Natural Language Oriented Virtual Human Architecture Arno Hartholt (ICT), Thomas Russ (ISI),
Proposed NWI KIF/CG --> Common Logic Standard A working group was recently formed from the KIF working group. John Sowa is the only CG representative so.
DenK and iCat Two Projects on Cooperative Electronic Assistants (CEA’s) Robbert-Jan Beun, Rogier van Eijk & Huub Prüst Department of Information and Computing.
Using Several Ontologies for Describing Audio-Visual Documents: A Case Study in the Medical Domain Sunday 29 th of May, 2005 Antoine Isaac 1 & Raphaël.
Introduction to Dialogue Systems. User Input System Output ?
ERDA : An Empathic Rational Dialog Agent1 Magalie Ochs (1),(2), Catherine Pelachaud (1) and David Sadek (2) (1) IUT de Montreuil, University Paris VIII,
Intelligent Robot Architecture (1-3)  Background of research  Research objectives  By recognizing and analyzing user’s utterances and actions, an intelligent.
1/21 Automatic Discovery of Intentions in Text and its Application to Question Answering (ACL 2005 Student Research Workshop )
The Message Chapter 5.
Information state and dialogue management in the TRINDI Dialogue Move Engine Toolkit, Larsson and Traum 2000 D&QA Reading Group, Feb 20 th 2007 Genevieve.
Modeling Speech Acts and Joint Intentions in Modal Markov Logic Henry Kautz University of Washington.
ACE TESOL Diploma Program – London Language Institute OBJECTIVES You will understand: 1. The terminology and concepts of semantics, pragmatics and discourse.
Ch- 8. Class Diagrams Class diagrams are the most common diagram found in modeling object- oriented systems. Class diagrams are important not only for.
Volgograd State Technical University Applied Computational Linguistic Society Undergraduate and post-graduate scientific researches under the direction.
EEL 5937 Agent communication EEL 5937 Multi Agent Systems Lotzi Bölöni.
A preliminary classification of dialogue genres Staffan Larsson Internkonferens 2003.
HiST The foundation for a dialogic grammar Ivar Tormod Berg Ørstavik.
Goteborg University Dialogue Systems Lab Comments on ”A Framework for Dialogue Act Specification” 4th Workshop on Multimodal Semantic Representation January.
Understanding Naturally Conveyed Explanations of Device Behavior Michael Oltmans and Randall Davis MIT Artificial Intelligence Lab.
Conversational role assignment problem in multi-party dialogues Natasa Jovanovic Dennis Reidsma Rutger Rienks TKI group University of Twente.
A Speech Interface to Virtual Environment Authors Scott McGlashan and Tomas Axling Swedish Institute of Computer Science.
Agent-Based Dialogue Management Discourse & Dialogue CMSC November 10, 2006.
AQUAINT Mid-Year PI Meeting – June 2002 Integrating Robust Semantics, Event Detection, Information Fusion, and Summarization for Multimedia Question Answering.
Viewpoint Modeling and Model-Based Media Generation for Systems Engineers Automatic View and Document Generation for Scalable Model- Based Engineering.
Interpreting as Process
Towards a framework for architectural design decision support
ASP.NET MVC Imran Rashid CTO at ManiWeber Technologies.
Presentation transcript:

Dialogue Modeling and Dialogue Management Frameworks Svetlana Stoyanchev Seminar on SDS, Columbia 2/16/2015

– Dialogue modeling: formal characterization of dialogue, evolving context, and possible/likely continuations – Theoretical approach – Dialogue management SDS module concerned with dialogue modeling – Practical implementation Seminar on Spoken Dialogue Systems, Columbia

Outline Plan-based dialogue Speech Act theory Dialogue Annotations Information State Approach DM Frameworks Seminar on Spoken Dialogue Systems, Columbia

Plan-based Models of Dialogue Theory originators: Cohen, Allen, Perrault This theory was developed Dialogues are treated as a special case of rational (non-communicative) behavior – Dialogue is part of the behavior, not just a communicative interface Participants are rational agents Seminar on Spoken Dialogue Systems, Columbia

Plan-based Dialogue Theory Agent’s actions are based on BDI: Belief, Desire & Intention Dialogue participants are rational agents Environment: kitchen Participants: A and B A is a robot that can move but is not familiar with the kitchen environment B can not move but knows where everything is Seminar on Spoken Dialogue Systems, Columbia

Knowledge and wants Knowledge – KNOW (B, P) B knows that P is true Wants: Make Tea P1: teacup is in a cupboard P2: teapot is on a countertop P3: water is in the sink etc Participant: A Participant: B Seminar on Spoken Dialogue Systems, Columbia

Make tea Participant: A Participant: B Where is a tea cup? Goals Plans: Goals Plans Infer A’s goal/plan (making tea) Infer A’s goal/plan (making tea) Personal goals Its in on the lower shelf in a cupboard above the sink Respond to A Boil water find cattle pour water turn cattle on Get a tea cup locate cup retrieve cup Get a tea bag locate retrieve Combine Boil water find cattle pour water turn cattle on Get a tea cup locate cup retrieve cup Get a tea bag locate retrieve Combine Provide additional info And the sugar is above it Plan construction & knowledge share Seminar on Spoken Dialogue Systems, Columbia

Speech Acts Utterances are actions that update dialogue context A link between the mental states of agents and purposeful communication Roots in Philosophy – Austin (1962), Searle (1975) Effects of an utterance – Locutionary Words that constitute an utterance – Illocutionary Request (semantics and intent of an utterance) – Perlocutionary Underlying intention, effect, pragmatics Seminar on Spoken Dialogue Systems, Columbia

Speech Acts “Are there any Flights from Boston” Effects of an utterance – Locutionary Words that constitute an utterance – Illocutionary Request information about Flights from Boston – Perlocutory Effect: requesting a clerk to provide information Seminar on Spoken Dialogue Systems, Columbia

Speech Acts Are used for planning in dialogue – A dialogue plan is dynamically constructed using Agent goals Agent knowledge Speech act preconditions Speech act effects Seminar on Spoken Dialogue Systems, Columbia

Common Speech Acts Inform Request Response Yes/No question Seminar on Spoken Dialogue Systems, Columbia

Common Speech Acts Inform – Speaker (S), Hearer (H), Preposition (P) – Preconditions: KNOW(S,P) and WISH (S, P, H) – Effect: KNOW(H,P) – Speaker informs the hearer of something by causing hearer to believe that the speaker wants them to know something Seminar on Spoken Dialogue Systems, Columbia

Why aren’t logic and plan‐based approaches used more today? Most systems are simpler, single domain, limited task, don’t need complex reasoning Focus on robustness/speed Need to deal with uncertainty in input processing Speech act theory is adapted for more structured (and limited) dialogue Seminar on Spoken Dialogue Systems, Columbia

Dialogue Annotation Schemes Schemes for annotating dialogues Used for : – building statistical models of dialogue prediction of next dialogue act in a live system with a combination of features – from the current utterance – Previous utterances (dialogue history) – Observer system that analyses human dialogue Can you think of examples? Seminar on Spoken Dialogue Systems, Columbia

Dialogue Annotation Schemes Schemes for annotating dialogues Used for : – building statistical models of dialogue prediction of next dialogue act in a live system with a combination of features – from the current utterance – Previous utterances (dialogue history) – Observer system that analyses human dialogue Meeting assistant Dialogue summarization Seminar on Spoken Dialogue Systems, Columbia

DAMSL generic annotation schema Dialogue Markup in Several Layers Core & Allen 1997 Addresses the problem with single label of speech acts – Because of multi-purpose nature of utterances DAMSL uses 3 layers – Each utterance can have multiple labels Inform-accept Inform Speech Act Seminar on Spoken Dialogue Systems, Columbia

DAMSL generic annotation schema 3-layer scheme – Forward communicative functions Traditional speech act theory – Backward communicative function How current utterance relates to the previous dialog – Accept/reject a proposal – Confirm understanding – Answer a question – Utterance Features Communicative process vs subject Seminar on Spoken Dialogue Systems, Columbia

DAMSL scheme Forward Functions Backward Functions Utterance Features Seminar on Spoken Dialogue Systems, Columbia

DAMSL scheme Forward Functions Backward Functions Utterance Features Seminar on Spoken Dialogue Systems, Columbia

DAMSL example Seminar on Spoken Dialogue Systems, Columbia

Evaluation of Annotation Scheme Do annotators agree on the tags? – Multiple annotators annotate the same dialogue and compare the agreement Kappa Statistic to measure inter-annotator agreement (IAG) – PA = % agreement – PE = % expected agreement (depends on the number of possible tags).67 is considered reliably good Seminar on Spoken Dialogue Systems, Columbia

DAMSL scheme Kappa IAG Forward Functions Backward Functions Utterance Features Seminar on Spoken Dialogue Systems, Columbia

Other generic annotation schemes Carletta (1997) H. Bunt (2013) – ISO standard – Motivated by all previous annotation schemes – Multifunctional (as DAMSL) A: Henry, can you take us through these slides? H: Oh Okay, just ordering my notes – A assigns the next turn to H – A formulates indirect request – Not limited to 3 dimensions: Each utterance is labeled with – 1 general purpose function – up to 9 dimension-specific functions Seminar on Spoken Dialogue Systems, Columbia

General function of DIT annotation scheme Seminar on Spoken Dialogue Systems, Columbia

General function of DIT annotation scheme Seminar on Spoken Dialogue Systems, Columbia U: Who answered the door?

General function of DIT annotation scheme Seminar on Spoken Dialogue Systems, Columbia U: Did Tom or Mary answer the door?

Dimensions of DIT scheme Task Feedback Turn management Time management Discourse structuring Social obligation management Seminar on Spoken Dialogue Systems, Columbia

Examples of dimension-specific annotations Seminar on Spoken Dialogue Systems, Columbia

How are annotation frameworks used? Analysis of human dialogues (meetings, customer service dialogues with live agents, non-task- oriented dialogues) – Analyzing dialogue structure & building predictive models Manual annotation of DA Build automatic models to classify DA Building models for SDS applications – Annotate user responses – Build predictive models for DA Seminar on Spoken Dialogue Systems, Columbia

Comments on generic annotation schemes Generic schemes are designed to handle general (human) conversation SDS scheme are simplified for application purposes Application-specific DA schemes Seminar on Spoken Dialogue Systems, Columbia

Dialogue Management

Dialog system components Voice input Hypothesis (automatic transcription) Text Speech Language Model/Grammar Acoustic model Grammar/Models Generation templates/ rules Logical form of user ’ s input Logical form of system ’ s output Seminar on Spoken Dialogue Systems, Columbia

Dialogue Manager (DM) Is a “brain” of an SDS Decides on the next system action/dialogue contribution SDS module concerned with dialogue modeling Seminar on Spoken Dialogue Systems, Columbia

Dialogue Management Address how SDS is implemented – Reusability of components Approaches: – Frame-based approach (structural) – Information State approach (Traum & Larsson 2003) Seminar on Spoken Dialogue Systems, Columbia

Information State Approach Formalizes theories of dialogue – Speech act Combines different theoretical approaches to SDS – Planning (more flexible and complex) – Structural (simple scripted dialogues) Leads to better engineering of SDS components – Separate development of modules – Facilitates reuse of components Seminar on Spoken Dialogue Systems, Columbia

ISU Architecture Seminar on Spoken Dialogue Systems, Columbia

Information State DM Formalize DM function as Information State Update – Identify relevant aspects of information How are they updated What controls this updates Dialogue context – Current state of the system including Known information Role of DM – Update dialogue context – Provide context-dependent expectations – Interface with task/domain processing – Decide what context to express next Seminar on Spoken Dialogue Systems, Columbia

Information state Private: – Belief set – Agenda: stack of actions Shared: – Belief set – QUD: stack of questions – LM: move Seminar on Spoken Dialogue Systems, Columbia

Declarative representation of system behavior – SDS designer writes update rules Seminar on Spoken Dialogue Systems, Columbia

Discussion questions for Traum& Larsson paper Do the dialogue moves and update rules have to be tailored for each specific? The authors mention that information state- based theory of dialogue includes informational components, such as common ground, beliefs, intentions, etc. How are these concepts extracted, and then represented in this approach? Seminar on Spoken Dialogue Systems, Columbia

SDS Trends 1966 Eliza – string pattern match DM 1970 – 2000: Plan-based systems, Speech act theories 2000 – 2010: Information State Update, Frame- based SDS (Communicator DARPA project) 2010 – 2014: Using web as data source, template- based DM, virtual assistants on phones (Siri, Cortana, Google Now), in-car SDS 2015 using inference engines on the back-end ( What’s next? Seminar on Spoken Dialogue Systems, Columbia

Presentations Seminar on Spoken Dialogue Systems, Columbia **David Traum and Staffan Larsson, The Information State Approach to Dialogue Management. in Current and New Directions in Discourse and Dialogue, Ed. Jan van Kuppevelt and Ronnie Smith, Kluwer, pages , Discussion papers: Ben Hixon, Rebecca Passonneau Open Dialogue Management for Relational Databases NAACL 2013 Presenter: Jee Hyun Wang Pierre Lison. Model-based Bayesian Reinforcement Learning for Dialogue Management. In Proceedings of the 14th International Conference of the Speech Communication Association (Interspeech 2013), Lyon, France, Presenter: Edward Li

Find Project Partner 2/16 Find a project partner 2/23 Project descriptions due Considerations when finding partners: – What is your focus (system vs. research question) – Which topics – Goals to submit a paper Seminar on Spoken Dialogue Systems, Columbia