Spoken Dialogue Systems SIG-AI Fall 2003 By: Sachin Kamboj.

Slides:



Advertisements
Similar presentations
2017/3/25 Test Case Upgrade from “Test Case-Training Material v1.4.ppt” of Testing basics Authors: NganVK Version: 1.4 Last Update: Dec-2005.
Advertisements

Jeremy S. Bradbury, James R. Cordy, Juergen Dingel, Michel Wermelinger
Analysis of Computer Algorithms
Chapter 6 Writing a Program
© 2005 by Prentice Hall Appendix 3 Object-Oriented Analysis and Design Modern Systems Analysis and Design Fourth Edition Jeffrey A. Hoffer Joey F. George.
Chapter 13: The Systems Perspective of a DSS
Requirements Engineering Process
Relational data objects 1 Lecture 6. Relational data objects 2 Answer to last lectures activity.
Limitations of the relational model 1. 2 Overview application areas for which the relational model is inadequate - reasons drawbacks of relational DBMSs.
1 Term 2, 2004, Lecture 6, Views and SecurityMarian Ursu, Department of Computing, Goldsmiths College Views and Security 3.
So far Binary numbers Logic gates Digital circuits process data using gates – Half and full adder Data storage – Electronic memory – Magnetic memory –
Making the System Operational
Communicating over the Network
Database Design Using the REA Data Model
Vision: A strong and capable civil society, cooperating and responsive to Cambodias development challenges 1.
Chapter 11: Models of Computation
OO databases 1 Object Oriented databases. OO databases 2 Developing OODBMS - motivation motivation more and more application areas require systems that.
OOAD – Dr. A. Alghamdi Mastering Object-Oriented Analysis and Design with UML Module 3: Requirements Overview Module 3 - Requirements Overview.
1 Contract Inactivation & Replacement Fly-in Action ( Continue to Page Down/Click on each page…) Electronic Document Access (EDA)
1 University of Utah – School of Computing Computer Science 1021 "Thinking Like a Computer"
The World Wide Web. 2 The Web is an infrastructure of distributed information combined with software that uses networks as a vehicle to exchange that.
1 Evaluations in information retrieval. 2 Evaluations in information retrieval: summary The following gives an overview of approaches that are applied.
IONA Technologies Position Paper Constraints and Capabilities for Web Services
Software Requirements
11-1 Intelligent Support Systems Copied from:
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 4 Slide 1 Software processes 2.
Lecture 8: Testing, Verification and Validation
Lecture 5: Requirements Engineering
Introduction to Databases
CS101: Introduction to Computer programming
Executional Architecture
Intelligent Architectures for Electronic Commerce Part 1.5: Symbolic Reasoning Agents.
25 seconds left…...
Systems Analysis and Design in a Changing World, Fifth Edition
Stephen E. Lucas C H A P T E R McGraw-Hill © 2007 Stephen E. Lucas. All rights reserved. 12 Delivery.
Chapter 12 Analyzing Semistructured Decision Support Systems Systems Analysis and Design Kendall and Kendall Fifth Edition.
McGraw-Hill/Irwin Copyright © 2007 by The McGraw-Hill Companies, Inc. All rights reserved. Chapter 12 View Design and Integration.
Chapter 11 user support. Issues –different types of support at different times –implementation and presentation both important –all need careful design.
Chapter 11 Describing Process Specifications and Structured Decisions
Chapter 13 The Data Warehouse
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 13 Slide 1 Application architectures.
14-1 © Prentice Hall, 2004 Chapter 14: OOSAD Implementation and Operation (Adapted) Object-Oriented Systems Analysis and Design Joey F. George, Dinesh.
Introduction into Simulation Basic Simulation Modeling.
Systems Analysis and Design
Chapter 8 Improving the User Interface
Information Extraction Lecture 7 – Linear Models (Basic Machine Learning) CIS, LMU München Winter Semester Dr. Alexander Fraser, CIS.
U1, Speech in the interface:2. Dialogue Management1 Module u1: Speech in the Interface 2: Dialogue Management Jacques Terken HG room 2:40 tel. (247) 5254.
Spoken Dialogue Technology How can Jerry Springer contribute to Computer Science Research Projects?
Article Review: Spoken Dialogue Technology: Enabling the Conversational User MICHAEL F.M C TEAR University of Ulster University of Ulster This article.
 A data processing system is a combination of machines and people that for a set of inputs produces a defined set of outputs. The inputs and outputs.
Lecture 1, 7/21/2005Natural Language Processing1 CS60057 Speech &Natural Language Processing Autumn 2005 Lecture 1 21 July 2005.
9/8/20151 Natural Language Processing Lecture Notes 1.
Lecture 12: 22/6/1435 Natural language processing Lecturer/ Kawther Abas 363CS – Artificial Intelligence.
Spoken dialog for e-learning supported by domain ontologies Dario Bianchi, Monica Mordonini and Agostino Poggi Dipartimento di Ingegneria dell’Informazione.
1 Computational Linguistics Ling 200 Spring 2006.
Spoken Dialog Systems and Voice XML Lecturer: Prof. Esther Levin.
Dept. of Computer Science University of Rochester Rochester, NY By: James F. Allen, Donna K. Byron, Myroslava Dzikovska George Ferguson, Lucian Galescu,
16.0 Spoken Dialogues References: , Chapter 17 of Huang 2. “Conversational Interfaces: Advances and Challenges”, Proceedings of the IEEE,
October 2005CSA3180 NLP1 CSA3180 Natural Language Processing Introduction and Course Overview.
Introduction to Computational Linguistics
Chap#11 What is User Support?
Programming Languages and Design Lecture 3 Semantic Specifications of Programming Languages Instructor: Li Ma Department of Computer Science Texas Southern.
Natural Language Processing (NLP)
Speech Processing 1 Introduction Waldemar Skoberla phone: fax: WWW:
Powerpoint Templates Data Communication Muhammad Waseem Iqbal Lecture # 07 Spring-2016.
Speaker Recognition UNIT -6. Introduction  Speaker recognition is the process of automatically recognizing who is speaking on the basis of information.
Presentation transcript:

Spoken Dialogue Systems SIG-AI Fall 2003 By: Sachin Kamboj

Spoken Dialogue SystemsOctober 6, 2003 Slide 2 Outline Introduction to Spoken Dialogue Systems (SDS) Applications of SDS Components of SDS Classification of SDS On the basis of dialogue control On the basis of initiative On the basis of the verification strategy Dialogue Manager Components Challenges in the Design of an SDS Speech Recognition Language Understanding Dialogue Manager Response Generation Speech Synthesis Domain Specific Components

Spoken Dialogue SystemsOctober 6, 2003 Slide 3 Introduction Any computer system that interacts with a human using natural language. Computer systems with which humans interact on a turn-by-turn basis and in which spoken natural language plays an important part in the communication. [Fraser 1997] Spoken Dialogue Systems provide an interface between the user and a computer-based application that permits spoken interaction with the application in a relatively natural manner. [McTear 2002]

Spoken Dialogue SystemsOctober 6, 2003 Slide 4 Applications Automated reservation systems CU Communicator System TOOT Mercury Flight Reservation System NL interfaces ELVIS ( Voice Interactive System) MailSec Planning & Problem Solving Systems TRIPS & TRAINS Circuit-Fix-It Shop System Virtual Immersive Worlds (Steve) Automated Banking Systems (Naunce) Multimodal Information Systems (MATCH)

Spoken Dialogue SystemsOctober 6, 2003 Slide 5 Components Dialogue Manager Speech Recognizer Text-to-Speech System Text-to-Speech System Response Generator Language Understanding Language Understanding Domain Specific Components Domain Specific Components

Spoken Dialogue SystemsOctober 6, 2003 Slide 6 Speech Recognition Involves the conversion of Spoken Sounds (user utterances) to Text (a string of words) Requires knowledge of Phonetics and Phonology Basic Idea: Ŵ = argmax w P(O/W) P(W) Challenges: Variability in speech signal due to the language, speaker and channel. Handling continuous spontaneous speech. Handling large vocabularies. Providing a Speaker Independent Recognition System

Spoken Dialogue SystemsOctober 6, 2003 Slide 7 Language Understanding Converts a sequence of words into a Semantic Representation that can be used by the Dialogue Manager. Involves the use of Morphology, Syntax and Semantics. Example: I want to fly to California want(speaker, fly(_x, California)) Need robust parsing mechanisms to account for errors in speech recognition and ungrammatical utterances.

Spoken Dialogue SystemsOctober 6, 2003 Slide 8 Dialogue Manager Manages all the aspects of the dialogue. It takes a semantic representation of the users utterance, figures out how the utterance fits in the overall context and creates a semantic representation of the systems response. Performs all of the following: Interprets the user's utterance within the current context. Deal with malformed or unrecognized utterances. Create a user model. Perform grounding so that the user and the system have a common set of beliefs. Manage initiative and system responses. Handle issues of pragmatics in generation.

Spoken Dialogue SystemsOctober 6, 2003 Slide 9 Response Generation Involves constructing the message that is to be spoken to the user. Requires the making of decision regarding: What information should be included. How the information should be structured. The form of the message The choice of words The syntactic structure Current systems use simple methods such as the insertion of retrieved data into predefined slots in a template.

Spoken Dialogue SystemsOctober 6, 2003 Slide 10 Speech Generation Translates the message constructed by the response generation component into spoken form. Two approaches may be used: Prerecorded canned speech may be used with spaces to be filled by retrieved or previously recorded samples. You have fifteen new s. Text-to-speech synthesis Also known as concatenative speech synthesis. Text-to-phoneme conversion. (sp ē ch, d īə lg) Phoneme-to-speech conversion.

Spoken Dialogue SystemsOctober 6, 2003 Slide 11 Domain Specific Components The dialogue manager usually needs to interface with some external software such as a database or an expert system. The query or plans thus have to be converted from the internal representation used by the dialogue manager to the format used by the external domain specific system (e.g. SQL or STRIPS style goals). This interfacing is handled by the domain specific components.

Spoken Dialogue SystemsOctober 6, 2003 Slide 12 Classification of SDS Based on the method used to control the dialogue with the user: Finite state (or graph) based systems Frame based systems Agent based systems Type of initiative User Initiative System Initiative Mixed Initiative Type of verification Explicit Verification Implicit Verification

Spoken Dialogue SystemsOctober 6, 2003 Slide 13 Finite State Based Systems The user is taken through a dialogue consisting of a sequence of predetermined steps or stages. The dialogue flow is specified as a set of dialogue states with transitions denoting various alternative paths through the dialogue graph. Get Destination Get Destination Verify Destination Verify Destination Get Travel Day Get Travel Day System: What is your destination? User: London System: Was that London? User: Yes System: What day do you want to travel? User: Friday System: Was that Sunday? User: No System: What day do you want to travel? Verify Travel Day Verify Travel Day

Spoken Dialogue SystemsOctober 6, 2003 Slide 14 Finite State Based System (2) Advantages: Simple to construct The required vocabulary and grammar for each state can be specified in advance Results in more constrained speech recognition and language understanding. Disadvantages: Inhibits the users ability to ask questions and take initiative. Do not allow over-informative answers. Dialogues are not actually natural. Example: Nuance demo banking system.

Spoken Dialogue SystemsOctober 6, 2003 Slide 15 Frame Based System User is asked questions that enable the system to fill slots in a template in order to perform tasks. Dialogue flow is not predetermined but depends on: the contents of the users input the information that the system has to elicit. System: What is your destination? User: London System: What day do you want to travel? User: Friday System: What is your destination? User: London on Friday, October 10 around 9 in the morning. System: I have the following connection… Destination City:London Departure Day: Friday Departure Date: October 10 Departure Time: 09 am

Spoken Dialogue SystemsOctober 6, 2003 Slide 16 Frame Based Systems (2) Act like rule-based systems, taking a particular action based on the current state of affairs. Questions and other prompts that the system can ask should be listed along with conditions that have to be true for that particular question. Advantages: User can provide over-informative answers. Allows more natural dialogues. Disadvantages: Cannot handle complex dialogues. Range of applications limited to systems that elicit information from users and act on the basis of the same. Example: Philips train timetable information system

Spoken Dialogue SystemsOctober 6, 2003 Slide 17 Agent Based Systems Allow complex communication between the system, the user and the underlying application in order to solve some problem or task. Many variations depending on the application. User: Im looking for a job in the Calais area. Are there any server? System: No, there arent any employment servers for Calais. However, there is an employment server for Pas-de-Calais and an employment server for Lille. Are you interested in one of these? User: What time does the bank open? System: 9 am but they only accept job applications at noon. User: What time does the bank open? System: 9 am but the guards come around 8.

Spoken Dialogue SystemsOctober 6, 2003 Slide 18 Agent Based Systems (2) Communication is viewed as interaction between two agents, each of which is capable of reasoning about its own actions and beliefs. The dialogue model takes the preceding context into account The dialogue evolves dynamically as a sequence of related steps that build on top of each other. Advantages: Allow natural dialogue in complex domains. Disadvantage: Such agents are usually very complex. Hard to build.

Spoken Dialogue SystemsOctober 6, 2003 Slide 19 Dialogue Manager Components Dialogue Model: contains information about: Whether the system or the user should take the initiative Whether explicit or implicit confirmation should be used The kind of speech acts that needs to be generated. User Model: contain the systems beliefs about: What the user knows The user's expertise, experience and ability to understand the system's utterances. Knowledge Base: contains information about the world and the domain. Discourse Context: contains the dialogue history and current discourse. Reference Resolver: performs reference resolution and handles ellipsis. Plan Recognizer and Grounding Module: Interprets the user's utterance given the current context Reasons about the user's goals and beliefs. Domain Reasoner/Planner: generates plans to achieve the shared goals. Discourse Manager: manages the flow of information between all of the above modules.

Spoken Dialogue SystemsOctober 6, 2003 Slide 20 Challenges in the Design of an SDS Recovery from errors Understanding pragmatically ill-formed utterances Design of system prompts Reference resolution Understanding inter-sentential ellipsis Plan recognition Detection of conflicts Performing grounding And many more…

Spoken Dialogue SystemsOctober 6, 2003 Slide 21 Recovery From Errors A SDS should be able to detect errors or misunderstandings and recover from them. Errors may be of the following types: Uncertainties – speech recognition o/p has a low confidence score. Inconsistencies – utterance conflicts with domain model/prev utterances Ambiguities – more than one interpretation of a sentence Luperfoy proposes a recovery strategy based on the following four stage algorithm: Detection Diagnosis (Classification of the error) Repair plan selection Interactive plan execution

Spoken Dialogue SystemsOctober 6, 2003 Slide 22 Pragmatically Ill-formed Utterances Listeners assume their beliefs of the world match the speakers Hence, listeners interpret the utterances with respect to their beliefs However, the speakers views of the world may differ from those of the listener: As a result, the speakers utterance may be syntactically and semantically correct – yet violate the pragmatic rules. Pragmatically Ill-formed utterances are of two types: Extensional failures How many women on the UD wrestling team are CIS majors? Intensional failures Which apartments are for sale? What advanced placement courses did BOB take in high school? What is Dr. Smiths home address?

Spoken Dialogue SystemsOctober 6, 2003 Slide 23 Design of System Prompts Prompt design is important for: Natural flowing conversations To overcome shortcomings in speech recognition technology One of the most challenging aspects is implicitly letting the user know what they can say. By not knowing: Users can go beyond the functionality of the system Not utilize the system as fully as they could Prompt design is related to initiative This is AZ Banking. How may I help you? This is AZ banking. Say check balance to check your balance, pay bill to pay a bill or transfer funds to transfer funds… Prompts should be more explicit in the case of recognition errors and less explicit as the user shows greater familiarity with the system.

Spoken Dialogue SystemsOctober 6, 2003 Slide 24 Reference Resolution Reference is the process by which speakers use expressions like he and it to refer to entities salient in the discourse. Reference resolution is the process of determining the referent entity of a referring expression. For example: John went to Bills car dealership to check out an Acura Integra. He looked at it for about an hour. Before he bought it, John checked over the Integra very carefully.

Spoken Dialogue SystemsOctober 6, 2003 Slide 25 Inter-sentential Ellipsis Is the use of a syntactically incomplete sentence fragment, along with the context in which the fragment occurs, to communicate a complete thought and accomplish a speech act. Examples: I want to cash this check. Small bills only please. Speaker 1: Who are the candidates for the consultants? Speaker 2: Mary Smith, Bob Jones and Ann Doe. Speaker 1: Toms recommendations?

Spoken Dialogue SystemsOctober 6, 2003 Slide 26 References Carberry, Sandra: Plan Recognition in Natural Language Dialogue, ACL-MIT Press Series on Natural Language Processing, MIT Press, 1990.

Spoken Dialogue SystemsOctober 6, 2003 Slide 27 Questions?