Spoken Dialog Management for an Astronaut’s Procedure Assistant Presented by: Dan Bohus Collaborators: Gregory Aist, RIALIST Group.

Slides:



Advertisements
Similar presentations
Key architectural details RavenClaw: Dialog Management Using Hierarchical Task Decomposition and an Expectation Agenda Dan BohusAlex Rudnicky School of.
Advertisements

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.
Developing Spoken Dialogue Systems in the Communicator / RavenClaw Framework Sphinx Lunch Talk Carnegie Mellon University, October 2004 Presented by:Dan.
Speech Synthesis Markup Language V1.0 (SSML) W3C Recommendation on September 7, 2004 SSML is an XML application designed to control aspects of synthesized.
1 Team Talk – A Report. 2 Introduction Project done as part of , Spring ’03. Design and implementation of a spoken dialog system. Thrust of project:
STEM Volunteer Corp Designing the Orientation Guides February 15, 2014.
® Copyright 2008 Adobe Systems Incorporated. All rights reserved. ADOBE® ACCESSIBILITY Achieving Accessibility with PDF Greg Pisocky Accessibility Specialist.
CSE 308 Project Management. SWE Communication Skills Management: Run a team meeting Presentation: Present aspects of your project during its development.
Building Enterprise Applications Using Visual Studio ®.NET Enterprise Architect.
SSP Re-hosting System Development: CLBM Overview and Module Recognition SSP Team Department of ECE Stevens Institute of Technology Presented by Hongbing.
Lab/Sessional -CSE-374. SYSTEM DEVELOPMENT LIFE CYCLE.
Knowledge Acquisitioning. Definition The transfer and transformation of potential problem solving expertise from some knowledge source to a program.
Avra Software CMPUT Process Quality and Software Assessment Case Study - slide#1©P. Sorenson and Amr Kamel Assessment Plan for Assessment Plan for.
RavenClaw Yet another (please read “An improved”) dialog management architecture for task-oriented spoken dialog systems Presented by: Dan Bohus
Eleventh Edition 1 Introduction to Information Systems Essentials for the Internetworked E-Business Enterprise Irwin/McGraw-Hill Copyright © 2002, The.
Madeleine, a RavenClaw Exercise in the Medical Diagnosis Domain Dan Bohus, Alex Rudnicky MITRE Workshop on Dialog Management, Boston, October 2003.
HAS. Patterns The use of patterns is essentially the reuse of well established good ideas. A pattern is a named well understood good solution to a common.
University of Toronto Department of Computer Science © Steve Easterbrook. This presentation is available free for non-commercial use with attribution.
Modules, Hierarchy Charts, and Documentation
RavenClaw An improved dialog management architecture for task-oriented spoken dialog systems Presented by: Dan Bohus Work by: Dan Bohus,
Supplement 02CASE Tools1 Supplement 02 - Case Tools And Franchise Colleges By MANSHA NAWAZ.
Copyright Course Technology Chapter 9: Project Communications Management.
Track: Speech Technology Kishore Prahallad Assistant Professor, IIIT-Hyderabad 1Winter School, 2010, IIIT-H.
Communication. Levels of Communication 3 levels: Social,Therapeutic, Collegial – Social: interactions for the purpose of accomplishing tasks or building.
Jason Morrill NCOAUG Training Day February, 2008
VoiceXML Builder Arturo Ramirez ACS 494 Master’s Graduate Project May 04, 2001.
6Th International Conference ICAO Global Aviation Training Office
System design technique  System Development Life Cycle  Prototyping  Rapid Application Development.
Student Learning Objectives The SLO Process Student Learning Objectives Training Series Module 3 of 3.
Modularly Adaptable Rover and Integrated Control System Mars Society International Conference 2003 – Eugene, Oregon.
1 © 2005 Cisco Systems, Inc. All rights reserved. Cisco Unity Connection 1.x Documentation What you need to know.
Chapter 2 The process Process, Methods, and Tools
THE PROTOTYPING MODEL The prototyping model begins with requirements gathering. Developer and customer meet and define the overall objectives for the software.
Requirements Analysis
Conversational Applications Workshop Introduction Jim Larson.
Chapter 1: Introduction to Systems Analysis and Design
Employability skills workshop This work has been produced on behalf of the National Quality Council with funding provided through the Australian Government.
XFEL The European X-Ray Laser Project X-Ray Free-Electron Laser Matthias Clausen, DESY XFEL Refrigerator Controls – April CSS-DCT (SNL) Training.
ATLAS Demystified: A Practical Introduction Christophe Laprun, Jonathan Fiscus, John Garofolo, Sylvain Pajot National Institute of Standards and Technology.
Spoken dialog for e-learning supported by domain ontologies Dario Bianchi, Monica Mordonini and Agostino Poggi Dipartimento di Ingegneria dell’Informazione.
1 The Software Development Process  Systems analysis  Systems design  Implementation  Testing  Documentation  Evaluation  Maintenance.
 Copyright 2005 Digital Enterprise Research Institute. All rights reserved. Semantic Web services Interoperability for Geospatial decision.
Moving into Implementation SYSTEMS ANALYSIS AND DESIGN, 6 TH EDITION DENNIS, WIXOM, AND ROTH © 2015 JOHN WILEY & SONS. ALL RIGHTS RESERVED.Roberta M. Roth.
The Voice-Enabled Web: VoiceXML and Related Standards for Telephone Access to Web Applications 14 Feb Christophe Strobbe K.U.Leuven - ESAT-SCD-DocArch.
Automated Assistant for Crisis Management Reflective Agent with Distributed Adaptive Reasoning RADAR.
Senior Design Experience Semester II Project Requirements and Timeline
Spoken Dialog Systems and Voice XML Lecturer: Prof. Esther Levin.
Fisheries Oceanography Collaboration Software Donald Denbo NOAA/PMEL-UW/JISAO Presented by Nancy Soreide NOAA/PMEL AMS 2002/IIPS 10.3.
1 Geospatial and Business Intelligence Jean-Sébastien Turcotte Executive VP San Francisco - April 2007 Streamlining web mapping applications.
Software Engineering Prof. Ing. Ivo Vondrak, CSc. Dept. of Computer Science Technical University of Ostrava
Study Group 1 Advanced Distributed Learning (ADL) Course in English for Military Operations Bureau for International Language Co-ordination.
ENTERFACE 08 Project 1 “MultiParty Communication with a Tour Guide ECA” Mid-term presentation August 19th, 2008.
Intelligent Robot Architecture (1-3)  Background of research  Research objectives  By recognizing and analyzing user’s utterances and actions, an intelligent.
1 Microsoft Project Solution Offerings and the next chapter of EPM September 17th, 2003 Brendan Giles, PMP Systemgroup Management Services.
WIN-WIN GMU Team Status Report March 31, System Prototype Process to date – Contents models, flowcharts – System-wide functions – Field templates.
Amendment Proposal to Doc 9868 – PANS - Training
© 2006 Pearson Addison-Wesley. All rights reserved 2-1 Chapter 2 Principles of Programming & Software Engineering.
® Copyright 2008 Adobe Systems Incorporated. All rights reserved. ADOBE® ACCESSIBILITY PDF Accessibility – Best Practices for Authoring Pete DeVasto Greg.
August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.
Service Proforma Middleware Workshop. Notes Please complete as much of this proforma as possible – it will help make the workshop more informative & productive.
Trade Secrets of Embedded Field-Level Help Gary Etzel Tonya McKee Russ Ward.
TRIUMF HLA Development High Level Applications Perform tasks of accelerator and beam control at control- room level, directly interfacing with operators.
W3C Multimodal Interaction Activities Deborah A. Dahl August 9, 2006.
BUS OPERATOR WORKSTATION PROCUREMENT TEAM TRAINING T O ENHANCE BUS OPERATOR ERGONOMICS, HEALTH, AND SAFETY A TRAINING TEMPLATE FOR TRANSIT AGENCIES [ADD.
1 Multimedia Development Team. 2 To discuss phases of MM production team members Multimedia I.
V7 Foundation Series Vignette Education Services.
Chapter 1 Assuming the Role of the Systems Analyst.
Slide title :40-47pt Slide subtitle :26-30pt Color::white Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial HUAWEI.
Building Enterprise Applications Using Visual Studio®
Guided Research: Intelligent Contextual Task Support for Mails
Presentation transcript:

Spoken Dialog Management for an Astronaut’s Procedure Assistant Presented by: Dan Bohus Collaborators: Gregory Aist, RIALIST Group NASA Ames/RIACS Summer Student Research Program August 2002 S.D.M.A.P.A.

Dialog Management for an Intelligent Procedure Assistant2 RIALIST NASA Ames  Location: NASA Ames / Moffett Field / Mountain View, CA (CMU West)  People  Jim Hieronymous  John Dowding  Beth-Ann Hockey  Greg Aist  + 7 summer students  Projects  Personal Satellite Assistant  Mobile Robots  Intelligent Procedure Assistant

Dialog Management for an Intelligent Procedure Assistant3 Motivation  Crew members on International Space Station (ISS) have to perform a wide variety of procedural tasks:  ISS maintenance and repair  Operations (i.e. EVA Suit Checkout)  Setup and running scientific experiments  Medical Exams

Dialog Management for an Intelligent Procedure Assistant4 Typical setup  Paper documentation  2 or more astronauts involved: one reads, one executes the procedure

Dialog Management for an Intelligent Procedure Assistant5 Proposed alternative  Build a spoken dialog system for procedural task guidance and support  Create more crew-time!  Used also for ground training  Potential for increased safety

Dialog Management for an Intelligent Procedure Assistant6 Outline  Motivation  The Intelligent Procedure Assistant  A Dialog Management Component for IPA  Demo  Future work

Dialog Management for an Intelligent Procedure Assistant7 The Intelligent Procedure Assistant Multi-modal spoken dialog system providing crew support and training for the execution of procedural tasks  Guide astronauts through procedural tasks  Provide information access (diagrams, docs)  Act as a training tool  Facilitate task delegation  Provide support for voice annotations, intercom

Dialog Management for an Intelligent Procedure Assistant8 IPA Version 0: June 2002  RIALIST IPA Version 0  Load procedure  Next  Previous  Drill-down  Describe remaining steps  Increase volume  Decrease volume  Say again

Dialog Management for an Intelligent Procedure Assistant9 Outline  Motivation  The Intelligent Procedure Assistant  A Dialog Management Component for the IPA  Demo  Future work

Dialog Management for an Intelligent Procedure Assistant10 Speech Recognition Dialog Management Language Generation Synthesis U DSI SONLS Semanticizer (parsing) Resources/Agents Contributions: Dialog Management & Resources

Dialog Management for an Intelligent Procedure Assistant11 Nuance [recognition] Dialog Management Gemini [generation] Festival U DSI SONLS Gemini [parsing] Resources/Agents Nuance [recognition] Dialog Management Gemini [generation] Festival U DSI SONLS Gemini [parsing] Resources/Agents Audio Dialog Agent RavenClaw DM framework (CMU) Contributions: Dialog Management & Resources CORE/ Conversational Skills Dialog Task Specification The actual domain task (IPA domain) Handles dialog flow, initiative, focus shifts, universal dialog mechanisms

Dialog Management for an Intelligent Procedure Assistant12 1. Resources  Procedures represented in XML (DTD, XSLT)  Title, Description.  Steps: Basic Steps, Conditional Steps, Messages, Images  Messages: Notes, Cautions  Images: Sub-images  Water Sampling Procedure  28 steps, 150 sub-steps, 18 conditional steps, 1 image with 5 sub-images  HTML version (using XSLT) HTML version

Dialog Management for an Intelligent Procedure Assistant13 2. Dialog Task Development  In RavenClaw, a dialog task is implemented as a hierarchy of dialog agents (C++ classes) IPA Welcome LoadProcedureDo Procedures DoProcedure1 IntroClosing DoProcedure2 Step1 Step28Image1 SubStep1 SubStep4 Help Suspend/ Resume

Dialog Management for an Intelligent Procedure Assistant14 2. Dialog Task Development (cont)  52 dialog task agents (types) were implemented:  Loading procedures by name  Guidance  Step-by-step execution  Drill-down  Branching  Presenting notes / cautions / images  Information access  Talking on/about images, objects  Navigational behavior  History / Trace through procedure  Next, Previous step/substep  Start over

Dialog Management for an Intelligent Procedure Assistant15 3. Conversational Skills Enhancements  Timeouts  System reacts after a timeout period  User Barge-in  Requests for repeat  Requests for help  generic “help!”  “what can I say ?”  Context establishment  “where are we ?”

Dialog Management for an Intelligent Procedure Assistant16 Outline  Motivation  The Intelligent Procedure Assistant  A Dialog Management Component for IPA  Demo  Future work

Dialog Management for an Intelligent Procedure Assistant17 Demo Guidance Volume control Information Access

Dialog Management for an Intelligent Procedure Assistant18 Outline  Motivation  The Intelligent Procedure Assistant  A Dialog Management Component for IPA  Demo  Future work  IPA System (RIALIST Group)  Advances in dialog management

Dialog Management for an Intelligent Procedure Assistant19 Future work for IPA system (RIALIST Group)  IPA system functionality goals  Guide astronauts through procedural tasks  Provide information access  Act as a training tool  Facilitate task delegation  Provide support for voice annotations, intercom  Iterative improvement process  Other features, including those suggested by JSC training personnel and astronauts

Dialog Management for an Intelligent Procedure Assistant20 Timeline …  Reliable prototype by the end of September  Guidance/Information Access  First fieldable system by March 2003

Dialog Management for an Intelligent Procedure Assistant21 NASA is cool !

Dialog Management for an Intelligent Procedure Assistant22 Being back is also cool !  2 new systems to be developed based on the Communicator SDS setup / RavenClaw DM  RoomLine system  Provide conference room reservation services within SCS  Ridy Lie  BusLine system  Provide access to bus line information  Project “Let’s go!” (Maxine/Alan)?  Brian Langner, Antoine Raux  Hope you can call them sometime before Christmas !!!