Semantic Parsing for Robot Commands Justin Driemeyer Jeremy Hoffman.

Slides:

Advertisements

Similar presentations

Natural Language Processing (or NLP) Reading: Chapter 1 from Jurafsky and Martin, Speech and Language Processing: An Introduction to Natural Language Processing,

Advertisements

CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)

CPSC 422, Lecture 16Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 16 Feb, 11, 2015.

Spoken Language Understanding in Dialogue Systems Svetlana Stoyanchev 02/02/2015.

Proceedings of the Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2007) Learning for Semantic Parsing Advisor: Hsin-His.

LING 388: Language and Computers Sandiway Fong Lecture 2.

15.0 Utterance Verification and Keyword/Key Phrase Spotting References: 1. “Speech Recognition and Utterance Verification Based on a Generalized Confidence.

March 1, 2009 Dr. Muhammed Al-Mulhem 1 ICS 482 Natural Language Processing Probabilistic Context Free Grammars (Chapter 14) Muhammed Al-Mulhem March 1,

Introduction to Natural Language Processing and Speech Computer Science Research Practicum Fall 2012 Andrew Rosenberg.

IDENTIFYING LANDMARKS AND RELATIONS IN GROUNDED ROUTE DESCRIPTIONS OF IITK ‘Smart Maps’ PROJECT PRESENTATION -MOHIT GUPTA & SHUBHAM MODI.

Natural Language and Speech Processing Creation of computational models of the understanding and the generation of natural language. Different fields coming.

CSE111: Great Ideas in Computer Science Dr. Carl Alphonce 219 Bell Hall Office hours: M-F 11:00-11:

1 Voice Command Generation for Teleoperated Robot Systems Authors : M. Ferre, J. Macias-Guarasa, R. Aracil, A. Barrientos Presented by M. Ferre. Universidad.

Are Linguists Dinosaurs? 1.Statistical language processors seem to be doing away with the need for linguists. –Why do we need linguists when a machine.

Some definitions Morphemes = smallest unit of meaning in a language Phrase = set of one or more words that go together (from grammar) (e.g., subject clause,

Parsing the NEGRA corpus Greg Donaker June 14, 2006.

Probabilistic Parsing Ling 571 Fei Xia Week 5: 10/25-10/27/05.

A Conversational Agent to Navigate in Virtual Worlds CHI 2000 Workshop on Natural Language Interfaces The Hague, The Netherlands Pierre Nugues, Christophe.

SI485i : NLP Set 9 Advanced PCFGs Some slides from Chris Manning.

Natural Language Understanding

Lecture 1, 7/21/2005Natural Language Processing1 CS60057 Speech &Natural Language Processing Autumn 2005 Lecture 1 21 July 2005.

ELN – Natural Language Processing Giuseppe Attardi

AQUAINT Kickoff Meeting – December 2001 Integrating Robust Semantics, Event Detection, Information Fusion, and Summarization for Multimedia Question Answering.

Richard Socher Cliff Chiung-Yu Lin Andrew Y. Ng Christopher D. Manning

Data collection and experimentation. Why should we talk about data collection? It is a central part of most, if not all, aspects of current speech technology.

Empirical Methods in Information Extraction Claire Cardie Appeared in AI Magazine, 18:4, Summarized by Seong-Bae Park.

Tree Kernels for Parsing: (Collins & Duffy, 2001) Advanced Statistical Methods in NLP Ling 572 February 28, 2012.

Some Thoughts on HPC in Natural Language Engineering Steven Bird University of Melbourne & University of Pennsylvania.

Experiments on Building Language Resources for Multi-Modal Dialogue Systems Goals identification of a methodology for adapting linguistic resources for.

+ 1. Pragmatics. - What is pragmatics? - Context 2. Speech acts. - direct speech acts - indirect speech acts.

CS 4705 Natural Language Processing Fall 2010 What is Natural Language Processing? Designing software to recognize, analyze and generate text and speech.

Creating a corpus of commands in a domotic environment (semi- automatically)?

Intro to Lexing & Parsing CS 153. Two pieces conceptually: – Recognizing syntactically valid phrases. – Extracting semantic content from the syntax. E.g.,

Natural Language Processing Rogelio Dávila Pérez Profesor – Investigador

Discriminative Models for Spoken Language Understanding Ye-Yi Wang, Alex Acero Microsoft Research, Redmond, Washington USA ICSLP 2006.

Machine Translation  Machine translation is of one of the earliest uses of AI  Two approaches:  Traditional approach using grammars, rewrite rules,

SI485i : NLP Set 14 Missing Parts and the Future.

Markov Logic and Deep Networks Pedro Domingos Dept. of Computer Science & Eng. University of Washington.

Artificial Intelligence: Natural Language

Unit 12: ＜ Consolidations ＞ By Kevin Chen.  Consolidation errors in grammar seem to be rooted in real differences between spoken and written sentences.

1 Intelligente Analyse- und Informationssysteme Frank Reichartz, Hannes Korte & Gerhard Paass Fraunhofer IAIS, Sankt Augustin, Germany Dependency Tree.

Introduction to Dialogue Systems. User Input System Output ?

What you have learned and how you can use it : Grammars and Lexicons Parts I-III.

Natural Language Processing Frank Bergman Mike Gallagher Jason Firestone Barcelona, October 19th, 2001.

DIALOG SYSTEMS FOR AUTOMOTIVE ENVIRONMENTS Presenter: Joseph Picone Inst. for Signal and Info. Processing Dept. Electrical and Computer Eng. Mississippi.

For Friday Finish chapter 24 No written homework.

CPSC 503 Computational Linguistics

LING 001 Introduction to Linguistics Spring 2010 Syntactic parsing Part-Of-Speech tagging Apr. 5 Computational linguistics.

Combining Speech Attributes for Speech Recognition Jeremy Morris November 9, 2006.

For Friday Finish chapter 23 Homework –Chapter 23, exercise 15.

Supertagging CMSC Natural Language Processing January 31, 2006.

Automatic Grammar Induction and Parsing Free Text - Eric Brill Thur. POSTECH Dept. of Computer Science 심 준 혁.

Syntax Analysis - Parsing Compiler Design Lecture (01/28/98) Computer Science Rensselaer Polytechnic.

CPSC 422, Lecture 27Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27 Nov, 16, 2015.

CS 4705 Lecture 17 Semantic Analysis: Robust Semantics.

1 An Introduction to Computational Linguistics Mohammad Bahrani.

SoarTech Proprietary Automatic Speech Recognition in Training Systems: An Introduction Presenter: Brian Stensrud, Ph.D. 21 Jan 2016 PAO Approval: 15-ORL

For Monday Read chapter 26 Homework: –Chapter 23, exercises 8 and 9.

Overview of Statistical NLP IR Group Meeting March 7, 2006.

NATURAL LANGUAGE PROCESSING

Maximum Entropy techniques for exploiting syntactic, semantic and collocational dependencies in Language Modeling Sanjeev Khudanpur, Jun Wu Center for.

Gaussian Mixture Language Models for Speech Recognition Mohamed Afify, Olivier Siohan and Ruhi Sarikaya.

By Kyle McCardle.  Issues with Natural Language  Basic Components  Syntax  The Earley Parser  Transition Network Parsers  Augmented Transition Networks.

Language Model for Machine Translation Jang, HaYoung.

Microsoft Word 2010 Prepared 2/20/11 Objectives:

PRESENTED BY: PEAR A BHUIYAN

Automatic Speech Recognition

Spoken Dialog System.

Natural Language - General

Artificial Intelligence 2004 Speech & Natural Language Processing

Presentation transcript:

Semantic Parsing for Robot Commands Justin Driemeyer Jeremy Hoffman

Purpose STAIR (STanford AI Robot) intelligent office assistant Spoken dialogue, e.g. commands to move “Robot, could you move forward two feet?” “Please go up to my office, room two-oh-four, and wait there.” “Oh, uh, wait, hey robot, stop.” Goal: robust language understanding Extract meaning of utterance from transcription Overcome unexpected words or phrasing, stuttering, errors of the speech recognition component

SPEECH RECOGNIZER Audio Input DIALOG MANAGER Text Utterance/ Command Semantics Robot action (e.g., speak, move) Dialogue System SEMANTIC PARSER

Semantic PCFG Parser Idea: PCFG parsing with semantic grammar Tag sentences with semantic labels instead of syntactic labels, e.g., “MoveWord,” “Destination,” “ExtraWord” Train our Ass’n 3 PCFG parser from a treebank of hand- labeled utterance transcriptions From a parsed sentence, can read off the important content directly Problem: no robot-command corpus (yet) Instead, automatically generate a fake treebank of tagged sentences We hand-wrote a PCFG for commands to move or control movement Also serves as “probability prior” once we get some real data

Frames and Slots In Our Grammar FrameMacroMoveMicroMoveControl SlotsDestination: holds either a room number or a named place (e.g., “front lobby”) that can be converted to a room number from a building map Direction: e.g., forward, back, left, right, around Distance: holds a number and a unit of distance (e.g., feet) CommandWord: e.g., stop, wait, cancel, go, ok, continue We focused on the three commands encoding movement.

Command Extraction Parse tree: (ROOT (S (MacroMove (MacroMoveWords (MacroMoveWord go) (MacroMoveWord to)) (Destination (NamedPlace (WordThe the) (NamedPlaceWord lab) (NamedPlaceWord room)))))) Desired Command: (MacroMove (Floor 0) (Room 100))

Command Extraction Two parse trees could be different, but encode the same command: Gold Standard: (ROOT (S (MacroMove (RobotAddress (ExtraWords (ExtraWord hey)) (RobotName (RobotNameWord stair)) (PoliteWords (PoliteWord could) (PoliteWord you))) (MacroMoveWords (MacroMoveWord follow) (MacroMoveWord me) (MacroMoveWord to)) (Destination (NamedPlace (NamedPlaceWord my) (NamedPlaceWord office)))))) Extracted: (ROOT (S (MacroMove (RobotAddress (Politeness (PoliteWords (PoliteWord hey))) (RobotName (RobotNameWord stair)) (Politeness (PoliteWords (PoliteWord could) (PoliteWord you) (PoliteWord follow) (PoliteWord me)))) (MacroMoveWords (MacroMoveWord to)) (Destination (NamedPlace (NamedPlaceWord my) (NamedPlaceWord office)))))) Both encode: Command Tree: (MacroMove (Floor 2) (Room 243))