Outline Grammar-based speech recognition Statistical language model-based recognition Speech Synthesis Dialog Management Natural Language Processing © 2013 by Larson Technical Services1

2 Natural Language Processing— A Moving Target Command and control System-directed dialogs Continuous speech recognition “How may I help you” and Statistical Language Models Dialogs with automatic error Mixed-initiative dialogs Multimodal dialogs Talking avatar Conversional dialogs Intelligent virtual agents—SIRI Advanced techniques

Dimensions of Natural Interaction Interaction style Semantic Interpretation Knowledge Reasoning Planning Language complexity Modalities Modality synchronization

Interaction Style System-directed – Menu or form fill-in Mixed initiative – Combination of system- directed and user- directed User-directed – Command and control Do something – Web searches Find something – Question and answer systems Who?, What?, When?, Where?, How? © 2013 by Larson Technical Services4

Semantic Interpretation Example grammar rule with Script Syntax: small out.size = "small"; medium out.size = "medium"; large out.size = " large"; green out.color = "green"; blue out.color = "blue"; white out.color = "white"; ECMAScript structure: action: { size: "large" color: "white" } © 2013 by Larson Technical Services5 Large white

Knowledge about the Device Battery status API – Retrieve information about the battery status of a (mobile) device (from HTML) – Geolocation API – Get current geo location (longitude, latitude, altitude) from HTML – Independent from location provider (GPS, WiFi, Cell-Id,...) – Orientation API – Get current device orientation (e.g., tilt) from HTML – – iPhone example: “Move the ball“ © 2013 by Larson Technical Services6

Knowledge about the User Biographical – Name, age, gender History – Visited websites – Recent purchases – Recent interactions © 2013 by Larson Technical Services7

Knowledge about the Domain of Discourse Web Ontology Language (OWL) Resource Description Framework (RDF) © 2013 by Larson Technical Services8 Tree MapleWhite Pine EvergreenRootsDeciduousTrunk IsA PartOf

Reasoning Grocery stores sell breakfast cereals Corn flakes are a breakfast cereal Grocery stores sell corn flakes © 2013 by Larson Technical Services9

Planning Convert a large task to a series of smaller tasks – Example origin: PDX; destination: NYC; origin: PDX; destination: MSP; origin: MSP; destination: NYC; © 2013 by Larson Technical Services10

Natural Language Processing “Natural Language Processing” means different things to different people Applies may artificial intelligence techniques to dialog management When it works, it works well, when it fails, it fails badly It’s like the wild west: from anarchy and confusion good systems will arise © 2013 by Larson Technical Services11