1. 1 Commercial Applications of Natural Language Technology April 14, 2005 Deborah A. Dahl Principal, Conversational Technologies Chair, World Wide Web Consortium Multimodal Interaction Working Group dahl@conversational-technologies.com

2. Conversational Technologies 2 Business Motivations  save money (operators, phone costs)  improve user satisfaction  provide new revenue-generating services  do something that couldn’t otherwise be done  legal requirements

3. Conversational Technologies 3 Technical Motivations for Natural Language Processing indirection complexity of intention complexity of language I’ve been having a lot of problems with my inkjet printer the last few weeks system: When do you want to depart? user: I need to be downtown for an 8:00 meeting system: When do you want to depart? user: I need to be downtown for an 8:00 meeting travel from San Francisco to Philadelphia aisle seat vegetarian meal no more than one stop red-eye ok if arrives after 6:00 a.m. and doesn’t stop in Chicago wheelchair needed should be on one of my preferred airlines unless fare is much higher travel from San Francisco to Philadelphia aisle seat vegetarian meal no more than one stop red-eye ok if arrives after 6:00 a.m. and doesn’t stop in Chicago wheelchair needed should be on one of my preferred airlines unless fare is much higher system: Where do you want to go? user: Philadelphia system: Where do you want to go? user: Philadelphia

4. Conversational Technologies 4 Natural Language Understanding in a Spoken Dialog System Recognizer Meaning extraction Dialog manager Back-end application Generate prompt Speech generation Language model Acoustic models NLU rules DM rules NLG rules Templates TTS Recordings

5. Conversational Technologies 5 Natural Language Processing in Commercial Spoken Dialog Systems  Form-filling applications  Classification of free-form spoken inputs  Standards

6. Conversational Technologies 6 Form-filling Spoken Dialog Systems  retail banking  voice portals  access to email, voice mail  travel reservations (Amtrak Julie)  package tracking

7. Conversational Technologies 7 Multimodal Form-Filling using XHTML+Voice  IBM Chinese food demo

8. Conversational Technologies 8 Government Applications -- NASA  Clarissa -- International Space Station's new speech-powered virtual assistant  Space station checklists are very long and complex with many branches, which often require 'fill-in-the-blank' answers.  General purpose 'procedure reader’  helps astronauts check out space suits and analyze drinking water quality  Scheduled to begin working with astronauts in May as part of International Space Station Expedition 11.

9. Conversational Technologies 9 Statistical Classification Sort user’s statement into bins of predefined topics (for example, place order, find out status of order, return item)  given examples of statements that go in different bins (training data)  sort new examples into the right bins  example of applying these kinds of techniques to text – spam filters

10. Conversational Technologies 10 Example of Classification: Spam Filters  FOR YOUR ATTENTION; Dear Sir, I am pleased to write you in view of the circumstances in which I now found myself. This rescuable situation, though with its attendant mutual benefit needs urgent action hence this letter,and I do hope you will not hesitate to come to my rescue…. 90% 84% 93% 98%84%

11. Conversational Technologies 11 Some Current Statistical Classification Systems  Nuance “Say Anything”  Scansoft “SpeakFreely”  ATT “VoiceTone”  BBN “Call Director”  TuVox

12. Conversational Technologies 12 Customer Service Calls Touchtone menu is complex with many layers Prompts are confusing Customer wants just to say what they need. “I’m closing up my summer home and want to turn off the phone.” Problem: Customer Service Destination? Entry Point: BillingPayment Make arrangements Repair Cancel Service Orders Copy of Bill Unauthorized call BalancePast due notice New Service Order Status Seasonal Order Move Order Change Pay now

13. Conversational Technologies BBN Call Director™ Automated Services Sales Billing Technical Support Speech Text IVR Router Topic “Please tell me briefly the reason for your call today.” Speech Recognizer Topic Classifier Statistical Grammars & Topic Models “I’m calling to check whether there is any better rate plans than the one I currently have.”

14. Conversational Technologies 14 Standards  Extremely important for commercial applications

15. Conversational Technologies 15 W3C Natural Language Standards  Aimed at form-filling dialogs  VoiceXML – defines dialogs  Speech Recognition Grammar Specification (SRGS): describes allowable sequences of words  Semantic Interpretation (SI): describes how sequences of words are to be interpreted  Extensible MultiModal Annotation (EMMA) represents final interpretation of user’s input

16. Conversational Technologies 16 Form-filling Dialog System: Welcome to the weather information service. What state? User: help System: Please speak the state for which you want the weather User: Pennsylvania System: Please speak the city for which you want the weather. User: Philadelphia

17. Conversational Technologies 17 VoiceXML Example Welcome to the weather information service. What state? Please speak the state for which you want the weather. What city? Please speak the city for which you want the weather.

18. Conversational Technologies 18 SRGS Examples  Context-free grammar  XML and ABNF formats are provided yes yeah uh huh <rule id=“yes-no” Other Features  optionality  language declaration  weighted alternatives  pronunciations  special rules  external rules  character encoding

19. Conversational Technologies 19 SRGS Specification  http://www.w3.org/TR/speech-grammar/  Status: W3C Candidate Recommendation  Quick Guide to the SRGS Specification  http://www.conversational- technologies.com/pages/5/index.htm

20. Conversational Technologies 20 Semantic Interpretation  Tags are added to the grammar to describe the semantics of the user’s input  Format uses ECMAScript compact profile (ECMAScript 327)

21. Conversational Technologies 21 Semantic Interpretation Example Three large pizzas with onions $.pizzasize=$foodsize; $.number=$number pizzas with $.topping=$tops Result: pizza.number = 3 pizza.pizzasize= “large” pizza.toppings = [“onions”] XML Result: 3 large onions

22. Conversational Technologies 22 Semantic Interpretation Specification  http://www.w3.org/TR/semantic- interpretation/ http://www.w3.org/TR/semantic- interpretation/ http://www.w3.org/TR/semantic- interpretation/  Status: W3C Working Draft

23. Conversational Technologies 23 EMMA  Developed by the W3C Multimodal Interaction Working Group  An XML-based approach to representing natural language meanings  Applicable to multimodal applications, but originally developed for speech

24. Conversational Technologies 24 EMMA  Represents user input  Vehicle for transmitting user’s intention throughout application  Focus on language input (text, handwriting, speech)  Three components  data model  interpretation  annotation (main focus of standard)

25. Conversational Technologies 25 Interpretation Example  I want to go from Denver to Pittsburgh Denver Pittsburgh Denver Pittsburgh

26. Conversational Technologies 26 <emma:emma emma:version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma#" EMMA Example <emma:absolute-timestamp emma:start="2003-03-26T0:00:00.15" emma:end="2003-03-26T0:00:00.2"/> Boston Denver 03112003 “I want to go from Boston to Denver on March 11, 2003”

27. Conversational Technologies 27 EMMA Specification http://www.w3.org/TR/emma Status: W3C Working Draft

28. Conversational Technologies 28 Natural Language Understanding W3C Standards Summary  VoiceXML: define spoken dialogs  SRGS: describes allowable sequences of words  Semantic Interpretation: describes what intentions are represented by sequences of words  EMMA: represents an interpretation of user’s input

29. Conversational Technologies 29 Summary of Deployed Spoken Dialog Systems  Form filling applications are by far the most common  Statistical classification systems are becoming more common and are popular with users  Standards are accelerating commercial adoption of technology

30. Conversational Technologies 30 Resources  Practical Spoken Dialog Systems, Springer, 2005. (D. Dahl, editor)  VB website http://www.w3.org/Voice/ http://www.w3.org/Voice/  VoiceXML  SRGS  SISR  MMI website http://www.w3.org/2002/mmi/ http://www.w3.org/2002/mmi/  EMMA  BeVocal website http://cafe.bevocal.com/ http://cafe.bevocal.com/  VoiceXML deployments (some with phone numbers you can try http://www.kenrehor.com/voicexml/#deployments) http://www.kenrehor.com/voicexml/#deployments  Guide to speech standards -- http://www.speechtechmag.com/issues/9_8/cover/11619-1.html http://www.speechtechmag.com/issues/9_8/cover/11619-1.html