1. Alicia Abella AT&T Labs – Research Florham Park, New Jersey
Semantic Information Processing of Spoken Language - How May I Help You? sm
Alicia Abella
AT&T Labs – Research
Florham Park, New Jersey

2. Motivation
Goal is to provide automated customer services via natural spoken dialog.
Natural means what people actually say, rather than what we’d like them to.
Shift burden from user to machine

3. Why Spoken Language Understanding?
User interface as the bottleneck to exploiting speech and language processing technological advancement
Spoken language as focus of this work
Machine Initiative
Menus (please say collect, calling card …)
Classes (please say credit card number, destination city, person name, date, …)
User Initiative
How may I help you?

4. Stroustrup on programs which communicate with people
"... it must cope with that person's whims, conventions and seemingly random errors. Trying to force the person to behave in a manner more suitable for the machine is often (rightly) considered offensive.”
from "The C++ Programming Language”(1987) pp. 76

5. A History of Applications
Department Store Call-Routing ( )
Almanac Data Retrieval (1992)
Airline Travel (1993)
Multimodal Blocks World (1993)
Operator Services (1995-9)
Customer Care (2000+)
Enterprise Customers (2002+)

6. How May I Help You? SM . . . Prompt is “AT&T. How may I help you?”
User responds with unconstrained fluent speech
System recognizes and determines the meaning of users’ speech, then routes the call
Dialog technology enables task completion
HMIHY
. . .
Local
Account
Balance
Calling
Plans
Unrecognized
Number

7. Extracting Meaning from Speech
Extracting meaning is primary in speech understanding systems.
How to quantify the information content of a natural language message?
Such theory is crucial to engineering devices which understand and act upon such messages.

8. Communication Paradigm
Goal of communication is to induce the machine to
perform some action
undergo some internal transformation
Communication is successful if the machine responds appropriately
Contrast with traditional communication theory

9. Shannon (1948)
The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning, … These semantic aspects of communication are irrelevant to the engineering problem.

10. Architecture for Natural Spoken Dialog
Voice reply to customer
Speech
Speech
Text-to-Speech Synthesis
Automatic Speech Recognition
TTS
ASR
Data
Words to be
synthesized
Words
Words
Words spoken
Spoken Language
Generation
SLG
SLU
Dialogue technologies are the closest to an application
Accurate understanding can often be achieved without correctly recognizing every word
SLU makes it possible to offer services where the customer can speak naturally without learning a specific set of terms.
Provide service differentiation, revenue and cost reduction.
UI is very critical in building an application
The technology and advancements in SLU and DM are at the state where ASR was over a decade ago. SLU and DM
Has only become possible over the past few year when ASR reached real-time performance.
You do not need to recognize every word to do accurate spoken language dialogue
I remember a comment Jont Allen made once about the woman in this image. I couldn’t have agreed more with him. Can we find a woman who doesn’t look like she’s coming from the 50s? Afterall we’re talking about technology for the 21st century we should show an image that is more “hip” with the times. 
Spoken Language Understanding
Action DM
Meaning
Meaning
Dialogue Management

11. Architecture for Natural Spoken Dialog
Play
prompt
ASR
Spoken
Language
Understanding
Dialog
Manager
User
speech
Acoustic
Models
Language
Models
Salient
Grammar
Fragments
Inheritance
Hierarchy

12. Technology Component Traits
Robustness
ASR
Large vocabulary > 10,000 words
Dialects (Nationwide deployment)
Real-time
SLU
Tolerance of varied phraseology
Many ways of saying the same thing
Similar way of saying different things
Dialog
Confirmation, Re-prompting, Context switching
Say anything, anytime, anyway

13. Examples of Customer Utterances
Account Balance
Other
General Billing
Rates and Calling Plans
Charge on Bill
Change Customer Info

14. Comparative Example: Unrecognized Number
Customer Care IVR
HMIHY
Sparkle Tone
“Thank you for calling AT&T…”
Sparkle Tone
“AT&T, How may I help you?”
Network Menu
Account Verification Routine
LEC Misdirect Announcement
Each “Menu” has 4 – 6 prompts within it.
The 0300 example was created off the system working today. It is not the new 8.0 Release that will be deployed in a few weeks. (The main difference is there is an additional menu in the current flow) Additionally this calling party is classified in RAMP as a customer who has some sort of credit problem and therefore receives the prompt about it
In this HMIHY demo it is sending the caller to the DTMF version from 0300, the plan is to use the ASR version of 0300
Account Verification Routine
Main Menu
LD Sub-Menu
Reverse Directory Routine

15. Example Dialogs Rate Plan Account Balance Local Service
Unrecognized Number
Threshold Billing
Billing Credit

16. The technology is in use today
AT&T Customer Care organizations
Consumer service live since Nov. 2000
Decreased servicing costs; reduced time spent in automation; reduced repeat calls and customer defections
Small Business service pilot underway
Supports 800#s used for billing inquiries and corporate calling card transactions.
Determines caller intent to perform activities such as making payments, requesting bill adjustments, ordering cards, reporting stolen cards
AT&T Enterprise customers
Several in beta trials
Delivers this functionality in a networked, managed environment.

17. Industry Specific Applications
Check account balances, Apply for mortgage, Request credit report, Locate branch
Verify coverage, Inquire about a claim, Check claim status
Benefit enrollment, Get a referral, Obtain test results, Pre-admissions procedures
Get store hours, Locate nearest store, Directions, Check inventory availability and order status
Obtain a price quote, Make reservations, Get a seat assignment, Check flight status, Redeem miles
Get instructions, Report a problem, Obtain problem status, Order
And Applications Common to All Industries:
Password Reset, PIN Reset, FAQs,
Help Desk, Locator Services, Order Entry and Status

18. Key Value Determinants
Enhanced Customer
Experience
Reduced wait and call times
even at peak times
Natural efficient and
personalized dialog
Properly fulfills/routes
request the first time
Decrease Costs
Increased use of automation
reduces servicing costs
Liberate agent headcount
Reduce handling time and
hang-ups and call-backs
Enhanced Business
Results
Implement new applications
to drive revenue
No capital investment to
build or support
Dynamic, customizable
resource sharing in secure
environment

19. Observed Benefits Increased automation Improved customer satisfaction
Improved routing
Ability to add more functionality
Improved customer satisfaction
Decreased repeat calls (37%)
Decreased customer defection rate (18%)
Decreased rep time per call (10%)
Decreased customer complaints (78%)

20. Research References at www. research. att. com/~algor/hmihy e. g
Research References at e.g. IEEE Computer Magazine, April 2002