1. 1 August 9, 20061 David Claiborn SLM Tuning: Lessons Learned

2. August 9, 20062  Define Statistical Language Model (SLM)  Advantages and disadvantages of SLM technology  How an SLM is used at Sprint/Nextel  Practical SLM Tuning considerations  Is an SLM right for your speech project?  Questions Agenda:

3. August 9, 20063 What is the Statistical Language Model? Definition of an SLM: A statistical language model is a probabilistic description of the constraints on word order found in a given language.(Bahl et al 1983)  For our purposes think of an SLM as the probability of utterances occurring in a particular dialog state. This probability is created from caller utterances captured in that specific dialog state.  Traditionally, SLM technology is employed to give callers the ability to make requests using natural or conversational speech. SLM dialog states are often referred to as “Say Anything” states.

4. August 9, 20064 Advantages and Disadvantages of an SLM over a finite state grammar Advantages:  Flexibility to callers  Able to serve natural speech requests  Minimized need for guidance from prompting Disadvantages:  Difficult to train and update  Transcription must be even more precise  Cost  Time

5. August 9, 20065 SLM Application at Sprint/Nextel  At Sprint/Nextel the SLM is literally the front door into the IVR.  In the diagram below we can see the SLM offers unique treatment to seven different “phone” centered requests. SLM/ SayAnything MainMenu General Support Agent Handset Support App Handset Sales Agent Lost Phone Accessories App Broken Phone Agent Technical Issues App “My phone won’t make calls.” “Phone” “I have a question about my phone.” “I want to buy a new phone.” “I lost my phone.”“I want to buy a phone charger.” “My phone is broken.”

6. August 9, 20066 Things to consider when Tuning an SLM:  Does the SLM need a new destination or training to fulfill design requirements?  When training the SLM, what is a statistically relevant number of utterances to train on?  Do I have the expertise to tune this Say Anything state in house?  Do I have quality transcription in place? Have they guaranteed to maintain a certain level of accuracy (above 98%)?  Have I established a baseline to judge post tuning improvement?

7. August 9, 20067 Is an SLM right for your speech project?  How many applications does the Customer Care IVR have today and what additional apps do you hope to add in the next five years.  How many callers enter the Customer Care IVR in a give year, what are the high and low months and are there certain months or times of each month where certain requests increase?  What level of call routing granularity are you looking to accomplish?  How rapidly will this system need to be taking calls?  What are your goals; increased CSAT and Call Completions, decreased agent to agent transfers? At these were the initial questions IBM Global Services asked Sprint which led to the creation of Sprint’s SLM:

8. August 9, 20068 Questions?

9. August 9, 20069 Bibliography: Bahl, L.R., Jelinek, F. & Mercer, R.L. (1983) "A Maximum Likelihood Approach to Continuous Speech Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, 5 (2), pp 179-190

10. August 9, 200610 David Claiborn VUI Designer and Tuner claiborn@us.ibm.com