"Dude, Where's My... Signals and Systems Textbook?" Joseph Picone Inst. for Signal and Info. Processing Dept. Electrical and Computer Eng. Mississippi State University Contact Information: Box 9571 Mississippi State University Mississippi State, Mississippi Tel: Fax: IEEE: MSU STUDENT BRANCH URL:

INTRODUCTION ABSTRACT A funny thing happened on the way to an IEEE section talk at IIT in I found a career. I'll relate the rest of this story, including a couple of nice demos during this talk (hint: her name is Julie). I'll review some of the early history of digital signal processing and describe some of the speech applications that transformed this field into one of the single largest semiconductor markets today. I will show how signal processing research elegantly combines mathematics, software, and computer hardware to produce innovative technology such as cell phones and computers enhanced with speech recognition. Oh yeah, and I'll try to explain why we choose to torture you with a class called Signals and Systems. Joseph Picone is currently a Professor in the Department of Electrical and Computer Engineering at Mississippi State University, where he also directs the Institute for Signal and Information Processing. He has previously been employed by Texas Instruments and AT&T Bell Laboratories. Dr. Picone received his Ph.D. in Electrical Engineering from Illinois Institute of Technology in He is a Senior Member of the IEEE and a registered Professional Engineer.

CASE STUDIES Introduced at the Summer Consumer Electronics Show in Chicago First commercial speech synthesis consumer toy Based on linear prediction Contained a proprietary speech synthesis chip SPEAK & SPELL™ (JUNE 1978) Left to right: Gene Frantz, Richard Wiggins, Paul Breedlove and Larry Brantingham (1978)

CASE STUDIES Yes / no / true / false recognizer Answer questions about history Variety of learning modules Speaker independent recognition Microphone + children??? Won several industry design awards for the mechanical design VOYAGER™ (JUNE 1988)

CASE STUDIES Worlds of Wonder approach TI in September of Can you put this toy on the market by Thanksgiving? 10-word speaker dependent isolated word recognizer 100 sentences for synthesis “Transparent training” First large-scale consumer toy application for a DSP JULIE (DECEMBER 1988)

CASE STUDIES Voice verification for calling card security First wide-spread deployment of recognition technology in the telephone network Stimulated interest in voice dialing and other user-programmable features Original application was obsolete before wide- scale deployment WATSON (EARLY 1990’S)

CASE STUDIES Jack Deller said: model voice with a digital all-pole filter identify a person by their filter coefficients diagnose a disease by shifts in these coefficients recognize the words being spoken using pattern matching techniques maximum entropy, linear prediction, and a few other big words I didn’t understand... and I was hooked... WHY ATTEND IEEE TALKS?

CASE STUDIES Deller said you can replace this: DIGITAL FILTERS With this: What are the advantages?

HUMAN LANGUAGE TECHNOLOGY SPEECH RECOGNITION RESEARCH? Why do we work on speech recognition? “Language is the preeminent trait of the human species.” “I never met someone who wasn’t interested in language.” “I decided to work on language because it seemed to be the hardest problem to solve.” Why should we work on speech recognition? Antiterrorism, homeland security, military applications Telecommunications, mobile communications Education, learning tools, educational toys, enrichment Computing, intelligent systems, machine learning Commodity or liability? Fragile technology that is error prone

HUMAN LANGUAGE TECHNOLOGY FUNDAMENTAL CHALLENGES

HUMAN LANGUAGE TECHNOLOGY STATISTICAL APPROACHES

SPEECH RECOGNITION BLOCK DIAGRAM OVERVIEW Core components: transduction feature extraction acoustic modeling (hidden Markov models) language modeling (statistical N-grams) search (Viterbi beam) knowledge sources

SPEECH RECOGNITION FEATURE EXTRACTION

SPEECH RECOGNITION ACOUSTIC MODELING

SPEECH RECOGNITION LANGUAGE MODELING

SPEECH RECOGNITION VITERBI BEAM SEARCH breadth-first time synchronous beam pruning supervision word prediction natural language

Traditional Output: best word sequence time alignment of information Other Outputs: word graphs N-best sentences confidence measures metadata such as speaker identity, accent, and prosody SPEECH RECOGNITION APPLICATION OF INFORMATION RETRIEVAL

APPLICATIONS COMMON EVALUATIONS FUEL RESEARCH Only elite participate ISIP is competitive for same level of technology More advanced systems use more heuristics and incremental technology Common evaluations: –common data/task –WER metric Performance improves over time (annually) Resource intensive

TECHNOLOGY Speech recognition –State of the art –Continuous speech –Large vocabulary –Speaker independent Goal: Accelerate research –Flexibility, Extensibility –Efficient (C++) –Easy to Use –Toolkits, GUIs Benefit: Technology –Standard benchmarks –Conversational speech PUBLIC DOMAIN SOFTWARE

APPLICATIONS INFORMATION RETRIEVAL Metadata extraction from conversational speech Automatic gisting and intelligence gathering Speech to text is the core technology challenge Machines vs. humans Real-time audio indexing Time-varying channel Dynamic language model Multilingual and cross-lingual

APPLICATIONS In-vehicle dialog systems improve information access. Advanced user interfaces enhance workforce training and increase manufacturing efficiency. Noise robustness in both environments to improve recognition performance Advanced statistical models and machine learning technology Multidisciplinary team (IE, ECE, CS). CAVS: DIALOG SYSTEMS FOR THE CAR

SUMMARY THREE THINGS TO THINK ABOUT Speech Recognition: Application of machine learning technology Requires multidisciplinary background A grand challenge (long-term problem) Career Development: Broaden your horizons! Position yourself at the upper end of the job market (e.g. graduate school) Signals and Systems: Introduces you to a brave new world Abstraction is essential in engineering Competency in math and physics is critical