Course Overview Lecture 1 Spoken Language Processing Prof. Andrew Rosenberg

Spoken Language Processing How do computers interact with speech? Computational approaches help us understand language and spoken communication. Symbolic and Direct Modeling of Prosody 1

Speech Recognition Voice Dialing Voice mail transcription Closed Captioning Interactive Voice Response / Spoken Dialog Systems Keyword Spotting Continuous Speech Recognition Domain Specific vs. Open Domain Symbolic and Direct Modeling of Prosody 2

Speech Synthesis Navigation Systems –Garmin –Google Maps IBM Watson Bank by phone Spoken Dialog Systems Screen Readers Symbolic and Direct Modeling of Prosody 3

How much information is in speech? Words (Lexical Content) –Syntax –Semantics –Pragmatics Speaker Identity –Gender, Personality Speaker State Discourse Acts Symbolic and Direct Modeling of Prosody 4

Other applications Video retrieval “Rich Transcription” Speech Segmentation Emotion Analysis Speech-to-speech translation Intelligence Applications –Deception –Trust –Language & dialect Identification Symbolic and Direct Modeling of Prosody 5

Broader Scientific Questions How do you produce sounds that other perceive as language? How does a hearer decode what you are trying to express? How and why do you use prosodic variation? –Phrasing –Emphasis –Intonational Contours –Emotion, sarcasm, etc. How does smooth turn taking happen? Symbolic and Direct Modeling of Prosody 6

What will be covered in this course. Project driven course. Spoken Dialog System –Recognize Speech CMU Sphinx –Make a decision –Generate Speech Festival Symbolic and Direct Modeling of Prosody 7

How the course is structured Speech Recognition Speech Synthesis Analysis of additional information from speech –Speaker ID –Prosody/Intonation –etc. Symbolic and Direct Modeling of Prosody 8

Project and Exams Build a Spoken Dialog System –4 Deadlines Project Description (and team membership) Speech Recognition Component Speech Synthesis Component Full system with demo. (Start on this early) –Project writeup. In class midterm. Symbolic and Direct Modeling of Prosody 9

The syllabus and course policies course webpage: – s.htmlhttp://eniac.cs.qc.cuny.edu/andrew/slp/syllabu s.html Symbolic and Direct Modeling of Prosody 10

Questions? Policies Course mechanics Expectations Symbolic and Direct Modeling of Prosody 11