Recent Advances in Speech Translation Systems ESSLLI-2002 Tutorial Course August 12-16, 2002 Course Organizers: Alon Lavie – Carnegie Mellon University Lori Levin – Carnegie Mellon University Fabio Pianesi – ITC-irst

August 12, 2002ESSLLI-02 Speech Translation Tutorial 2 Course Objectives and Format Present the NESPOLE! System and Research Project as a “Case Study” for state-of-the-art speech translation systems: –Survey the underlying language technology involved, design considerations: components and architecture –Challenges, capabilities and limitations –Tasks and methodologies involved –Lessons learned Format: –Each day devoted to a different “theme” aspect(s) –Presentations by topic experts among senior researchers working on the NESPOLE! Project

August 12, 2002ESSLLI-02 Speech Translation Tutorial 3 Course Outline and Schedule Monday, 12 August: –Introduction and System Overview (Lavie and Pianesi) –Nespole! System Architecture (Lavie) –Data Collection in Nespole! (Costantini) Tuesday, 13 August: –Interchange Format (Levin) Wednesday, 14 August: Speech Recognition Challenges and Solutions: –ASR and Scalability (Vaufreydaz) –ASR and Robustness (Metze) –Multilinguality in Automatic Speech Recognition Systems (Schultz)

August 12, 2002ESSLLI-02 Speech Translation Tutorial 4 Course Outline and Schedule Thursday, 15 August: Analysis and Generation Approaches: –Trainable Analysis Approach (Lavie) –French Analysis and Generation Approaches (Blanchon) –Italian Generation (Pianta) Friday, 16 August: –Experimenting with Direct Approaches: Statistical Machine Translation (Vogel) –Evaluation in Nespole! (Lavie, Levin, Costantini) –Conclusion and Future Directions (Pianesi and Lavie)

August 12, 2002ESSLLI-02 Speech Translation Tutorial 5 Introduction Evolution of Speech Translation Systems

August 12, 2002ESSLLI-02 Speech Translation Tutorial 6 NESPOLE! System Overview Human-to-human spoken language translation for e-commerce application (e.g. travel & tourism) (Lavie et al., 2002) English, German, Italian, and French Translation via interlingua Translation servers for each language exchange interlingua to perform translation –Speech recognition (Speech  Text) –Analysis (Text  Interlingua) –Generation (Interlingua  Text) –Synthesis (Text  Speech)

August 12, 2002ESSLLI-02 Speech Translation Tutorial 7 Interchange Format Interchange Format (IF) is a shallow semantic interlingua for task-oriented domains Utterances represented as sequences of semantic dialog units (SDUs) IF representation consists of four parts –Speaker –Speech Act –Concepts –Arguments speaker : speech act +concept* +arguments* } Domain Action

August 12, 2002ESSLLI-02 Speech Translation Tutorial 8 Example “Hello. I would like to take a vacation in Val di Fiemme.” hello i would like to take a vacation in val di fiemme c:greeting (greeting=hello) c:give-information+disposition+trip (disposition=(who=i, desire), visit-spec=(identifiability=no, vacation), location=(place-name=val_di_fiemme)) ENG: Hello! I want to travel for a vacation at Val di Fiemme. ITA: Salve. Io vorrei una vacanza in Val di Fiemme.