1. ACL, ECCAI and the Verbmobil/SmartKom Consortia German Research Center for Artificial Intelligence Stuhlsatzenhausweg 3, Geb. 43.1 66123 Saarbrücken Tel.: (0681) 302-5252/4162 Fax: (0681) 302-5341 E-mail: wahlster@dfki.de WWW: http://www.dfki.de/~wahlster Wolfgang Wahlster ACL Workshop, Hong Kong 7 October 2000

2. Hosted by DFKI (Thierry Declerck) http://registry.dfki.de 130 links to software systems (shareware, freeware, open source, trial versions) all registered software packages are classified software from academic and industrial groups Classification according to NSF/EU „State of the Art in Language Technology“ Cooperation with ELRA, LDC, ELSNET, CLASS, TELRI, BAS The International NL Software Registry of ACL

3. Russia RAAI Sweden SAIS Switzerland SGAICO Slovenia SLAIS Slovak Republic SSKI SAV Spain AEPIA Catalonia ACIA ECCAI MEMBERS

4. Founded in 1982, currently 26 member societies Represents more than 6000 AI researchers The world‘s largest AI society Web site: WWW.ECCAI.ORG Every two years: ECAI conference and ACAI Summer school ETAI electronic journal, neo-classical reviewing Hardcopy distributed by Royal Swedish Academy of Sciences ECCAI: European Coordination Committee on Artificial Intelligence

5. UNIVERSITÄT DES SAARLANDES RUHR-UNIVERSITÄT BOCHUM Phase 2 UNIVERSITÄT HAMBURG UNIVERSITÄT KARLSRUHE UNIVERSITÄT BIELEFELD TECHNISCHE UNIVERSITÄT MÜNCHEN FRIEDRICH- ALEXANDER- UNIVERSITÄT ERLANGEN-NÜRNBERG UNIVERSITÄT STUTTGART RHEINISCHE FRIEDRICH WILHELMS-UNIVERSITÄT BONN LUDWIG MAXIMILIANS UNIVERSITÄT MÜNCHEN TU-BRAUNSCHWEIG EBERHARDT-KARLS UNIVERSITÄT TÜBINGEN  W. Wahlster, DFKI The Verbmobil Consortium

6. Multilingual and Mobile Communication Assistants Multimodal Interfaces SmartKom Speech-based Web Access to Multilingual Web pages WAP Phones WebTV Multilingual Audio Retrieval and Audio Mining Discussions Lecture Notes Organizers Multilingual Indexing and Annotation of Videos Video Archives News Archives Call Centers ECommerce Mobile Travel Assistance Telephone Translations Verbmobil Dialog Translation International Research Trends in Multilingual Systems Multilingual Language Technology S peech Recognition, Language Understanding, Language Generation, and Speech Synthesis Multilingual Language Technology S peech Recognition, Language Understanding, Language Generation, and Speech Synthesis Spontaneous Speech, Robust Processing and Translation, Semantic and Pragmatic Understanding

7. Verbmobil‘s Massive Data Collection Effort Transliteration Variant 1 Transliteration Variant 2 Lexical Orthography Canonical Pronounciation Manual Phonological Segmentation Automatic Phonological Segmentation Word Segmentation Prosodic Segmentation Dialog Acts Noises Superimposed Speech Syntactic Category Word Category Syntactic Function Prosodic Boundaries The so-called Partitur (German word for musical score) orchestrates fifteen strata of annotations 3,200 dialogs (182 hours) with 1,658 speakers 79,562 turns distributed on 56 CDs, 21.5 GB

8. Treebank: 85,000 Trees 79,562 turns 1,520,000 words 76,210 dialog acts Collection Sites: All over Germany American English: CMU Japanese: ATR Mandarine: Philips Taiwan The Verbmobil Corpora

9. Machine Learning for the Integration of Statistical Properties into Symbolic Models for Speech Recognition, Parsing, Dialog Processing, Translation Transcribed Speech Data Segmented Speech with Prosodic Labels Annotated Dialogs with Dialog Acts Treebanks & Predicate- Argument Structures Aligned Bilingual Corpora Hidden Markov Models Neural Nets, Multilayered Perceptrons Probabilistic Automata Probabilistic Grammars Probabilistic Transfer Rules Extracting Statistical Properties from Large Corpora

10. MediaInterface European Media Lab Uinv. Of Munich Univ. of Stuttgart Saarbrücken Aachen Dresden Berkeley Stuttgart MunichUniv. of Erlangen Heidelberg Main Contractor Project Management Testbed Software Integration DFKI Saarbrücken The SmartKom Consortium: Project Budget: $ 34 M Project Duration: 4 years D AIMLER C HRYSLER Ulm SmartKom: Intuitive Multimodal Interaction  W. Wahlster, DFKI

11. User(s) Media Analysis Design Media Fusion Output Rendering Representation and Inference User Model Discourse Model Domain Model Task Model Media Models Interaction Management Media Analysis Input Processing Information Applications People Intention Recognition Media Design Application Interface Discourse Modeling User Modelin g Presentation Design Language Graphics Gesture Biometrics Language Graphics Gesture Animated Presentation Agent The Architecture of the SmartKom Agent (cf. Maybury/Wahlster 1998)  W. Wahlster, DFKI

12. SmartKom-Home/Office: A Versatile Agent-based Interface SmartKom-Public: A Multimodal Communication Booth SmartKom-Mobile: A Handheld Communication Assistant Media Analysis Kernel of SmartKom Interface Agent Interaction Management Application Manage- ment Media Design SmartKom: A Transportable and Transmutable Interface Agent  W. Wahlster, DFKI

13. Smartcard/ Credit Card for authentication and billing Docking station for PDA/Notebook/ Camcorder high speed and broad bandwidth Internet connectivity High-resolution scanner Loudspeaker Room microphone Face-tracking camera Virtual touchscreen protected against vandalism Multipoint video conferencing SmartKom-Public: A Multimodal Communication Booth  W. Wahlster, DFKI

14. SmartKom Domains: EPG and Cinema Selection Multimodal Selection Of Cinema Multimodal Selection of Seats

15. Camera GPS Microphone Loudspeaker Stylus-Activated Sketch Pad Wearable Compute Server Docking Station for Car PC Biosensor for Authentication & Emotional Feedback GSM for Telephone, Fax, Internet Connectivity SmartKom-Mobile: A Handheld Communication Assistant  W. Wahlster, DFKI

16. SmartKom‘s Data Collection of Multimodal Dialogs User Side-view Camera Face-tacking Camera with Microphone Environmental Noise Microphone Array Screen Projected Webpage Face-tacking Camera Loudspeaker Microphone Array User Bird’s-eye Camera LCD Beamer SIVIT- Camera

17. BAS in Munich (Prof. Tillmann, Dr. Schiel) 68 GB of transliterated speech data 590 hours of speech, 2500 different speakers works for Verbmobil and SmartKom consortium treebanks, gesture and dialog act annotations very large corpora of GSM phone dialogs multi-channel recordings of spontaneous dialogs WOZ data one year exclusive use for German labs and industry after one year distribution by ELRA and then by LDC The German Center for Language Resources