New work-package WP5: Multimodal Processing and Interaction MUSCLE JPA3 Leaders: Petros Maragos, ICCS-NTUA Alexandros Potamianos, TSI-TUC

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France Outline of new WP5 Work in JPA3 T1. Book on Multimodal Processing and Interaction T2. Audio-Visual Speech Analysis and Recognition  T2.1 Audio-Visual Feature Extraction and Fusion  T2.2: Dynamic Models for AV-ASR, Evaluation  T2.3: Audio-Visual to Articulatory Speech Inversion T3. Multimodal Integration for MM Analysis & Recognition  T3.1: Video Analysis & Integration of Asynchronous Time-evolving Modalities  T3.2: Multimodal Saliency  T3.3: Integrated Multimedia Content Analysis T4. Interfaces to Multimedia  T4.1: Multimodal Dialogue Interfaces  T4.2: Eye-tracking Interfaces for Information Retrieval  T4.3: Mobile Interfaces T5. Coordination of research and Dissemination of results

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France Next steps & Proposed Research Orientations Merge research tasks of JPA1-JPA2/WP6+WP10  JPA3/WP5 Notify partners. The main axes of research in Multimodal Processing, Interaction, Interfaces are fundamental and still timely (multimodal information fusion for multimedia understanding). Focus on specific areas Need for even closer collaboration between the WP partners on specific tasks of common interest (jointly planned research agendas, joint papers, fellowships, joint proposals for new added-value in-depth projects). Contributions of each partner to deliverables should be more clearly specified to help people focus their research towards specific WP goals and collaborate with other groups. Further develop Synergies with other WPs. E-teams Book

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France e-Teams: Goals & Objectives E-Team 10: Audio-Visual Speech Analysis & Recognition  AV Feature Extraction and Feature Fusion  Dynamical Models for AV-ASR, Evaluation  Audio-Visual to Articulatory Speech Inversion. E-Team 11: Multimodal Processing & Multimedia Understanding  Video Analysis and Integration of Asynchronous Time-evolving Modalities  Audio-Visual Attention Modeling and Salient Event Detection  Integrated Multimedia Content Analysis E-Team 12: Multimodal Interfaces  Multimodal Recognition and Dialogue Systems  Mobile Services  Novel Interfaces (Eye-tracking)

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France BOOK Title: Multimodal Processing and Interaction: Audio, Video, Text Contents: State-of-Art Reviews of WP6 + WP10 (updated) Contributed Research Chapters: New Work Agenda: Scope and Thematic Areas discussed during Audio-Conf & Meetings Each interested participant s preliminary title + abstract Table-of-Contents of selected chapters is discussed with all participants  Publisher is contacted

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France Dissemination and Publicity Necessary to attract better research and collaboration. Website Showcase Demonstrations of prototype systems sampling MUSCLE research. Organize MUSCLE-related Special Sessions at related Conferences: e.g., ICASSP, EUSIPCO, ICME, ICSL, ICIP, ECCV, ICCV Publish Updated State-of-the-Art report. Present & Publish joint papers at conferences and journals. Book.

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France Multimodal Research and Systems are Central in MUSCLE Research Challenges & Goals

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France MMSP 2007 IEEE Workshop on Multimedia Signal Processing in 2007  co-organized by A. Potamianos (GC) and P. Maragos (TC)  Where: Chania, Crete  When: Oct 1-3, 2007 Synergy with MUSCLE  Multimedia a central research area in MUSCLE  MUSCLE sponsors four student best paper MMSP 2007 awards  Possibility of organizing a MUSCLE special session (spec. session proposal deadline March 9 th )  Discussion of hosting a Plenary MUSCLE meeting in Chania as a satellite workshop to MMSP

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France CFP: Multimedia Interaction and Communication Multimedia Interaction  Modeling of multimodal perception  Modeling of multimedia interaction  Human-human and human-computer dialog  Multimodal fusion and fission Multimedia Communication  Multimedia peer-to-peer communication  Multimedia wireless communication  QoS in multimedia communication  Cross-layer design for multimedia communication Supporting Methods and Systems  Single- and multimedia analysis  Single- and multimedia compression  Single- and multimedia security  Single- and multimedia indexing  Single- and multimedia search Emerging Applications  Distributed multimedia  Entertainment and gaming  eHealth and telemedicine Advanced Implementations  Implementations on graphics cards  Implementations on multimedia processors  Wearable multimedia implementations

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France The Venue Panorama Hotel Panorama Hotel Location: 10 km from Chania on the beach Location: 10 km from Chania on the beach Conference rooms with capacity up to 800 Conference rooms with capacity up to 800 Easy access to restaurants nearby and town of Platanias Easy access to restaurants nearby and town of Platanias Numerous alternative hotels nearby Numerous alternative hotels nearby

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France

MUSCLE WP5 MUSCLE Plenary, Dec. 2006, France MMSP 2007 Important Dates Special Sessions (contact the respective chair by)  March 9, 2007 Papers (full paper, 4 pages, to be received by)  April 6, 2007 Notification of acceptance by  June 8, 2007 Camera-ready paper submission by  July 6, Posters and CFP available!