1. National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory http://www.image.ece.ntua.gr A ROADMAP IN MULTIMEDIA KNOWLEDGE TECHNOLOGIES Prof. STEFANOS KOLLIAS stefanos@cs.ntua.gr Image, Video and Multimedia Systems Laboratory Computer Science Division National Technical University of Athens May 15, 2002

2. National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory http://www.image.ece.ntua.gr vs I. MULTIMEDIA-BASED KNOWLEDGE MANAGEMENT vs KNOWLEDGE-BASED MULTIMEDIA MANAGEMENT MPEG-7 generated progress in multimedia retrieval and management; focus on semantics is under development in MPEG-21. Knowledge Technologies should specify A/V descriptors, create hooks in Description Languages and update shared knowledge w.r.t. real information (semi or fully automatically). Automatic semantic analysis of A/V content is difficult; semantics are diversely understood and context dependent. The challenge: interweave multimedia analysis and A/V knowledge representation (integrate developments in MPEG-7/MPEG-21 and KT). Semantic Video Segmentation and Retrieval : to be combined with Evaluation in the Context of the video/video-shot description & available visual object representations (involving Tracking /Matching /Recognition).

3. National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory http://www.image.ece.ntua.gr II. A SPECIFIC FRAMEWORK: ADAPTIVE KNOWLEDGE BASED HUMAN INTERACTION Speech, Face, Gesture and Emotion Recognition are the means to understand users intentions and goals. Virtual humans/agents in virtual/mixed reality environments are the means to personify in a natural way the machine part of the interaction. Integration of multimedia knowledge is required in both analysis of the multimodal systems input and synthesis of its output: inclusion of AV cues / descriptions in knowledge use and update of knowledge for and according to analysis of multimedia inputs, users profiles and context of interaction