March 31, 1998NSF IDM 98, Group F1 Group F Multi-modal Issues, Systems and Applications

March 31, 1998NSF IDM 98, Group F2 Mission Statement Our mission is to develop the technology necessary to support fully-integrated multi-media database systems including many different types of data, such as text, images, signals, presentations, videos, audio clips, dynamic data, scientific data, and software.

March 31, 1998NSF IDM 98, Group F3 The new technology will include  New data representations and storage schemes  To encode data in ways that facilitate retrieval and other operations  Full integration (fusion) of multimedia information  Combining different types of information from different sources  New types of queries, new definitions of “matching”  New ways of organizing, indexing, and retrieving data  New multimodal interfaces that allow users to interact with diverse data  Constructing multimedia documents  Data acquisition tools and systems  Mining of multimedia data

March 31, 1998NSF IDM 98, Group F4 Research Directions: Most Important What are the most important problems to solve? Short term? Long term?  Features  Efficient algorithms and tools for manipulation and retrieval of compressed data.  New features and new extraction techniques for complex or nontraditional data types such as volumetric images, dynamic sequences, hyperspectral images.  Feature extraction in transformed domains.  Pose-invariant features. Data analysis and Modeling  Content analysis of multimedia data Feature extraction Pattern recognition Spatio-temporal analysis Multi-level analysis  (Collaborative work with related disciplines) - Content modeling

March 31, 1998NSF IDM 98, Group F5 Research Directions: Most Important (cont.)  Search and indexing  For multimedia documents  With multimedia queries*  For high-dimensional data  For nontraditional data  By similarity measures at multiple levels * Include text (NLP), images and signals (IP, SP), videos (VP), audio (SP), scientific data (visualization), source code. Methods of evaluation  Benchmarks  Quality metrics Query formulation  What kinds of queries  What kinds of user interfaces

March 31, 1998NSF IDM 98, Group F6 Research Directions: Most Important (cont.)  Impact  Use of multimedia data in:  Interfaces for the handicapped  Improving k-12 education (the digital earth, the solar system, art…)  Distance learning  Electronic commerce  Multimedia patient records (he electronic patient)  Needs:  What research facilities are needed? (i.e. repositories, computing facilities, major infrastructure)  Standard test databases, ground truth  Images, outdoor scenes, video clips,indoor scenes, medical, remote sensing, faces.

March 31, 1998NSF IDM 98, Group F7 Research Directions: Midterm Plans  Nontraditional Data types  3D (volumetric data)  VR data  Animations  Interoperability  Establish a common language for content-based search.  Self-creating, self-maintaining databases  Accepts data of different multi-media types  Creates structures based on metaknowledge  Learns from sequences of user interactions

March 31, 1998NSF IDM 98, Group F8 Research Directions: Not Our Emphasis  System concerns  Efficient storage of multimedia data  Optimization of multimedia queries  Time/quality tradeoffs for rapid access