MS I Multimedia Databases Arif Ghafoor Professor Electrical and Computer Engineering Purdue University

MS I 2 Multimedia Database Laboratory Facilities at Purdue University n Distributed Multimedia Systems Lab (ECE) Ghafoor, Kashyap, KhanGhafoor, Kashyap, Khan n Multimedia Systems Lab (CS) Aref, Elmagarmid, PrabhakarAref, Elmagarmid, Prabhakar n Raid Lab (CS) BhargavaBhargava n Video and Image Processing Lab (ECE) DelpDelp

MS I 3 On-Going Research Projects in Multimedia Databases n Multimedia database management (NSF, Fuji Electric, AT&T) Video modeling and managementVideo modeling and management Multimedia document managementMultimedia document management n Distributed multimedia systems (NSF, AFRL, IBM, Intel, Siemens) n High-performance multimedia database architecture for storage management (NSF, AT&T)

MS I 4 Example of Event-Based Video Data (Touch-down Scenario)

MS I 5 Research Challenges for Video Databases n Development of video processing techniques for automatic object identification and motion tracking n Development of data models with powerful semantic expressiveness n Designing efficient indexing, classification and event-based searching techniques with high degree of precision n Providing suitable video querying environment n Addressing scalability

MS I 6 Video Parsing and Segmentation Object Recognition Granularity Level of Abstraction Iconic-based Grouping and Browsing Motion Detection Analysis Knowledge-based Higher-level Semantics FineCoarse Low High Semantic Video Data Modeling Complexity

MS I 7 Raw Video Database Raw Image Database Physical Object Database Frame Object Description Image Features Spatial-Semantics of Objects (human,building,…) Sequence of Frames (indexed) Object Identification and Tracking Intra/Inter-Frame Analysis (Motion Analysis) Inter-Object Movement (Analysis) Semantic Association (President, Capitol,...) Spatio-Temporal Semantics: Formal Specification of Event/Activity/Episode for Content-Based Retrieval Spatial Abstraction Temporal Abstraction Object Definitions (Events/Concepts) An Architecture for Video Database System

MS I 8 Comparison of Different Video Database Management Projects

MS I 9 Digestive System Esophagus Stomach Omasum Abomasum Protozoa Salivary Glands Small Intestine Large Intestine Rumen Reticulum Exit Multimedia Browsing Graph The Ruminant Digestive System Four compartment stomach that: - utilizes complex carbohydrates such as cellulose - utilizes non-protein nitrogen sources - utilizes feeds of no nutritive value for humans Rumen The rumen, the first compartment of the ruminant stomach, is essentially non- functional at birth but develops rapidly when calves are given solid feeds such as hay and grain. Sample Multimedia Document

MS I 10 Document Representation and Specification Complexity Media Types and Attributes Complexity Structure Complexity SGML/XML HTML PDF Interleaf/ Word Framemaker VRML MPEG HyTime Style Complexity

MS I 11 Specification - Reliability - Resolution - Rate of Presentation - Display Area - Temporal Synchronization ( Intra/Inter ) Meta Data / User Interface Translation - End-to-End Delays - Jitter Delay - Bandwidth - Packet Loss Rate - Storage Throughput/ Bandwidth - Storage Delays - Distributed Database Coordination (QoS) - Storage Throughput/ Bandwidth - Storage Delays - Distributed Database Coordination (QoS) - CPU Throughput - Memory Overflow and Reliability - Intrusion Detection - Access Control - Intrusion Detection - Access Control Database SecurityOS Network Negotiation Dependency Model Analysis and QoS Adjustment End-to-End Run Time Scheduling End-to-End Resource Allocation and Scheduling End-to-End QoP / QoS Management

MS I 12 Multimedia Document Management n Quality of presentation requirements Resolution, reliability, rateResolution, reliability, rate n Synchronization requirements Temporal, spatial, and logical structure specificationTemporal, spatial, and logical structure specification n Media processing requirements Coloring, enhancements, dubbing, etc.Coloring, enhancements, dubbing, etc. n Security attributes

MS I 13 TextVideoImageTextVideoAudio  1  2  3  4 Petri-Net Based Model for Synchronization and Quality Specification of Multimedia Documents

MS I 14 Rumen Reticulum Digestive System Esophagus Stomach Omasum Abomasum Protozoa Salivary Glands Small Intestine Large Intestine Image Audio Video Text Audio Image Text Multimedia Browsing Environment

MS I 15 Research Issues for Quality and Performance of Multimedia Document Management Systems n Document modeling (Dimensionality/Flexibility/Portability) Languages (SGML/XML)Languages (SGML/XML) Graphical (OCPN)Graphical (OCPN) n Transformation among models and heterogeneity management of models n Development of efficient search and indexing techniques for multimedia documents using these models n Clustering of documents Based on contents for fast searchBased on contents for fast search Controlled accessControlled access

MS I 16 Comparison of Different Multimedia Document Models

MS I 17 User Multimedia Structuring Module OODBMS and MQL Distributed Query Processor HyTime/SGML/XML Translator HyTime/SGMLXML Parser --> Multimedia Query Language Document Generator Tool Library Temporal Synchronization Manager Multimedia Meta-Data Video Database Text Database Image Database Audio Database To Presentation Device Software Architecture of a Multimedia Database Management System

MS I 18 Presentation Device Audio DBMS Network A Network B Image Audio Text Video DBMS Distributed Multimedia Database Systems

MS I 19 Research Issues in Distributed Multimedia Database Management n Distributed Object Management n Efficient distributed query processing techniques n Network caching techniques for composition of distributed objects n Adaptive data filtering techniques to satisfy diverse user QoS requirements

MS I 20 Multimedia Database Client Visual Tool for Multimedia Document Generation Multimedia Database Interface Multimedia Presentation Subsystem Multimedia Database Server Multimedia Database Server Multimedia Database Server... TextImageVideoAudio Meta Data Database Management System Media Server Subsystem Database Connectivity Integrated Multimedia Information Server Directory Management Multimedia Meta Data Management Distributed Query Processor API for SBS Network Architecture of a Distributed Multimedia Database Management

MS I 21 Planned Experiments n Development of video database management system ( using database compute servers and storage system) n Development of multimedia document management system n Development of distributed multimedia object management system using SBS architecture