1. Multimedia Databases and Querying techniques
By: Rohit Kulkarni
CS 2310 – Spring 2008

2. Agenda
Background
Problem Definition
Challenges

3. Background What is MMDBMS? Normalization framework
What is data fusion?
The general problem
Querying technique

4. MMDBMS architecture

5. Requirements for MMDBMS
Traditional DBMS capabilities
Huge capacity for storage management
Information retrieval capabilities
Media integration, composition and representation
Multimedia query support
Multimedia interface and interactivity
Performance

6. Issues in MMDBMS Multimedia data modeling Multimedia object storage
Multimedia integration, presentation and QOS
Multimedia indexing, retrieval and browsing
Multimedia query support
Distributed multimedia database management
System support

7. Problem definitions Extended Dependencies The relational model
Similarity theory
Tuple distance function

8. An Example
Define a functional dependency between attributes FINGERPRINT and PHOTO of police database, and use the fingerprint matching function FINGERCODE for comparing digital fingerprint [JPH00], and the similarity technique used by QBIC for comparing photo images, we would write as follows
FINGERPRINTFINGERCODE(t’) PHOTOQBIC(t’’)

9. Inference rules for MFDs
Reflexive rule
Augmentation rule
Transitive rule
Decomposition rule
Union rule
Pseudotransitive rule

10. Normal forms in Multimedia databases
Normal forms are used to derive database schemes that prevent manipulation anomalies
Similar anomalies can arise in multimedia database
Types of normal forms are
1MNF, 2MNF , 3MNF and 4MNF

11. Sensor data fusion Background:
Need for Multiple Sensors:- data from a single sensor yields poor results in object recognition
Sensor Management Model

12. Sensor Management Model

13. sensor data fusion Problems:
Association of objects from different Sensors
Tracking

14. Need for query technique
Problem with existing query techniques
Why not SQL?
To support the retrieval and fusion of multimedia information from multiple sources and distributed databases, a spatial/temporal query language called QL has been proposed

15. Features of QL Easy to learn as syntax is similar to SQL
Allows user to specify queries for both Multimedia data sources and Multimedia databases
Supports multiple sensor sources and systematic modification of queries

16. Operator Classes
The operators in QL can be categorized with respect to their functionality.
The two main classes are:
transformational operators (the σ‑operators)
fusion operators (the ‑operators).

17. Transformational Operators
Definition: A σ‑operator is defined as an operator to be applied to any multi-dimensional source of objects in a specified set of intervals along a dimension. The operator projects the source along that dimension to extract clusters

18. Transformational Operators (contd)
As an example, if we write a σ‑expression for extracting the video frame sequences in the time intervals [t1-t2] and [t3-t4] from a video source VideoR.
The expression will be
is σtime([t1-t2], [t3-t4]) VideoR
where VideoR is projected along the time dimension to extract clusters (frames in this case) whose projected positions along the time dimension are in the specified intervals.

19. Fusion Operators Much more complex as it deals with Sensor data fusion
Requires input data in different time periods from multiple sensors
The output of the fusion‑operator is some kind of high level, qualitative representation of the fused object, and may include object type, attribute values and status values.

20. Is there a moving vehicle present in the given area and in the given time interval?

21. Is there a moving vehicle present in the given area and in the given time interval?
Corresponding query:
type,position, direction
(motion(moving) type(vehicle)
xy(*)
 (T)T mod 10 = 0 and T>t1 and T <t2
media_sources (video)media_sources
ype (vehicle) xyz(*)
 (T) T>t1 and T<t2
media_sources(laser_radar) media_sources)

22. Experimental Prototype

23. Challenges Handle large number of different sensors
Replacing manual query with a semi-automatic or fully automatic query refinement process

24. Applications of MMDBMS
Education- digital libraries, training, presentation
Healthcare- telemedicine, health information management
Entertainment- interactive TV, video on demand
Information dissemination- news, TV broadcasting
And many more!

25. References
Intelligent Querying Techniques for Sensor Data Fusion by Shi- Kuo Chang, Gennaro Costagliola, Erland Jungert and Karin Camara
A Normalization Framework for Multimedia Databases by S.K. Chang, V. Deufemia, G. Polese
Querying distributed Multimedia databases and data sources for sensor data fusion by S.K.Chang, Gennaro Costagliola, Erland Jungert and Francesco Orciuoli
Multimedia database management-requirements and issues by Donald A. Adjeroh and Kingsley C. Nwosu
Fuzzy Queries in Multimedia database system by Ronald Fagin
Bayesian Approaches to Multi-Sensor data fusion by Olena Punska, St. John’s CollegeMultimedia database management-requirements and issues by Donald A. Adjeroh and Kingsley C. Nwosu Querying distributed Multimedia databases and data sources for sensor data fusion by S.K.Chang, Gennaro Costagliola, Erland Jungert and Francesco Orciuoli

26. Multimedia Database and Querying techniques
By: Rohit Kulkarni
CS 2310 – Spring 2008
Thank you