Multimedia Alicja Wieczorkowska Multimedia database systems and GIS

A.Wieczorkowska /482 Databases Relational databases Object-Oriented Databases

A.Wieczorkowska /483 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /484 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /485 Multidimensional data structures k-d trees Point Quadtrees MX-Quadtrees R-Trees

A.Wieczorkowska /486 Multidimensional data structures k-d trees Point Quadtrees MX-Quadtrees R-Trees

A.Wieczorkowska /487 k-d trees k-d tree is used to store k-dimensional point data such as that shown below

A.Wieczorkowska /488 2-d trees Each node has a certain record structure nodetype=record INFO: infotype XVAL: real YVAL: real LLINK: nodetype RLINK: nodetype end INFOXVALYVAL LLINKRLINK

A.Wieczorkowska /489 2-d trees 2-d tree is any binary tree satisfying the following conditions: –If N is a node in the tree such that level(N) is even, then every node M in the subtree rooted at N.LLINK has the property that M.XVAL =N.XVAL –If N is a node in the tree such that level(N) is odd, then every node M in the subtree rooted at N.LLINK has the property that M.YVAL =N.YVAL

A.Wieczorkowska / d trees - example

A.Wieczorkowska / d trees - example

A.Wieczorkowska /4812 Multidimensional data structures k-d trees Point Quadtrees MX-Quadtrees R-Trees

A.Wieczorkowska /4813 Point Quadtrees Used to represent point data in 2D spaces always splits regions into 4 parts Node: qtnodetype = record INFO: infotype; XVAL: real; YVAL: real; NW, SW, NE, SE: qtnodetype end

A.Wieczorkowska /4814 Point Quadtrees

A.Wieczorkowska /4815 Multidimensional data structures k-d trees Point Quadtrees MX-Quadtrees R-Trees

A.Wieczorkowska /4816 MX-Quadtrees The shape (and height) of of the tree is independent of the number of nodes present in the tree, as well as the order of insertion of these nodes We assume that the map being represented is split up into a grid of size 2 k x 2 k

A.Wieczorkowska /4817 MX-Quadtrees

A.Wieczorkowska /4818 MX-Quadtrees ChildXLBXUBYLVYUB NWN.XLBN.XLB+w/2N.YLB+w/2N.YLB+w SWN.XLBN.XLB+w/2N.YLBN.YLB+w/2 NEN.XLB+w/2N.XLB+wN.YLB+w/2N.YLB+w SEN.XLB+w/2N.XLB+wN.YLBN.YLB+w/2 W=N.XUB-N.XLB, root: XLB=0, XUB=2 k, YLB=0, YUB=2 k

A.Wieczorkowska /4819 Multidimensional data structures k-d trees Point Quadtrees MX-Quadtrees R-Trees

A.Wieczorkowska /4820 R-Trees Used to store rectangular regions of an image or map particularly useful in storing very large amounts of data on disc each R-tree has an associated order, which is an integer K; each nonleaf R-tree node contains a set of at most K rectangles and at least rectangles

A.Wieczorkowska /4821 R-Trees Intuitively, each nonleaf node in R-tree, with the exception of the root, must be at least half full the height of the R-tree used to store a collection of rectangles is usually quite small a rectangle is either a “real” rectangle or a group rectangle Structure: rtnodetype = record Rec 1,…, Rec K : rectangle P 1,…, P K : rtnodetype

A.Wieczorkowska /4822 R-Trees

A.Wieczorkowska /4823 Comparison of different data structures Point quadtrees are very easy to implement –Point containing k nodes may have height k K-d trees are very easy to implement –In general point containing k nodes may have height k, in practice path lengths from root to leaf longer than in point quadtrees MX-quadtrees have guaranteed height of at most O(n), where n is the number of records in the tree The same applies to R-trees; fewer disc accesses –Bounding rectangles may overlap, so we might follow multiple paths down the tree R-trees are generally preferred over k-d trees

A.Wieczorkowska /4824 Selected commercial systems Informix –MapInfo Geocoding Oracle Universal Server Intergraph VISION (Vision International – Sybase’s partner) ARC/INFO (ESRI)

A.Wieczorkowska /4825 Geographic Information Systems GIS is a System of computer software, hardware and data, and personnel to help manipulate, analyze and present information that is tied to a spatial location –spatial location – usually a geographic location –information – visualization of analysis of data –system – linking software, hardware, data –personnel – a thinking explorer who is key to the power of GIS

A.Wieczorkowska /4826 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /4827 Image databases Querying image databases is often fundamentally different from querying textual databases and is further complicated by the usually imprecise techniques for image analysis describing the content of an image can be done either automatically or manually; in both cases, structures to store the results are needed image databases can be implemented as: –extensions of the relational model –using n-dimensional data structures –using image transformations

A.Wieczorkowska /4828 Image representations Raw images –shape descriptor - describes the shape/location of the region within the object is located –property descriptor - for example RGB values Compressed Image Representations –DFT –DCT –wavelet transform

A.Wieczorkowska /4829 Image databases Image processing: segmentation –homogeneous regions with respect to some homogeneity predicate over k% of cells have the same color use a “baseline” function and a maximal permissible noise level Similarity-based retrieval

A.Wieczorkowska /4830 Similarity Metric approach –we assume a distance metric; given an input image i, we look for the “nearest” neighbor of i in the image archive Transformation approach –the users should specify what they consider to be similar –for 2 given objects o 1, o 2, the level of dissimilarity between o 1 and o 2 is proportional to the (minimum) cost of transforming o 1 into o 1 or vice versa

A.Wieczorkowska /4831 Representing image DBs Relational model with spatial data structures –R-trees, generalized R-trees, etc. Using image transformations –DCT –DFT

A.Wieczorkowska /4832 Selected commercial systems Knoware (Camrax) –for artwork, real estate, and personnel management systems Informix –images indexed using specialized techniques –visual image retrieval datablade DB2 (IBM) –facilitate retrieval of data based on attributes of images such as colors, texture and so on

A.Wieczorkowska /4833 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /4834 Text/document databases Synonymy: –various words may possess the same meaning polysemy: –the same word may mean many different things in different contexts evaluating the performance of text retrieval systems all relevant returned

A.Wieczorkowska /4835 Text/document databases Precision - of algorithm A with respect to the predicate relevant and the test set D test is P t % for topic t  –how many of the answers returned by the algorithm are in fact correct Recall –how many of the right documents are in fact retrieved by the query

A.Wieczorkowska /4836 Text/document databases Stop list - set of words that are deemed “irrelevant”, even though they may appear frequently (the, and, for) Word stems (drug, drugged, drugs) Frequency Tables - in a frequency table FreqT, each document d n is represented by the n-th column, and the occurrence of each term/word t n is represented by n-th row

A.Wieczorkowska /4837 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /4838 Video databases Video is a sequence of images organizing video content: –which aspect to choose –content extraction segmentation methods usually require some restricted conditions

A.Wieczorkowska /4839 Which aspects of video to store? The content of a video v is described by: –OBJ - set of objects of interest in the video v –AC - set of activities of interest in v – - function, tells which objects and which activities are associated with any given frame f example: –educational databases: lecturers, topics, lecture, questions, answers

A.Wieczorkowska /4840 Indexing video content Compact representation of the video content: –Frame segment trees 2 arrays are created: object array and activity array, ordered linked lists of pointers the frame segment tree is constructed from the segment table –R-segment trees each R-tree node has a special structure to specify, for each rectangle, which object or activity is associated with it

A.Wieczorkowska /4841 Video standards MPEG Cinepak MPEG-2

A.Wieczorkowska /4842 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /4843 Audio databases Audio signals are certain kinds of continuous analytic functions compression into discrete representation –transforms: DFT, DCT indexing audio data –TV-trees – telescope vector trees - can be used (technique applied in text/document databases)

A.Wieczorkowska /4844 Audio databases Metadata to represent audio content: –singers, score, transcript Segmentation –split up the audio signal into relatively homogeneous windows –window size specified a priori or user-defined Feature extraction –intensity, loudness, pitch, brightness –statistical properties: variance, correlation

A.Wieczorkowska /4845 Selected commercial systems Audio databases are still in their infancy Informix – MuscleFish DB2 (IBM)

A.Wieczorkowska /4846 Organizing multimedia content Multidimensional data structures Image databases Text/document databases Video databases Audio databases Multimedia databases

A.Wieczorkowska /4847 Architectures for content organization –the principle of autonomy –the principle of uniformity –hybrid organization Multimedia databases

A.Wieczorkowska /4848 Discs CD-ROM Tapes Source V.S. Subrahmanian: Principles of Multimedia Database Systems, Morgan Kaufmann Publishers, San Francisco, CA, USA, 1998 Storage