IMAGE DATABASES Prof. Hyoung-Joo Kim OOPSLA Lab. Computer Engineering Seoul National University.

Slides:



Advertisements
Similar presentations
1 DATA STRUCTURES USED IN SPATIAL DATA MINING. 2 What is Spatial data ? broadly be defined as data which covers multidimensional points, lines, rectangles,
Advertisements

Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual.
Presented by Xinyu Chang
Presented By: Vennela Sunnam
Image Information Retrieval Shaw-Ming Yang IST 497E 12/05/02.
Clustering & image segmentation Goal::Identify groups of pixels that go together Segmentation.
3D Shape Histograms for Similarity Search and Classification in Spatial Databases. Mihael Ankerst,Gabi Kastenmuller, Hans-Peter-Kriegel,Thomas Seidl Univ.
Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed.
Multimedia DBs. Multimedia dbs A multimedia database stores text, strings and images Similarity queries (content based retrieval) Given an image find.
Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.
Spatial Mining.
Mining for High Complexity Regions Using Entropy and Box Counting Dimension Quad-Trees Rosanne Vetro, Wei Ding, Dan A. Simovici Computer Science Department.
Image Search Presented by: Samantha Mahindrakar Diti Gandhi.
SWE 423: Multimedia Systems
Visual Querying By Color Perceptive Regions Alberto del Bimbo, M. Mugnaini, P. Pala, and F. Turco University of Florence, Italy Pattern Recognition, 1998.
Chapter 3: Data Storage and Access Methods
Spatial Indexing I Point Access Methods.
Spatial and Temporal Data Mining
Techniques and Data Structures for Efficient Multimedia Similarity Search.
Fast Image Retrieval Using Color-spatial Information NTUT CSIE D.W. Lin
1 An Empirical Study on Large-Scale Content-Based Image Retrieval Group Meeting Presented by Wyman
Presented by Zeehasham Rasheed
Euripides G.M. PetrakisIR'2001 Oulu, Sept Indexing Images with Multiple Regions Euripides G.M. Petrakis Dept.
CS292 Computational Vision and Language Visual Features - Colour and Texture.
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.
A fuzzy video content representation for video summarization and content-based retrieval Anastasios D. Doulamis, Nikolaos D. Doulamis, Stefanos D. Kollias.
Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.
Content-based Image Retrieval (CBIR)
Fast Subsequence Matching in Time-Series Databases Christos Faloutsos M. Ranganathan Yannis Manolopoulos Department of Computer Science and ISR University.
Introduction --Classification Shape ContourRegion Structural Syntactic Graph Tree Model-driven Data-driven Perimeter Compactness Eccentricity.
Multimedia and Time-series Data
Wavelet-Based Multiresolution Matching for Content-Based Image Retrieval Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung.
Spatial Data Management Chapter 28. Types of Spatial Data Point Data –Points in a multidimensional space E.g., Raster data such as satellite imagery,
Multimedia Databases (MMDB)
Image and Video Retrieval INST 734 Doug Oard Module 13.
Chapter 7 Web Content Mining Xxxxxx. Introduction Web-content mining techniques are used to discover useful information from content on the web – textual.
Multimedia Information Retrieval
Presented by Tienwei Tsai July, 2005
Content-Based Image Retrieval
Texture. Texture is an innate property of all surfaces (clouds, trees, bricks, hair etc…). It refers to visual patterns of homogeneity and does not result.
COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL Presented by 2006/8.
PMLAB Finding Similar Image Quickly Using Object Shapes Heng Tao Shen Dept. of Computer Science National University of Singapore Presented by Chin-Yi Tsai.
Introduction --Classification Shape ContourRegion Structural Syntactic Graph Tree Model-driven Data-driven Perimeter Compactness Eccentricity.
Content-Based Image Retrieval Using Fuzzy Cognition Concepts Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung University.
2005/12/021 Content-Based Image Retrieval Using Grey Relational Analysis Dept. of Computer Engineering Tatung University Presenter: Tienwei Tsai ( 蔡殿偉.
2005/12/021 Fast Image Retrieval Using Low Frequency DCT Coefficients Dept. of Computer Engineering Tatung University Presenter: Yo-Ping Huang ( 黃有評 )
Intelligent Database Systems Lab N.Y.U.S.T. I. M. Externally growing self-organizing maps and its application to database visualization and exploration.
Levels of Image Data Representation 4.2. Traditional Image Data Structures 4.3. Hierarchical Data Structures Chapter 4 – Data structures for.
Spatial Database 2/5/2011 Reference – Ramakrishna Gerhke and Silbershatz.
An Approximate Nearest Neighbor Retrieval Scheme for Computationally Intensive Distance Measures Pratyush Bhatt MS by Research(CVIT)
Content-Based Image Retrieval Using Block Discrete Cosine Transform Presented by Te-Wei Chiang Department of Information Networking Technology Chihlee.
Euripides G.M. PetrakisIR'2001 Oulu, Sept Indexing Images with Multiple Regions Euripides G.M. Petrakis Dept. of Electronic.
Content-Based Image Retrieval QBIC Homepage The State Hermitage Museum db2www/qbicSearch.mac/qbic?selLang=English.
Database Management Systems, R. Ramakrishnan 1 Algorithms for clustering large datasets in arbitrary metric spaces.
Indexing OLAP Data Sunita Sarawagi Monowar Hossain York University.
Query by Image and Video Content: The QBIC System M. Flickner et al. IEEE Computer Special Issue on Content-Based Retrieval Vol. 28, No. 9, September 1995.
VISUAL INFORMATION RETRIEVAL Presented by Dipti Vaidya.
Relevance Feedback in Image Retrieval System: A Survey Tao Huang Lin Luo Chengcui Zhang.
1 Faculty of Information Technology Enhanced Generic Fourier Descriptor for Object-Based Image Retrieval Dengsheng Zhang, Guojun Lu Gippsland School of.
Content-Based Image Retrieval Using Color Space Transformation and Wavelet Transform Presented by Tienwei Tsai Department of Information Management Chihlee.
Visual Information Retrieval
Data Mining Soongsil University
Spatial Indexing I Point Access Methods.
UZAKTAN ALGIILAMA UYGULAMALARI Segmentasyon Algoritmaları
Self-Organizing Maps for Content-Based Image Database Retrieval
Image Segmentation Techniques
Computer Vision Lecture 16: Texture II
Ying Dai Faculty of software and information science,
Improving Retrieval Performance of Zernike Moment Descriptor on Affined Shapes Dengsheng Zhang, Guojun Lu Gippsland School of Comp. & Info Tech Monash.
Presentation transcript:

IMAGE DATABASES Prof. Hyoung-Joo Kim OOPSLA Lab. Computer Engineering Seoul National University

Contents Introduction Image database systems Indexing issues and basic mechanisms A taxonomy on image indexes Color-spatial hierarchical indexes Signature-based color-spatial retrieval Summary

Introduction Traditional DBMS –effective in managing structured data –not effective for images that are non-alphanumeric and unstructured Applications to manage images –medical application –geographic information system –satellite image database –criminal database –interior design –art galleries and museum management –etc...

Introduction(2) Content-based image retrieval techniques –techniques that retrieve image based on their visual properties such as texture, color, shape, etc –sequentially comparing image feature is time- consuming and impractical –need content-based image index

Image Database System Additional functionalities –feature extraction the system must be able to analyze an image to extract key features such as shape, color, texture –feature-based indexing the system must build indexes based on the features extracted –content-based retrievals the system should support a wide range of queries that involve the contents of the image –measure of similarity the system requires a measure to capture what we humans perceive as similarity between two images

Architecture of Image DBMS Input Image PREPROCESSING MODULE QUERY MODULE Image Input/Scanner Feature Extraction Update Index/ Database Interactive Query Formulation Browsing & Feedback Feature Extraction Feature Matching Runtime Processor Concurrency Control & Recovery Manager Feature/Image Database User Query Output Retrieved Image

Indexing Issues Three issues in content-based index design –determine a representation for the indexing feature –determine a similarity measure between two images based on their representation –determine an appropriate index organization

Indexing Issues Desirable properties of a representation(first issue) –exactness –space efficiency –computationally inexpensive similarity matching –preservation of the similarity between the features –automatic extraction –insensitivity to noise, distortion, rotation

Indexing Issues Main criterion of the similarity measure(second issue) –if two images are similar under the indexing feature, then their representations should remain so. Several alternatives to determine the similarity –exact match the representation of an image feature is usually coarse, in sense that images with similar feature will be mapped to the same representation –approximate match the degree of similarity between the image representations is computed based on some approximation techniques

Indexing Issues Criteria for selection of an index structure –the similarity measure can be supported efficiently –storage efficiency –maintenance(update) overhead Appropriate index structure –be determined by the representation and similarity measure –example if image feature is represented as a vector, an the similarity measure is the Euclidean distance, then a natural choice is the multi-dimensional point access method

Basic Index Schemes Spatial access methods(SAMs) –basic idea extract k image features for each image map images into points in a k-dimensional feature space use SAMs such as the grid file, quad-tree, the family of R- tree –problem: “high-dimensionality curse” these techniques perform no better than sequential scanning as the dimension becomes sufficiently large

Basic Index Schemes Inverted file –basic idea an inverted list is created for each distinct key(indexed features) the inverted list consists of a list of pointers to the objects that contain features that are similar to the indexed feature –problem high storage overhead & expensive update Signature file –refer [Text Database]

Taxonomy on Image Indexes Shape feature Spatial relationship Texture Color

Taxonomy on Image Indexes Content-based indexes Spatial relationship Texture Objects Shape 1-D string Inverted file Numerical vectors Color Color histogramColor-Spatial Signature2-D string Multi-level signature file Sequential file Multi-dimensional index Multi-dimensional index Multi-level histogram 2-level B+-tree 3-tier color index Tamura features Multi-dimensional index Signature file Similarity against representative objects Signature Inverted file

Shape Feature Representation –boundary information –a collection of rectangle that forms a rectangular cover of the shape –mathematical morphology –pattern spectrum –numerical vector using 2-D Fourier transformation or Wavelet transformation Problems –shape vary widely from object to object –unless the images have very distinct shape, the performance may suffer

Spatial relationship Representation –2-D string semantic representation for spatial relationship using a two-dimensional string projection of the symbols along the x-axis and y-axis example : “O1 is to left of O2 which is left of O3” the projection on the x-axis - O1 < O2 < O3 variations and extensions to the 2-D string have been proposed –Problem spatial relationships can be drastically affected by the orientation of the image

Texture and Color Texture –can be represented by the coarseness, contrast, directionality –the extraction of text information is a computationally intensive operations Color –color histogram that captures the color composition of images –color alone is not sufficient to characterize an image consider two image - one with the top half blue and bottom half red, while the other’s left half is red and it’s right half is blue although these two images are similar in color composition, the are entirely different to a human observer. –recent studies have proposed to integrate color and its spatial distribution

Color-spatial hierarchical Indexes Hierarchical Indexes –multiple indexing mechanisms are integrated to form a single index structure –three indexes that have been proposed to integrated color and spatial information for retrieval Two-level B + -tree structure Three-tier color index Sequential Multi-Attribute Tree(SMAT)

Two-level B + -tree Feature extraction –the color-spatial information of an image is modeled by splitting the image into 9 equal sub- areas(3  3) –the color information within each sub-area is represented by a color histogram –one can obtain a more accurate similarity by matching the corresponding color histogram of two image –the color histogram within each sub-area is mapped into a numerical key Retrieval technique

Two-level B+-tree Retrieval technique –use two level information –the first level describes the composition of colors corresponding to the histogram of the region the colors are grouped into 11 bins each group is assigned a range which bounds the percentage of pixels in the mage with colors of group –the second level contains the average H, average V, and average C values of all the 11 histogram bins

Two-level B+-tree Level 1: B+-tree on Normal Pixel Count Level 2: B+-tree on Average H,V, and C values

Three-tier Color Index Layer 1: –dominant color classification a fixed number of dominant colors is extracted the dominant colors are those with the largest number of pixel count Layer 2: –multidimensional R-tree structure the image can be assigned to a partition prune away image within the candidate partitions that are not relevant Layer 3: –multi-level color histogram(quad-tree structure) compare the histograms of the query image with those of remaining potential candidate image

Tree-tier Color Index Tier 1: Dominant Color Classification Tier 2: R-tree Tier 1: Multi-Level Color Histogram K = 1 K = 2 K = n(no.of colors) (0,1) (0,2).. (0,n) (1,2) …...…(2,n) …. (0,1,2)... (0,1,n) ……. (0,2,n) …...… …. K : dominant colors

SMAT Height-balanced color-spatial index –the problem with the two approaches individual tree structures(B+-tree, R-tree, Dominant Color Classification) are height-balanced, the entire hierarchical index structure may not be so. –height-balancing of SMAT SMAT is a multi-tier index structure similar to two approaches, but remains height-balancing of the entire index structure by controlling the growth of the tree using a certain threshold which is application-specific value. refer to the VLDB Journal(1998) 7: for more information

Signature-based Color-Spatial Retrieval Representation of color-spatial information –an image is partitioned into a grid of m  n cells of equal size –the colors that can be used to represent a cell are determined –for a given color, each cell is examined to determine the percentage of the total number of pixels in the cell having that color An image partitioned into a 4x8 grid Cell does not satisfy the threshold Cell does satisfy the threshold - an image is represented by 32-bit color signature,

The retrieval process –an image with k colors has k color signatures –let Q i and D i denote the signature of color i for a query image Q and a database image D. –let the representative color sets of Q and D be C Q and C D. Signature-based Color-Spatial Retrieval

Summary Promising area that require further research –performance evaluation comparative study will be useful for application designers and practitioners to pick the best method for their applications –more on access method designing efficient access methods will make the content- based retrieval techniques more practical and useful –concurrent access and distributed indexing we expect to see more real-time application as well as application running in parallel or distributed environment

Summary –Integration and optimization most content-based image retrieval techniques only capture a part of the image’s semantics important issues –selecting an “optimal” set of image features that fits best for an application –developing techniques that can integrate them into achieve the optimal results one promising method(semantic-based retrieval technique) –use content-based techniques as the basis, but also exploits semantic meanings of the image and queries to support concept-based queries