INTERACTIVELY BROWSING LARGE IMAGE DATABASES Ronald Richter, Mathias Eitz and Marc Alexa.

Slides:



Advertisements
Similar presentations
Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus IMAGE RETRIEVAL WITH RELEVANCE FEEDBACK Hayati CAM Ozge CAVUS.
Advertisements

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih-Fu Chang Published in the Journal of Visual.
Presented by Xinyu Chang
Content-Based Image Retrieval
Efficiently searching for similar images (Kristen Grauman)
Query Specific Fusion for Image Retrieval
Lazy vs. Eager Learning Lazy vs. eager learning
Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed.
Data-driven Visual Similarity for Cross-domain Image Matching
Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.
A Novel Scheme for Video Similarity Detection Chu-Hong Hoi, Steven March 5, 2003.
Fast and Compact Retrieval Methods in Computer Vision Part II A. Torralba, R. Fergus and Y. Weiss. Small Codes and Large Image Databases for Recognition.
SST:an algorithm for finding near- exact sequence matches in time proportional to the logarithm of the database size Eldar Giladi Eldar Giladi Michael.
CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman ICCV 2003 Presented by: Indriyati Atmosukarto.
Scalable and Distributed Similarity Search in Metric Spaces Michal Batko Claudio Gennaro Pavel Zezula.
Stockman MSU CSE1 Image Database Access  Find images from personal collections  Find images on the web  Find images from medical cases  Find images.
1998/5/21by Chang I-Ning1 ImageRover: A Content-Based Image Browser for the World Wide Web Introduction Approach Image Collection Subsystem Image Query.
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman University of Oxford ICCV 2003.
1 An Empirical Study on Large-Scale Content-Based Image Retrieval Group Meeting Presented by Wyman
Presented by Zeehasham Rasheed
KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.
Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.
Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.
Supporting Information Needs by Ostensive Definition in an Adaptive Information Space Iain Campbell 1995 Gretchen Schwarz.
Face Processing System Presented by: Harvest Jang Group meeting Fall 2002.
A fuzzy video content representation for video summarization and content-based retrieval Anastasios D. Doulamis, Nikolaos D. Doulamis, Stefanos D. Kollias.
Fast multiresolution image querying CS474/674 – Prof. Bebis.
Creating and Exploring a Large Photorealistic Virtual Space INRIA / CSAIL / Adobe First IEEE Workshop on Internet Vision, associated with CVPR 2008.
Computer vision.
Multimedia and Time-series Data
Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology Mining Rare and Frequent Events in Multi-camera Surveillance.
Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.
Content-Based Image Retrieval
An Efficient Approach to Clustering in Large Multimedia Databases with Noise Alexander Hinneburg and Daniel A. Keim.
Glasgow 02/02/04 NN k networks for content-based image retrieval Daniel Heesch.
1 Motivation Web query is usually two or three words long. –Prone to ambiguity –Example “keyboard” –Input device of computer –Musical instruments How can.
Video Google: A Text Retrieval Approach to Object Matching in Videos Josef Sivic and Andrew Zisserman.
80 million tiny images: a large dataset for non-parametric object and scene recognition CS 4763 Multimedia Systems Spring 2008.
IEEE Int'l Symposium on Signal Processing and its Applications 1 An Unsupervised Learning Approach to Content-Based Image Retrieval Yixin Chen & James.
Beyond Sliding Windows: Object Localization by Efficient Subwindow Search The best paper prize at CVPR 2008.
The Curse of Dimensionality Richard Jang Oct. 29, 2003.
Event retrieval in large video collections with circulant temporal encoding CVPR 2013 Oral.
Query Sensitive Embeddings Vassilis Athitsos, Marios Hadjieleftheriou, George Kollios, Stan Sclaroff.
Intelligent Database Systems Lab N.Y.U.S.T. I. M. Externally growing self-organizing maps and its application to database visualization and exploration.
1Ellen L. Walker Category Recognition Associating information extracted from images with categories (classes) of objects Requires prior knowledge about.
An Approximate Nearest Neighbor Retrieval Scheme for Computationally Intensive Distance Measures Pratyush Bhatt MS by Research(CVIT)
Multi-object Similarity Query Evaluation Michal Batko.
A Novel Visualization Model for Web Search Results Nguyen T, and Zhang J IEEE Transactions on Visualization and Computer Graphics PAWS Meeting Presented.
Content-Based Image Retrieval QBIC Homepage The State Hermitage Museum db2www/qbicSearch.mac/qbic?selLang=English.
Yixin Chen and James Z. Wang The Pennsylvania State University
Query by Image and Video Content: The QBIC System M. Flickner et al. IEEE Computer Special Issue on Content-Based Retrieval Vol. 28, No. 9, September 1995.
Computer Vision Group Department of Computer Science University of Illinois at Urbana-Champaign.
Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC Relevance Feedback for Image Retrieval.
Similarity Measurement and Detection of Video Sequences Chu-Hong HOI Supervisor: Prof. Michael R. LYU Marker: Prof. Yiu Sang MOON 25 April, 2003 Dept.
NN k Networks for browsing and clustering image collections Daniel Heesch Communications and Signal Processing Group Electrical and Electronic Engineering.
CSCI 631 – Foundations of Computer Vision March 15, 2016 Ashwini Imran Image Stitching.
ITree: Exploring Time-Varying Data using Indexable Tree Yi Gu and Chaoli Wang Michigan Technological University Presented at IEEE Pacific Visualization.
A Self-organizing Semantic Map for Information Retrieval Xia Lin, Dagobert Soergel, Gary Marchionini presented by Yi-Ting.
Fast multiresolution image querying
Efficient Image Classification on Vertically Decomposed Data
Self-Organizing Maps for Content-Based Image Database Retrieval
Efficient Image Classification on Vertically Decomposed Data
Image Retrieval Longin Jan Latecki.
A Fast and Scalable Nearest Neighbor Based Classification
Instance Based Learning
Multimedia Information Retrieval
Magnet & /facet Zheng Liang
Color Image Retrieval based on Primitives of Color Moments
Presentation transcript:

INTERACTIVELY BROWSING LARGE IMAGE DATABASES Ronald Richter, Mathias Eitz and Marc Alexa

Motivation: Specifying an image Humans lack communication channel for describing an image Image out ? Audio in Image in Audio out

Motivation: Traditional image queries Queries are typically based on –Words –Existing images –(Hand-drawn) Sketches This is not how humans –think or –communicate images Communication is a dialogue

Human centric image search Interactive, human-in-the-loop Basic interaction: browsing –“…exploration of potentially very large spaces through a sequence of local decisions or navigational choices.” [Heesch(2008)] –The image is not like this, more like that Goal: interactivity with real-world collections

Overview Current retrieval systems rely on common scheme: … 0.17, s … … 0.15, s ≈

Color Histogram Descriptor L a b

Color Layout Descriptor L a b

Gist Descriptor [Torralba’01]

Nearest Neighbor Search query smallest distance corresponds to most similar image in database

Combining Features need to combine nearest neighbor results linear combination of distance vectors fixed combination may not reflect the users mental image ++=

NNk Search [Heesch’04] importance of each image feature is unknown use multiple (convex) linear combinations with different weights 1, 0, 0 0, 1, 00, 0, 1 ⅓, ⅓, ⅓ uniformly sampled weight space

NNk Search 2/6 1/6 4/6 = ++ 2/61/64/6

NNk Search - compute nearest neighbor for all weightings - set of unique nearest neighbors = NNk result

NNk Search expensive computation –precomputation possible, but O(N²) –large databases with N > 1,000,000 our contribution: –compute distances only to probable NNk candidates –use fast approximate nearest neighbor search

NNk Approximation Compute NNk on a subset of the database –compute distances for n approximate nearest neighbors –complement missing distances –apply NNk NNk

K-means tree: construction root … … [Fukunaga’75]

K-means tree: query root … … [Fukunaga’75] q query result:

Results 10K approximate nearest neighbors, 1M images –precision: 85% –speed: 267 ms

video

Limitations - Video

Limitations

Future Work Out-of-core data structures –handle even larger image collections Localized Browsing –fix image parts as prior for next search Evaluation –reachability of all images –number of iterations to desired image

Thanks!

NNk Search Step 2: Normalize distances mean = 0 std = 1

NNk Search 1:0.5:1.5 1:1:1 1.5:1:0.5

NNk Search 1.5:1:0.5

NNk search result visualization

Feature Space

Euclidian distance Manhattan distance Similarity between Images

System Overview Image database Image database Color histogram k-means tree k-means tree Approx. NNk search Browser Color layout Color layout k-means tree k-means tree Gist k-means tree k-means tree Image descriptors Index structures Search algorithm User interface Offline computation

Results: Precision

Results: Speed 1 million images in database approximation uses 1% of probable images