Download presentation
Presentation is loading. Please wait.
1
Bag of Features Approach: recent work, using geometric information
2
Problem Search for object occurrences in very large image collection
3
2 sub problems Object Category Recognition and Specific Object Recognition
4
Motivation Look for product information Look for similar products
5
Related work on large scale image search Most systems build upon the BoF framework [Sivic & Zisserman 03] – Large (hierarchical) vocabularies [Nister Stewenius 06] – Improved descriptor representation [Jégou et al 08, Philbin et al 08] – Geometry used in index [Jégou et al 08, Perdoc’h et al 09] – Query expansion [Chum et al 07] – … Efficiency improved by: – Min-hash and Geometrical min-hash [Chum et al. 07-09] – Compressing the BoF representation [Jégou et al. 09]
6
Local Features - SIFT
7
Creating a visual vocabulary 12 34
8
Inverted Index Index construction Searching
9
Use geometry Possible directions: – Change/optimize spatial verification stage – Insert a new geometric information to the index Ordered BOF Bundled features Visual phrases – Change the searching algorithm
10
Survey for today Spatial Bag-of-features [Cao, CVPR2010] Image Retrieval with Geometry-Preserving Visual Phrases [Zhang Jia Chen, CVPR2011] Smooth Object Retrieval using a Bag of Boundaries [Arandjelovi Zisserman, ICCV2011]
11
Spatial BOF Basic idea:
12
Spatial BOF Constructing linear and circular ordered bag- of-features:
13
Spatial BOF Translation invariance:
14
Spatial BOF Pros: – Gets better performance than BOF+RANSAC for large scale dataset* – Same format as standard BOF Cons: – Is dataset dependent because of need of training Do not present the results for large scale dataset with transfer learning from another dataset Future work – Check it with cross training for large dataset. Otherwise, it is not worth working further.
15
Geometry-Preserving Visual Phrases Basic idea:
16
Geometry-Preserving Visual Phrases Representation – Quantize image to 10x10 grid – Histogram of GVPs of length k – GVP dictionary size is “choose k from N visual words”
17
Geometry-Preserving Visual Phrases Pros: – Outperforms BOV + RANSAC Cons: – Only translation invariant because of memory Future work
18
BOF for smooth objects Idea: The information used for retrieval Query object Segment Gradient
19
BOF for smooth objects Results:
20
BOF for smooth objects Segmentation phase Over segmentation with super-pixels Classification of super-pixels: 3208 feature vector (median(Mag(Grad)), 4 bits, color histogram, BOF) SVM Post-processing
21
BOF for smooth objects Boundary description phase: Sample points on the boundary Calculate HoG at each point in 3 scales 340 dimensional L2 normalized vector * The descriptor is not rotation invariant
22
BOF for smooth objects Retrieval procedure: Boundary descripors are quantized (k=10k) Standard BOF scheme* Spatial verification for top 200 with loose affine homography (errors up to 100pixs) * No spatial information is recorded in the histogram
23
BOF for smooth objects Pros: – Solves the smooth object retrieval problem – Fast Cons: – Is dataset dependent because of need of training – Limited to objects with “solid” materials – segmentation has to catch the object’s boundary Future work – Eliminate the training step
24
Summary There is an active research in the field of CBIR to exploit geometry information. Each method with its limitations Still no widely accepted solution – Like spatial verification with RANSAC
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.