1. Boundary Preserving Dense Local Regions
Jaechul Kim and Kristen Grauman
Univ. of Texas at Austin

2. Local feature detection
A crucial building block for many applications
Image retrieval
Object recognition
Image matching
Key issue:
How to detect local regions for feature extraction?

3. Related work Interest point detectors Dense sampling
e.g., Matas et al. (BMVC 02), Jurie and Schmid (CVPR 04), Mikolajczyk and Schmid (IJCV 04)
Dense sampling
e.g., Nowak et al. (ECCV 06)
Segmented regions and Superpixels
e.g., Ren and Malik (ICCV 03) , Gu et al. (CVPR 09),
Todorovic and Ahuja (CVPR 08),
Malisiewicz and Efros (BMVC 07), Levinshtein et al. (ICCV 09)
Hybrid
e.g., Tuytelaars (CVPR 10), Koniusz and Mikolajczyk (BMVC 09)

4. What makes a good local feature detector?
Desired properties:
- Repeatable
- Boundary-preserving
- Distinctively shaped
Existing methods lack one or more of these criteria, e.g.,
Lack repeatability
Lack distinctive shape,
straddle boundaries
Segments
Dense sampling
Interest points

5. Our idea: Boundary Preserving Local Regions (BPLRs)
Boundary preserving, dense extraction
Segmentation-driven feature sampling and linking
Repeatable local features capturing objects’ local shapes

6. Approach: Overview Sampling elements Linking elements
Initial elements for each segment are sampled based on distance transform of the segment
A segment
Sampled elements
Linking elements
A single graph structure reflecting main shapes and segment layout
Min. spanning tree
Grouping elements
Grouping neighboring elements into BPLR
Neighbor elements
BPLR

7. Approach: Sampling Input image Segment Sampled elements
Linking
Grouping
Zoom-in view x
An ”element” x
Input image
Segment
Sampled elements
from “all” segments
Distance transform
Dense regular grid
Sampled elements

8. Sampling
Approach: Linking
Linking
Grouping
Minimum spanning tree
Sampled elements’ locations (i.e., elements’ centers)
Global linkage structure

9. Role of spanning tree linkage
Sampling
Role of spanning tree linkage
Linking
Grouping
Min spanning tree prefers to link closer elements +
Multiple sampling
Due to distance transform-based sampling same-segment elements more likely linked
Due to multiple segmentations elements in overlapping segments more likely linked

10. Approach: Grouping Intersection of topology and Euclidean neighbor
Sampling
Approach: Grouping
Linking
Grouping
Descriptor
Intersection of topology and Euclidean neighbor
Reference element’s location
BPLR
Intersection of topology and Euclidean neighbor
Reference element’s location
Neighbor elements
Reference element’s location
Zoom-in view
Reference element’s location
Euclidean neighbor elements’ location
Euclidean neighbor
Reference element’s location
Topological neighbor elements’ location
Topological neighbor
Example detections of BPLRs
(Subset shown for visibility)
Reference element’s location

11. Example matches of BPLRs
Leak object boundary

12. Experiments 20-200 segments  ~7000 BPLRs in 400 x 300 image Tasks:
2-5 seconds to extract BPLRs per an image
PHOG + gPb descriptor used
Tasks:
Repeatability
Localization
Foreground segmentation
Object classification
Baselines:
Dense sampling (+ SIFT)
MSER (+ SIFT) [1]
Semi-local regions (+ SIFT) [2,3]
Segmented regions (+ PHOG) [4]
Superpixels [5]
[1] Matas et al., BMVC 02. [2] Quack et al., ICCV 07.
[3] Lee and Grauman, IJCV 09. [4] Arbelaez et al., CVPR 09.
[5] Ren and Malik, ICCV 03.

13. Example feature extractions
Proposed BPLRs
(Subset shown for visibility)
Segmented
regions
Superpixels
Interest regions
(MSERs)
Dense sampling

14. Repeatability for object categories
Bounding Box Hit Rate – False Positive Rate [Quack et al. 2007]
Applelogo
Test image
Giraffe
Bottle
Train images
Swan
Mug
True match
False positive
Comparison to baseline region detectors on ETHZ shape classes

15. Localization accuracy
Bounding Box Overlapping Score – Recall
Applelogo
Giraffe
Bottle
Compute overlapping score by projecting the training exemplar’s bounding box into the test image
Swan
Mug
Comparison to baseline region detectors on ETHZ shape classes

16. Localization accuracy
Test image
Database images with best matches to test BPLRs

17. Foreground segmentation
Replacing superpixels with BPLRs in GrabCut segmentation
Approach
Accuracy(%)
BPLR + GrabCut (Ours)
85.6
Superpixel + GrabCut
81.5
Superpixel ClassCut (Alexe et al., ECCV 10)
83.6
Superpixel Spatial Topic Model (Cao et al., ICCV 07)
67.0
Foreground segmentation in Caltech-28 dataset

18. Object classification
Nearest-neighbor results on Caltech-101 benchmark
Feature
Accuracy(%)
BPLR + PHOG (Ours)
61.1
Dense + SIFT
55.2
Segment + PHOG
37.6
Dense + PHOG
27.9
Comparison of features using the same Naïve Bayes NN [Boiman et al. 2008] classifier.

19. Conclusion Dense local detector that preserves object boundaries
Capture object’s local shape in a repeatable manner
Feature sampling and linking driven by segmentation
Generic bottom-up extraction
Code available: