Svetlana Lazebnik, Cordelia Schmid, Jean Ponce Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories Svetlana Lazebnik, Cordelia Schmid, Jean Ponce Presented by: Lubomir Bourdev Many of the slides by: Svetlana Lazebnik

Key Idea Pyramid Match Kernel (Grauman & Darrell) Pyramid in feature space, ignore location Spatial Pyramid (this work) Pyramid in image space, quantize features

Algorithm Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM

Algorithm OR Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM OR Weak (edge orientations) Strong (SIFT)

Algorithm Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM Vector quantization Usually K-means clustering Vocabulary size (16 to 400)

Algorithm Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM

Algorithm Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM

Algorithm Extract interest point descriptors (dense scan) Construct visual word dictionary Build spatial histograms Create intersection kernels Train an SVM

My experiment: Butterfly Classification Peacock Zebra

Butterflies Dataset from Lazebnik / Schmid / Ponce 70 train / 64 test Images centered on the butterfly Significant background clutter Large pose/viewpoint variations Scale variations: up to x4

Butterfly Results Spatial pyramid levels: 1 (No pyramid) Linear Intersection Weak (16) 82.6% Strong (200) 81.9% 89.5% Dims 16 200 Spatial pyramid levels: 4 Linear Intersection Weak (16) 88.6% 86.7% Strong (200) 84.8% 89.5% Dims 1360 17000