First Law Of Geography: "Everything is related to everything else, but near things are more related to each other.” He’s at UCSB -Waldo Tobler (1970)

Pyramids of Features For Categorization Presented by Greg Griffin Project Partner Will Coulter

Pyramids of Features For Categorization Presented by Greg Griffin Project Partner Will Coulter

Buckets of Features For Categorization Presented by Greg Griffin Project Partner Will Coulter

This talk is mostly about this paper: With a little bit about benchmarking: CALTECH 256

Images Features A Number What is this number?

Images Features A Number “How well do they match” Now get specific about features “How well do they match”

“Weak Features” Show you an implm of their wf

“Weak Features” (I think?) “…points whose gradient magnitude in a given direction exceeds a minimum threshold.” This is just their toy example They use SIFT descriptors as “Strong Features”. But you could use any features you want!

Images Features A Number Number that indicates matching

1 2 3 4 5 6 7 8

Images Features X1 X5 Y1 Y5

Images Features X1 X5 The point of this exercise Y1 Y5

Features X1 X5 Y1 Y5

Features Start By Matching Reds Progressively smaller cells, count matches

Features Start By Matching Reds 1 l is… D is… I is…

Features Start By Matching Reds 10 1 10 10

Features Start By Matching Reds 1 10 4

Features Start By Matching Reds 7 1 1 10 4 2 6 1 1 2

Features Start By Matching Reds 7 1 1 10 4 8 2 6 1 1 2

Features Start By Matching Reds 1 10 4 8 2 16

Features 5 2 1 2 1 3 2 1 1 1 1 Start By Matching Reds 1 10 4 8 2 16 6 1 10 4 8 2 16 6 2 1 Have table, throw out im,features 3 2 1 1 1 1

Start By Matching Reds 1 10 4 8 2 16 6

A compact set of matches is preferable to widely dispersed matches Start By Matching Reds 1 10 4 8 2 16 6 A compact set of matches is preferable to widely dispersed matches

Start By Matching Reds 1 10 4 8 2 16 6

Start By Matching Reds 1 10 4 8 2 16 6 For noise…

Features X vs. Purely Isotropic Y 1 10 4 5.5 2 16 1.9 A Sanity Check: Features X vs. Purely Isotropic Y

Start By Matching Reds, Then The Blues, Then… 1 10 4 8 2 16 6

1 2 3 4 5 6 7 8 M = 8

Features 1 10

Features 1 10 4 8

Features 1 10 4 8 2 16 6

Foreach feature M=1…m Foreach level = 0…L Foreach cell i=1…D

Training Set Test Set SL(X,Y) 3.4 15 Categories 5.6 7.8 office 1.5 store Disjoint! 5.4 coast street 100 Images per Category 100-300 Images per Category suburb

Confusion Matrix Train on 100 Test on 100-300 (per category)

Scene Database Caltech 101

Hypothesis Pyramid Matching works well when: Objects are aligned and localized ie. certain Caltech 101 categories biased by different values of Ntest? A few common features that define the category get randomly permuted through many positions, thanks to a large dataset ie. scene database now and then pyramid matching gets lucky example: Library books

Test How well will Pyramid matching work? Objects are not well aligned, or cluttered Caltech 256 is more challenging in this respect example: Library books

Scene Database Caltech 101

Cluster Matching example: Library books k-means of SIFT positions

More Flexible Than A Grid? k-means of SIFT positions + color

Position Invariance & Clutter Grid can cut across ducks’ chest etc. Alignment Any cluster can match any cluster Clusters respect duck / water boundaries (sort of)

Tackles Alignment Problem But… how to match efficiently? How many clusters? How big?

Summary Spatial Pyramid Matching is efficient and handles a range of scales, but seems to be sensitive to translation and clutter. Cluster Matching has the potential to improve translational invariance and tolerance of clutter. But inefficient. Less principled: how many clusters are optimal? How big should they be? No scale invariance. Can we have the best of both worlds?

Try Sliding Spatial Pyramids? Slide puzzle photo from: http://www.tenfootpolesoftware.com/products/slidepuzzle/