1. Paper Presentation: Shape and Matching
Aashish Thite, Qinyuan Sun, and Dongxi Zheng
11/28/2012

2. Overview Problems to solve Inter-paper relationship Paper 1 Paper 2
Summary

3. Problems to Solve
Are two objects similar? If yes, how similar are they?
Are there similar shapes in distinct pictures?

4. Problems to Solve
How to match two sets of features of logically similar objects efficiently and effectively? (RANSAC is slow and not effective)

5. Problems to Solve Scene (or object) recognition?
How to recognize this scene as an office?
How to recognize this animal as a horse?

6. Overview Problems to solve ✔ Inter-paper relationship Paper 1 Paper 2
Summary

7. Inter-paper relationship
Paper 1 (PAMI 2002):
Shape Matching and Object Recognition Using Shape Contexts
Paper 3 (ICCV 2005):
The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features
Paper 4 (CVPR 2006):
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Paper 2 (CVPR 2007):
Matching Local Self-Similarities across Images and Videos

8. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1
Summary

9. Shape Matching and Object Recognition Using Shape Context
By Serge Belongie, Jitendra Malik and Jan Puzicha
PAMI 2002

10. Descriptor: Shape Context

11. Advantage of Shape Context
A rich descriptor
Scale invariant
Translation invariant
Robust under small geometrical distortion
Can be made rotation invariant

12. Cost Between Matching Two Points

13. Bipartite Graph Matching
Minimize
Add dummy nodes to each point set with constant matching cost to handle outliers and square the points in two shapes

14. Modeling Transformation
Thin plate spline
minimizing the bending energy

15. Some More Constraints Regularization Parameter Transformation
Control the amount of smoothing
Transformation

16. Shape Distance Shape Context Distance Image Appearance Distance
Bending Energy

17. Illustration

18. Choosing Prototypes What example to store as prototypes?
A novel editing algorithm based on shape distance and K-medoids.

19. Trademark Retrieval Examples

20. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1 ✔
Summary

21. Matching Local Self-Image Similarity Across Image and Videos
by Eli Shechtman and Michal Irani
CVPR 2007

22. Local Self-similarity Descriptor
Correlation Surface
Image patch 5x5
Surrounding image region 40 pixels
varnoise photometric variations
varauto maximum variance of
the difference of the patches
within a small neighborhood

23. Local Self-similarity Descriptor (cont.)
Log-polar histogram
20 angles, 4 radial intervals
maximum correlation for each bin
normalize to [0..1]

24. Properties of Descriptor
Local descriptor
log-polar representation
account for local affine deformation
Allow for non-rigid deformation
Contain rich information
No explicit segmentation
Handle regions without clear boundary

26. Global Ensemble of Descriptors
Compute local self-similarity densely
computed on both F and G
5 pixels apart
All Local descriptor in F form ensemble of descriptors

27. Matching Global Ensembles of Descriptors
Filter out non-informative descriptors
No local self-similarity
High self-similarity (eg. large homogeneous region)
Efficient ensemble matching algorithm from another paper
Generates a dense likelihood map
Done in multi-scale
Scale invariant

28. Applications Objection Detection Image Retrieval by sketching
Action Detection in video

29. Image Retrieval Results

31. Action Detection Result

32. Comparison with other descriptor

33. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1 ✔
Summary

35. Bag of Words

37. Image Representation

48. Summary: Pyramid match kernel
• Linear time complexity
• Insensitive to clutter
• Positive-definite function

49. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1 ✔
Summary

50. Svetlana Lazebnik, Cordelia Schmid, Jean Ponce, 2006
Many of the Slides by Svetlana Lazebnik:

51. Key Idea •Pyramid Match Kernel (Grauman & Darrell)
Pyramid in feature space, ignore location
•Spatial Pyramid (this work)
Pyramid in image space, quantize features

52. Quantize Feature Space

53. Spatial Histograms

54. Spatial Histograms

57. Spatial Matching Kernel
Train an SVM using this Kernel

59. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1 ✔
Summary

60. Summary
An effective descriptor for features around a shape should be informative, e.g., shape context (paper 1), self-similarity descriptor (paper 2).
For shape or scene recognition, the features in two images typically have no 1-on-1 mapping relation, not even the same amount. Multi-level histograms (paper 3 and 4) are an efficient and effective matching approach.

61. Overview Problems to solve ✔ Inter-paper relationship ✔ Paper 1 ✔
Summary ✔
Questions

62. Thank You!