1. Computer Vision Group University of California Berkeley Matching Shapes Serge Belongie *, Jitendra Malik and Jan Puzicha U.C. Berkeley * Present address: U.C. San Diego

2. Computer Vision Group University of California Berkeley Biological Shape D’Arcy Thompson: On Growth and Form, 1917 –studied transformations between shapes of organisms

3. Computer Vision Group University of California Berkeley Deformable Templates: Related Work Fischler & Elschlager (1973) Grenander et al. (1991) Yuille (1991) von der Malsburg (1993)

4. Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

5. Computer Vision Group University of California Berkeley Comparing Pointsets

6. Computer Vision Group University of California Berkeley Shape Context Count the number of points inside each bin, e.g.: Count = 4 Count = 10... FCompact representation of distribution of points relative to each point

7. Computer Vision Group University of California Berkeley Shape Context

8. Computer Vision Group University of California Berkeley Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs C ij [Jonker & Volgenant 1987]

9. Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

10. Computer Vision Group University of California Berkeley 2D counterpart to cubic spline: Minimizes bending energy: Solve by inverting linear system Can be regularized when data is inexact Thin Plate Spline Model Duchon (1977), Meinguet (1979), Wahba (1991)

11. Computer Vision Group University of California Berkeley Matching Example modeltarget

12. Computer Vision Group University of California Berkeley Outlier Test Example

13. Computer Vision Group University of California Berkeley Synthetic Test Results Fish - deformation + noiseFish - deformation + outliers ICPShape ContextChui & Rangarajan

14. Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

15. Computer Vision Group University of California Berkeley Terms in Similarity Score Shape Context difference Local Image appearance difference –orientation –gray-level correlation in Gaussian window –… (many more possible) Bending energy We used the weighted sum Shape context + 1.6*image appearance + 0.3*bending

16. Computer Vision Group University of California Berkeley Object Recognition Experiments COIL 3D Objects Handwritten Digits Trademarks FExactly the same algorithm used on all experiments

17. Computer Vision Group University of California Berkeley COIL Object Database

18. Computer Vision Group University of California Berkeley Error vs. Number of Views

19. Computer Vision Group University of California Berkeley Prototypes Selected for 2 Categories Details in Belongie, Malik & Puzicha (NIPS2000)

20. Computer Vision Group University of California Berkeley Error vs. Number of Views

21. Computer Vision Group University of California Berkeley Handwritten Digit Recognition MNIST 60 000: –linear: 12.0% –40 PCA+ quad: 3.3% –1000 RBF +linear: 3.6% –K-NN: 5% –K-NN (deskewed) : 2.4% –K-NN (tangent dist.) : 1.1% –SVM: 1.1% –LeNet 5: 0.95% MNIST 600 000 (distortions): –LeNet 5: 0.8% –SVM: 0.8% –Boosted LeNet 4: 0.7% MNIST 20 000: –K-NN, Shape Context matching: 0.63%

22. Computer Vision Group University of California Berkeley

23. Computer Vision Group University of California Berkeley Trademark Similarity

24. Computer Vision Group University of California Berkeley Conclusion Introduced new matching algorithm matching based on shape contexts and TPS Robust to outliers & noise Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm