1. Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on Andrea Frome, EECS, UC Berkeley Yoram Singer, Google, Inc Fei Sha, EECS, UC Berkeley Jitendra Malik, EECS, UC Berkeley

2. Outline Introduction Training step Testing step Experiment & Result Conclusion

3. Outline Introduction Training step Testing step Experiment & Result Conclusion

4. What we do? Goal – classify an image to a more appropriate category Machine learning Two steps – Training step – Testing step

5. Outline Introduction Training step Testing step Experiment & Result Conclusion

6. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Input distances to SVM for training, evaluate W Compute distance dji, dki

7. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Input distances to SVM for training, evaluate W Compute distance dji, dki

8. Choosing features Dataset: Caltech101 Patch-based Features – SIFT Old school – Geometric Blur It’s a notion of blurring The measure of similarity between image patches The extension of Gaussian blur

9. Geometric blur

10. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Input distances to SVM for training, evaluate W Compute distance dji, dki

11. Triplet dji is the distance from image j to i It’s not symmetric, ex: dji ≠ dij dki > dji djidki

12. How to compute distance L2 norm 1 2 3 dji, 1 m features dji, 1 distance vector dji Image j Image i

13. Example Given 101 category, 15 images each category 101*15 Feature j 101*15 distance vector Image j vs training data

14. Flow chart: training Generate features each image from dataset, ex: SIFT or geometric blur Input distances to SVM for training, evaluate W Compute distance dji, dki

15. Machine learning: SVM Support Vector Machine Function: Classify prediction Supervised learning Training data are n dimension vector

16. Example Male investigate – Annual income – Free time Have girlfriend?

17. Ex: Training data

18. space free income vector

20. Mathematical expression(1/2)

21. Mathematical expression(2/2)

22. Support vector Model free income

23. But the world is not so ideal.

24. Real world data

25. Hyper-dimension

26. Error cut

27. SVM standard mathematical expression Trade-off

28. In this paper Goal: to get the weight vector W 101*15 feature Image weight wj of W wj, 1 wj

29. Visualization of the weights

30. How to choose Triplets? Reference Image – Good friend - In the same class – Bad friend - In the different class Ex: 101category, 15 images per category – 14 good friends & 15*100(1500) bad friends – 15*101(1515) reference images – total of about 31.8 million triplets

31. Mathematical expression(1/2) Idealistic: Scaling: Different: The length of Weight i 00 triplet

32. Mathematical expression(2/2) Empirical loss: Vector machine:

33. Dual problem

34. Dual variable Iterate the dual variables:

35. Early stopping Satisfy KTT condition – In mathematics, a solution in nonlinear programming to be optimal.mathematicsnonlinear programming Threshold – Dual variable update falls below a value

36. Outline Introduction Training step Testing step Experiment & Result Conclusion

37. Flow chart: testing Query an image i Output the most appropriate category Calculate Dxi, x is all training data, except itself.

38. Flow chart: testing Query an image i Output the most appropriate category Calculate Dxi, x is all training data, except itself.

39. Query image? Goal: classify the query image to an appropriate class Using the remaining images in the dataset as the query image

40. Flow chart: testing Query an image i Output the most appropriate category Calculate Dxi, x is all training data, except itself.

41. Distance function(1/2) Query image i Image i feature 101*15 distance vector Image i vs all training data dxi, 1

42. Distance function(2/2) 101*15 Image I vs all the training data Dji

43. Flow chart: testing Query an image i Output the most appropriate category Calculate Dxi, x is all training data, except itself.

44. How to choose the best image? Modified 3-NN classifier no two images agree on the class within the top 10 – Take the class of the top-ranked image of the 10

45. Outline Introduction Training step Testing step Experiment & Result Conclusion

46. Experiment & Result Caltech 101 Feature – Geometric blur (shape feature) – HSV histograms (color feature) 5, 10, 15, 20 training images per category

48. Confusion matrix for 15

49. Outline Introduction Training step Testing step Experiment & Result Conclusion

50. Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification