1. Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1

2. Outline  Introduction  Method Overview  Segment Categorization  Segment Post-Processing  Experiment  Conclusion 2

3. Introduction  Object detector : Top-down approaches 3

4. Introduction  Semantic segmentation results produced by our algorithm 4

5. Introduction  Ideally segment 1. Can model entire object 2. At least sufficiently distinct parts of them 5

6. 6

7. Introduction  Constrained Parametric Min Cuts algorithm (CPMC) [6] 7

8. Method Overview This paper focus CPMC Number of segment 8

9. Method Overview  Recognition framework 1. Segment categorization 2. Segment post-processing 9

10. Segment categorization 1. Scoring function 2. Sort 3. Combine high-rank segment 10

11. 11

12. Segment post-processing COW 12

13. Segment Categorization  Multiple Segment and Features  Learning Scoring Functions with Regression  Learning the Kernel Hyperparameters  Compare with Structural SVM 13

14. Multiple Segment and Features  Model object appearance ： 1. Extracted four bag of words of SIFT 2. Two on foreground 3. Two on Background, aim to improve recognition 14

15. Multiple Segment and Features  Encode shape information ： 1. A bag of word of local sharp contexts [2] : measure similarity between shapes 2. Three pyramid HOGs [5] : classifying images by the object categories they contain 15

16. Multiple Segment and Features  Chi-square kernel ： Computed from each histogram feature and use a weighted sum of such kernel for regression 16

17. Learning Scoring Functions with Regression  : Image I with ground truth segments  : Segmentation algorithm provides a set of segment : Denote the K object categories : Indicator function 17

18. Learning Scoring Functions with Regression  Quality function :  Measure overlap with all denote the value for and is the maximal overlap with ground truth segments belonging to, and do not appear 18

19. Learning Scoring Functions with Regression  Learn the function for each : use nonlinear SVR(Support Vector Regression) to regress against,the features extracted from 19

20. Learning Scoring Functions with Regression  Use kernel trick : : support vector from training set : obtained by the SVR optimizer  : maximal score of the segment  : final class of the segment 20

21. Learning the Kernel Hyperparameters  Fundamental equation (3) is infeasible to estimate all kernel hyperparameter via grid search  Use subset of data comprised segments that best overlap each ground truth segment 21

22. Compare with Structural SVM  The structural SVM(in [3]) formulation for sliding window prediction is : 22

23. Connections with Structural SVM  Our algorithm VS Structural SVM Structural SVM score the bounding box and Our algorithm score the segment  Important advantage 1. Guarantee the highest rank for the ground truth 2. Correct ranking for all segment 23

24. Segment Post-Processing  Simple decision rule : avoid the post- processing and direct choose the segment, cannot detect multiple objects  Our methodology : weighted consolidation of segment and sequential interpretation strategy 24

25. Segment Post-Processing 25

26. Segment Post-Processing  To decide which segments to combine  Consider segment with intersection > 0.75 for combination 26

27. Segment Post-Processing 1. Highest-scoring segment as seed 2. Group segments that intersect it 3. Generated a final mask 4. Proceed with the next higher rank segment 5. Choose segment that are not overlapping with 3 27

28. Segment Post-Processing  Generate the score for the pixels in the mask by (9), only pixels with score > 0.65 are displayed in the mask. 28

29. Experiments  Classification : Caltech-101  Detection : ETHZ Shape classes  Segmentation : VOC 2009 29

30. Classification 30

31. Classification 31

32. Detection 32

33. Detection 33

34. Detection 34

35. 35 OWT-UCM Masks

36. Segmentation Bounding box 36

37. Conclusion  CPMC Categorization Post- processing  Achieve good performance  Future work : improve the scalability to be able to process hundreds of thousands of image 37