Object Segmentation Based on Multiple Features Fusion and Conditional Random Field CASIA_IGIT National Laboratory of Pattern Recognition(NLPR) Institute of Automation, Chinese Academy of Sciences(CASIA) Reporter：Kun Ding（丁昆） 2013.10.17

Outline System Overview System Characteristics  Results and Conclusions

Outline System Overview System Characteristics  Results and Conclusions

System Overview Object Segmentation Pipeline Stage 1 : Superpixel Segmentation Feature Extraction SVM Classification GrabCut Feature Engineering Input Image Superpixels Features Probabilistic Output Final Results Stage 1 : Superpixel Classification Stage 2 : Pixel-based CRF Smoothing

System Overview Superpixel Classification Superpixel Segmentation Graph-based image segmentation Feature Extraction: To be detailed in next section SVM Classification[1] RBF kernel with Probabilistic Output

System Overview Pixel-Based CRF Smoothing Fusing several kinds of information as data term Solving with GrabCut with only a few iterations First Iteration Second Iteration Binarize SVM Probabilistic Output CRF Smoothing Output

Outline System Overview System Characteristics  Results and Conclusions

System Characteristics Superpixel Segmentation -- Efficient Graph-Based Image Segmentation[2] Fast, property of edge-preserving Speeding up the whole procedure Improving the separability between foreground and background Superpixels and their edge-preserving property

System Characteristics Feature Engineering – Superpixel-Based Multiple Features Fusion Gradient Dense SIFT[3][4] dictionary with Bag-of-Words description Texture Multi-scale LBP histogram Color and skin RGB histogram and HS histogram with skin detection Density of rectangles returned from AdaBoost as probability, with manifold ranking[6] refinement PCA Geometrical Position, direction and roundness Saliency Color spatial distribution, multi-scale local and global contrast Probability derived from AdaBoost, with manifold ranking[6] refinement Results of Object Detection

System Characteristics Feature Engineering – Superpixel-Based Multiple Features Fusion Illustration of object detection Object Detection result Rectangle Density as Probability Refined with Manifold Ranking

System Characteristics Pixel-Based CRF Smoothing – GrabCut[7] Modified data term Solving by maxflow iteratively SVM Result Object Detection Result GMM Result for Foreground and Background CRF Smoothing Output

Outline System Overview System Characteristics  Results and Conclusions

Conclusion and Results Exhibition

Conclusion and Results Exhibition Superpixel classification Feature fusion works CRF smoothing improves the results of SVM Object parts sometimes lost Context information is inadequate

Selected References [1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm. [2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2): 167-181. [3] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110. [4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of computer vision algorithms[C]//Proceedings of the international conference on Multimedia. ACM, 2010: 1469-1472.

Selected References [5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011, 33(2): 353-367. [6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173 [7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts[C]//ACM Transactions on Graphics (TOG). ACM, 2004, 23(3): 309-314.

Thank you very much! Any questions? CASIA_IGIT Leader: Ying Wang （王颖） Members: Kun Ding （丁昆） Huxiang Gu （谷鹄翔） Yongchao Gong （宫永超） E-mails: {ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn