1 Color-Based Image Salient Region Segmentation Using Novel Region Merging Strategy IEEE Transaction on Multimedia 2008 Yu-Hsin Kuan, Chung Ming Kuo, and Nai-Chung Yang 授課教授連震杰 教授 指導教授吳宗憲 教授 實驗室多媒體人機通訊實驗室 組員 P 林仁俊 P 魏文麗 P 劉家瑞

2 Outline Introduction Related work The proposed method  Dominant color extraction and image quantization  Region merging strategy Experimental results Conclusion

3 Introduction

4 Introduction (cont.) Color images are extensively used in multimedia applications (retrieval, index). Low-level visual features such as color, shape, texture (i.e. global features) has received much attention in recent years.  Retrieve too many unrelated images  Performances are unsatisfactory

5 Introduction (cont.) High-level semantic descriptors (object, scene, place) should be more consistent with human perception. How to narrow down the gap between low- level features and human perception?  Use spatial local features instead of global features of images The main purpose of this paper  Find the salient regions that are relatively meaningful to human perception

6 Related work What is salient region?  It should be compact, complete and significant enough. (a)(b)

7 Related work (cont.) Region-based methods [1]  To depend on initial seeds  Over-segmentation Boundary-based methods [2]  Noise, unconnected edges  Over-segmentation Hybrid –based methods [3]  Integrate the region and edge information  Enhance the drawbacks

8 Related work (cont.) Histogram-based methods [4]  Generally deal with gray-level images  Color images represented by 3-D histogram  Select a global threshold or dominant color in 3-D space is difficult Graph-based methods [5]  By minimizing the weight that cut a graph into sub-graphs  High computational complexity

9 The proposed method

10 The proposed method (cont.) Dominant color extraction and image quantization  The dominant colors are extracted based on nonparametric density estimation Kernel Density Estimator X is sample data N is the total pixel number of image σis the bandwidth for the kernel

11 The proposed method (cont.) 3 local maxima

12 Dominant color extraction ─ Q&A Question 1:  Why Gaussian smoothing Answer :  避免過多的 local maxima (too many dominant color – over segmentation) Question 2:  的改變，對 histogram 的影響 ? Answer :  會有 over smoothing ，或是不夠 smooth 的情形發 生

13 The proposed method

14 The proposed method (cont.) Dominant color extraction and image quantization 1 Y1Y1 U1U1 V1V1 2 Y1Y1 U1U1 V2V2 3 Y1Y1 U1U1 V3V3 4 Y1Y1 U2U2 V1V1 5 Y1Y1 U2U2 V2V2 6 Y1Y1 U2U2 V3V3 7 Y2Y2 U1U1 V1V1 8 Y2Y2 U1U1 V2V2 9 Y2Y2 U1U1 V3V3 10 Y2Y2 U2U2 V1V1 11 Y2Y2 U2U2 V2V2 12 Y2Y2 U2U2 V3V3 Dominant colors

15 The proposed method (cont.) Dominant color extraction and image quantization 1 Y1Y1 U1U1 V1V1 2 Y1Y1 U1U1 V2V2 3 Y1Y1 U1U1 V3V3 4 Y1Y1 U2U2 V1V1 5 Y1Y1 U2U2 V2V2 6 Y1Y1 U2U2 V3V3 7 Y2Y2 U1U1 V1V1 8 Y2Y2 U1U1 V2V2 9 Y2Y2 U1U1 V3V3 10 Y2Y2 U2U2 V1V1 11 Y2Y2 U2U2 V2V2 12 Y2Y2 U2U2 V3V3 1 Y1Y1 U1U1 V1V1 2 Y1Y1 U1U1 V2V2 3 Y1Y1 U2U2 V1V1 4 Y1Y1 U2U2 V2V2 5 Y2Y2 U1U1 V1V1 6 Y2Y2 U1U1 V2V2 7 Y2Y2 U2U2 V1V1  It may cause too many candidates of dominant colors.  We eliminate the candidates that the image pixels assignment is lower than a pre-defined threshold.

16 The proposed method (cont.) Source image Quantized image

/10/7 The proposed method (cont.)

18 The proposed method (cont.) Region merging strategy  Important index computation The number of pixels Total number of pixels with color label i Total number of pixels of an image (image size) A region with color label i, Region index j a a a b b c

19 The proposed method (cont.) Region merging strategy  Threshold : Segmentation result Merge into an adjacent region

20 The proposed method (cont.)

21 The proposed method (cont.) Region merging strategy  Attraction computation  Assume is a region to be merged and are its neighboring regions

22 The proposed method (cont.) Region merging strategy  Attraction computation

23 The proposed method (cont.) Initial regionAfter region merging strategyFinal segmentation result

24 Experimental results Two parameters need to be preset  Bandwidth of the convolution kernel  Merge threshold For CIF format images, the average speed is around 0.6 second for each image  Pentium 4 PC, 2.66 GHz CPU with 512MB RAM

25 Experimental results (cont.) (a)Source image (b) After quantized and region merge (c) Segmentation result (a)(c)(b)(c)(a)(b)

26 Experimental results (cont.) Source imageOver-segmentation[25] Our method Source imageOver-segmentation[25]Our method D. Comaniciu and P. Meer, “Robust analysis of feature spaces: color image segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997

27 Conclusion The proposed approach efficiently extracts salient regions in color images. Segmentation results satisfied our definition of saliency. Effectively addressed the over-segmentation problem.

28 Reference [1] M. G. Montoya, C. Gil, and I. Garcia, “The load unbalancing problem for region growing image segmentation algorithms,” J. Parallel Distrib. Comput., vol. 63, pp. 387–395, 2003 [2] W. Y. Ma and B. S. Manjunath, “Edge flow: a technique for boundary detection and image segmentation,” IEEE Trans. Image Process., vol. 9, no. 8, pp. 1375–1388, Aug [3] T. Gevers, “Adaptive image segmentation by combining photometric invariant region and edge information,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 6, pp. 848–852, Jun [4] H. D. Cheng, X. H. Jiang, and J. Wang, “Color image segmentation based on homogram thresholding and region merging,” Pattern Recognit., vol. 35, pp. 373–393, Feb [5] A. Tremeau and P. Colantoni, “Regions adjacency graph applied to color image segmentation,” IEEE Trans. Image Process., vol. 9, pp. 735–744, Apr. 2000

29 Thank you~