Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5, MAY 2014 Zhixiang Ren, Shenghua Gao, liang-Tien Chia, and Ivor Wai-Hung Tsang 1

Overview Introduction Related Work Proposed Method of Saliency Experiments For Saliency Detection Conclusion 2

Introduction 3

Visual Saliency  Measure to what extent a region attracts human attention. Potential Application  Adaptive compression, image retargeting, object detection 4

Introduction (cont.) Many saliency detection algorithms (pixel-grid) have been proposed  [36]-[38], [57], [68] Drawbacks Perform poorly in the images with large salient regions Suffer from the messy background, e.g. natural scenes. 5

Introduction (cont.) [17], [18] suggest that early feature like color, contrast, and orientation indirectly affect human attention  Human is attracted by objects not by individual pixels It is natural to work with those perceptually meaningful image regions in saliency detection  Concept of superpixel [54] 6

Introduction (cont.) Proposed work applies two existing techniques to improve saliency detection  Superpixel representation – Used to represent the input image  PageRank algorithm – Applied to propagate saliency among similar clusters and refine saliency map 7

Related Work 8

Saliency detection methods can be divided into two categories  Top-down method : Task-dependent and based on prior knowledge about the object and their interrelations  Bottom-up method : Hypothesis for saliency is that salient stimulus is distinct from its surrounding stimuli. (contrast) For bottom-up method, research usually focus on identifying those regions with high contrast. 9

Related Work (cont.) [38] proposed to determine the contrast by DoG [57] measured the likeness of a pixel to its surroundings by the local regression kernels [1] measured the saliency of each pixel by the difference between the feature of each pixel and mean of the whole image. [68] measured the global contrast with all the other pixels. [30] model both local and global contrast by taking the positional distance into account Most of approach represent the input image in pixel-grid manner, and these method may failed to detect the homogeneous and quite large salient objects 10

Related Work (cont.) 11

Proposed Method 12

Proposed Method Superpixel Extraction and Clustering Salient Region Detection Saliency Refinement With Propagation 13

Superpixel Extraction 14

Superpixel Extraction (cont.) Mean shift  Mean shift is a procedure for locating the maxima of a density function given discrete data sampled from that function. 15 Scale parameter

Superpixel Extraction (cont.) 16 m(x)

Superpixel Clustering After mean shift, every superpixel will obtain a unique RGB color GMM is introduced to cluster superpixels in RGB color space The RGB value of this superpixel will be set as a 3-D vector to represent the superpixel during GMM 17 R139 G160 B127

Superpixel Clustering (cont.) 18

Superpixel Clustering (cont.) 19

Superpixel Clustering (cont.) 20

Superpixel Clustering (cont.) 21

Superpixel Clustering (cont.) 22

Superpixel Clustering (cont.) 23

Superpixel Clustering (cont.) 24

Superpixel Clustering (cont.) 25

Salient Region Detection Idea : background has larger spread in spatial domain i.e., the more compact the clusters are spread, the more salient they will be [32] proposed compactness metric to evaluate the spread of cluster Inter-cluster distances defined as 26 Cluster Spatial Center

Refinement With Propagation In some situation, the perceptually meaningful regions are less than the cluster number. That is, some regions, which should belong to one cluster, will be grouped into several clusters. 27 R139 G160 B127

Refinement With Propagation (cont.) If one cluster is over-segmented into several subclusters, the compactness may be highly distorted. Thus PageRank algorithm is proposed to propagate saliency between similar clusters. Original PageRank algorithm Question : How the original PageRank come from? 28

Refinement With Propagation (cont.) Idea : A page linked by many pages with high PageRank receives high rank as well. Modified algorithm 29

Refinement With Propagation (cont.) 30

Experiment Result 31

Experimental DataSet and Compared Method Dataset  EPFL dataset [1], CMU dataset [4], MSRA dataset [46],Itti’s method (ITTI) [38] Method  Spectral residual method (SR) [37]  Graph-based saliency method (GB) [36]  Frequency-tuned method (FT) [37]  Method based on color and orientation distributions (COD) [32]  Region contrast method (RC) [12] (Region-based)  Context-based method (CB) [39] (Region-based) 32

Experiment Result 33

Experiment Result (cont.) Linear Correlation Coefficient 34

Experiment Result (cont.) 35

Conclusion 36

Conclusion The paper proposes a promising saliency detection approach, which can generate accurate saliency maps with well-defined object boundary. Mean shift, GMM are used to extract meaningful superpixel. Saliency value is refined as well with a modified PageRank algorithm. 37