Extraction of Salient Contours in Color Images Vonikakis Vasilios, Ioannis Andreadis and Antonios Gasteratos Democritus University of Thrace

Presentation Overview  Problem definition  Biological background  Description of the model  Results  Conclusions

Definitions: Salient Contours Salient contours: The most evident contours that draw the attention of an observer Problem definition

Applications of salient contours  Create the ‘primal sketch’ of the image  Filter the optical data and keep only the significant information  Reduce the amount of visual information that a visual system processes Problem definition

The Human Visual System Biological background Retina Visual Cortex V1, V2… Optic nerve light (ganglion cells)

Double opponent cells Biological background  They are located in area V1  Two chromatic and one achromatic  They have a center-surround receptive field  They receive opposite signal to center and surround  They respond only to changes between center and surround – edges detectors R+G-B B-R-G G-R R-G -R-G-B R+G+B Blue-Yellow Red-Green Black-White

The primary visual cortex V1 Biological background  The visual cortex analyses the retinal output in 3 different maps: 1.Color (double-opponent cells) 2.motion-depth 3.orientation of edges  At every position of the visual field, the V1 has cells (orientation filters) of all possible orientations

Favorite connections of a horizontal orientation cell Biological background  Orientation cells prefer to be connected with others that create co-circular paths  This favors the smooth continuity of contours Connection of orientation cells

Block diagram of the model Description of the model Input Image Extraction of color edges Salient Contours network

Extracting color edges Description of the model Center Surround 9x9 mask  Similar to the double- opponent cells of V1

Extracting color edges Description of the model max { RG, BY, BW } RG BY BW

Orientation filters Description of the model 60 kernels 10×10 pixel size 12 orientations all possible positions within every orientation  The image is divided to 10×10 non-overlapping regions  For every region all 60 kernels are convolved  The higher response defines the kernel that best describes the orientation of the region Objective: to encode the orientation of the edges

Encoded edges Description of the model Color edge image Image with oriented filters Kernel 24:75° Kernel 19:135°

Computing the connection matrix Description of the model  We have calculated the connection matrix of all the 60 kernels, in a 5×5 kernel neighborhood Kernel 6 Kernel 17 Kernel 54 Connection matrix: weight [60] [5] [5] [60] weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the to kernel ‘n’ in the (i,j) position weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the 5×5 region, to kernel ‘n’ in the (i,j) position Connection matrix: weight [60] [5] [5] [60] weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the to kernel ‘n’ in the (i,j) position weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the 5×5 region, to kernel ‘n’ in the (i,j) position i j m n n n n n n n n n n n

Influence between kernels Description of the model Basic influence equation of kernel m (i,j) to kernel n out n (t) = out n (t-1) + weight m(i,j)→n × out m (t)  If weight m(i,j)→n >0 (kernel m is in the favorite curves of n) the influence is excitatory (out n (t)>out n (t-1))  If weight m(i,j)→n <0 (kernel m is not in the favorite curves of n) the influence is inhibitory (out n (t)<out n (t-1))

Activation function of kernel k Description of the model  Only the kernels with equal excitation in both lobes achieve high output  This favors the good continuation of salient contours F Lobe 1 Lobe 2  L1: total excitatory influence to Lobe 1  L2: total excitatory influence to Lobe 2  inh: total inhibitory influence k

Iterations Description of the model Oriented filters t=0 t=1 t=9 t=19  Salient contour kernels gradually increase their values  Kernels of non-salient contours gradually decrease their values  Usually 10 iterations are necessary

Results Original image Color edges Salient contours 700×700: 2.7 sec 700×576: 2.3 sec

More results Results Original image Color edges Salient contours 1000×768: 4.8 sec

More results Results Original image Color edges Salient contours 500×750: 2.1 sec 672×496: 1.8 sec

Conclusions  The proposed extraction of edges exhibits better results, especially for isoluminant areas, than the gradient of R,G and B  The proposed kernel set is an adequate way of coding the orientation of edges  The proposed method successfully extracts some of the most salient contours of the image  The execution time of the method when executed by a conventional PC is small compared to other saliency algorithms in the field

Thank you!