IDEAL 2005, 6-8 July, Brisbane Multiresolution Analysis of Connectivity Atul Sajjanhar, Deakin University, Australia Guojun Lu, Monash University, Australia Dengsheng Zhang, Monash University, Australia Tian Qi, I 2 R, Singapore

IDEAL 2005, 6-8 July, Brisbane Outline Background of Traditional Approach Motivation of Coherent Features Coherence Features in Contour Shapes Connectivity in Region Shapes Multiresolution Shape Analysis Principal Component Analysis of Shape Features Experiment Results Conclusions

IDEAL 2005, 6-8 July, Brisbane Background CBIR—content based image retrieval, using low level features such as color, texture and shape Shape—one of the important image features  Contour based approach  Region based approach Existing techniques ignore the geodesic feature of the shape which is an important information We attempt to employ coherence feature in shape to represent shape more accurately

IDEAL 2005, 6-8 July, Brisbane Traditional Histogram Approach Consider contour based shape retrieval  Shape is first abstracted by the centroidal distance function which is the distance from the centroid to the points on the boundary.  The centroidal distances are discretised into buckets  The values of the buckets are normalised as the shape features  Distance between shapes is computed as the sum of differences between corresponding buckets.

IDEAL 2005, 6-8 July, Brisbane Traditional Histogram r N-1 r1r1 r0r0 riri r i+1 r0r0 (x c, y c ) r N-1 (b) (a)

IDEAL 2005, 6-8 July, Brisbane Motivation of Coherent Features Drawback of traditional histograms is the lack of spatial continuity information, as a result, shapes which are completely different may have the same histograms An analogy is drawn from Color Coherence Vectors (CCV) proposed by Pass and Zabih. CCV is used for image retrieval based on colour: Pixels are classified as coherent or incoherent. Coherent pixels are part of a sizable contiguous region of similar colour while incoherent pixels are not.

IDEAL 2005, 6-8 July, Brisbane Coherence In the case of shape representation, we define “distance coherence” of pixels on the shape boundary. If the centroidal distance of a sample point on the shape boundary is part of a segment in which all the sample points have the same discretised centroidal distance then it is coherent. On the other hand, if the discretised centroidal distance of a sample point is different from the discretised centroidal distances of it’s immediate neighbours then it is incoherent.

IDEAL 2005, 6-8 July, Brisbane Coherence Based Histogram Each bucket is decomposed into coherent and incoherent

IDEAL 2005, 6-8 July, Brisbane Limitation Coherence can be conveniently defined for contour based shape retrieval However, it cannot be directly applied to region based shape like the image below.

IDEAL 2005, 6-8 July, Brisbane Connectivity In the case of region based shape, we define “connectivity” of pixels in an image. The state of the nearest 8-neighbours is computed for each dark pixel. Connectivity of a dark pixel is obtained as the number of dark pixels amongst the nearest 8-neighbours. Decomposition by Connectivity

IDEAL 2005, 6-8 July, Brisbane Distance Histogram of Region Shapes A distance function is computed by finding the number of shape pixels on concentric circles centered at the shape centroid The distance function is then discretised into buckets to obtain the distance histogram The similarity between shapes is determined by the sum of differences between corresponding buckets.

IDEAL 2005, 6-8 July, Brisbane Connectivity Based Histograms

IDEAL 2005, 6-8 July, Brisbane Multiresolution Analysis Shapes can be corrupted by noise Require multiresolution analysis

IDEAL 2005, 6-8 July, Brisbane Multiresolution Analysis Multiple resolutions of each image is obtained using Gaussian filters. Each resolution of the image is decomposed by connectivity. The family of 3d-histograms is obtained for each image. The process of obtaining histograms is preceded by pixel intensity thresholding. Pixel intensity thresholding is especially useful for fine shapes which would otherwise have significant rates of change in histogram densities. Principal components are obtained for the histogram family and used as an index for each image. During querying, the similarity between two images is determined by the L2-difference of their principal components

IDEAL 2005, 6-8 July, Brisbane Multiresolution Images Multiple resolutions are obtained by using Gaussian filters. Family of histograms is obtained for multiple resolutions of an image.

IDEAL 2005, 6-8 July, Brisbane Multiresolution Distance Histogram It is observed that the rates of change of histograms is high for fine edges. Hence, it is important to use pixel intensity thresholding.

IDEAL 2005, 6-8 July, Brisbane PCA Principal Component Analysis (PCA) is a statistical approach for reducing the dimensionality of data. We apply PCA to feature vectors and the resulting low dimensional data is used to index images. Singular value decomposition for the data matrix A is computed as U S V'. Where V is consisted of the eigenvectors of A'xA and U is consisted of the eigenvectors of AxA'. The eigenvalues obtained from the products of AxA' and A'xA, when square-rooted, make up the columns of S. The diagonal of S is said to be the singular values of the original matrix, A. Each eigenvector described above represents a principle component.

IDEAL 2005, 6-8 July, Brisbane PCA In the example shown, the first eigenvalue for the first eigenvector is 1 = The other eigenvalues are 2 = 5.3 and 3 = Thus, the first eigenvector contains almost all the energy. The data could thus be approximated in one-dimension. The first eigenvector (P1) has the largest eigenvalues to the direction of the largest variance. The second (P2) and the third (P3) eigenvectors are orthogonal to the first one.

IDEAL 2005, 6-8 July, Brisbane Experiment Dataset is MPEG-7. It consists of Traditional approach – 2d histograms are obtained for each image Proposed approach – 3d histograms are obtained for each image Principal components are obtained for the histogram family and used as an index for each image. During querying, the similarity between two images is determined by the L2-difference of their principal components

IDEAL 2005, 6-8 July, Brisbane Results

IDEAL 2005, 6-8 July, Brisbane Conclusions A set of new region based shape feature is proposed based on coherence distance histogram Multiresolution image analysis and PCA feature analysis have been used to obtain the accurate and compact shape features Experiment results show the new shape features outperforms the traditional distance histogram approach In future, the technique will be applied to gray level or color images