Content-Based Image Indexing Joel Ponianto Supervisor: Dr. Sid Ray

Outline Introduction to Content-Based Image Indexing Image’s Features Extraction Tree Structure System Model Retrieval Approach Experiment Results Conclusion

Introduction to Content-Based Indexing Content-Based Image Indexing (CBII) is an interrelated issue with Content-Based Image Retrieval (CBIR). CBIR depends on CBII and vice versa. CBIR focus on how to retrieve image accurately and efficiently. While CBII concern with how to support retrieval process.

Introduction to Content-Based Image Indexing Cont… CBiI as pre-process of CBIR sequences. Cannot ignore retrieval process to create good indexing structure. The idea of indexing is similar with a library Every book has a unique id Every book has properties

Introduction to Content-Based Image Indexing Cont… Examples: title, author, publisher, etc Those properties are used to search the book. People know it as “keyword” Similar idea with images, however not that simple. Cannot represent an image with simple text. (can but not make sense)

Introduction to Content-Based Image Indexing Cont… How to represent an image? By using its properties such as, colour, shape, texture and others. Choose which properties need to be extracted for indexing purpose ( and also retrieval). Also choose which method to extract those properties / features.

Image’s Features Extraction Cont… Colour, shape and texture have their own sub- features. Colour: grey level, RGB/HUE value, grey sigma, local histogram and average colour value. Shape: area, centroid, circularity and moment invariant. Texture: contrast, orientation and anisotropy.

Image’s Features Extraction Cont… The selection of features is also effected by the data set. what we want to achieve at the retrieval stage is effected by the data set. If the data set is full of houses’ image and a user want to look for a car image. Try to select features that can differentiate each class in the data set.

Image’s Features Extraction Cont… For this project I select the following features: – Colour Sigma (Global) – Edge density (Global) – Colour Average (Global) – Boolean edge Density (Global) – Edge Direction (Global) – Region area (Region) – Moment invariant (Region) – Grey level (Region)

Image’s Features Extraction Cont… Colour Sigma – Find the standard deviation (σ) of the image, for each colour layer.

Image’s Features Extraction Cont… Edge Density – Enhance the pixels that belong to the edges and boundaries by using a standard edge detector. Pixels far from edges will drop to 0 and those near to an edge will increase to max. calculate the mean pixel value of the resultant image. Colour Average – Sum all the pixel value for each colour layer and divide by the number of pixel.

Image’s Features Extraction Cont… Boolean Edge Density – From above edge density, the image is thresholded so that what could be called edge pixels are white (1) and non-edge pixels are black (0). Count white pixel in the image. Edge Direction – With some edge detection (Sobel Operator), allow us to make a crude estimation of a edge direction for particular region.

Image’s Features Extraction Cont… Area, Grey Value and moment invariant – These features is calculate on regional basis. – The region is calculated with combination of “k- mean clustering” and “Connected Component labelling Algorithm” – Calculate a grey level value of an image and perform the k-mean clustering. – Use the connectivity algorithm to group similar grey value by its location.

Image’s Features Extraction Cont… ments/momentEquations.html ments/momentEquations.html I use the first four of seven invariant moment for this project.

Image’s Features Extraction Cont…

Quantisation – To be suitable for computer processing and features extraction (colour), an image must be digitized in amplitude. – The idea is to reduce the colour space while gaining the ability to localize colour information spatially. – this project applies quantisation at HSV colour space.

Image’s Features Extraction Cont…

RGB to HSV – Let RGB values ranged from 0 to 1 and MIN/MAX corresponds with RGB values.

Image’s Features Extraction Cont… HSV to RGB – H range from – V and S range from 0 – 1 – If S == 0 then RGB = V Else use next formula

Image’s Features Extraction Cont…

Tree Structure There are many choices of tree structures that can handle multi-dimensional space. Such as R-Tree, R*-Tree and Vp-Tree We look at R-Tree tree structure: – This project used R-Tree to simplify the computation. – Other tree structures can be use on the system.

Tree Structure Cont… R-Tree (Antonin Guttman) – A R-Tree is a height balance tree and all leaves are on the same level. – Root node has at least two children unless it is the leaf node. – Every non-leaf node contains between m and M entries unless it is the root. – For each entries (I, childnode-pointer) in a non-leaf node, I is the smallest rectangle that spatially contains all rectangles in its child nodes. – Every leaf node contains between m and M index records unless it is the root. – For each index record (I, tuple-identifier) in a leaf node, I is the smallest rectangle that spatially contains the n-dimensional data object represented by the indicated tuple.

Tree Structure Cont…

System Model Put into data base Original Image Quantised Image K-mean clustering Binary ThresholdApply Global features extraction. Connected Component labelling Apply Region features extraction. Insert into tree structure

System Model Cont… The System input around 300 images into the data base. Those images is divided into 10 different classes: animal, car, flower, face, fruit, house, lake, mountain, plane and sunset. Store into persistence storage.

System Model Cont… In the “binary threshold” stage, I attempt to separate the background image with the object. Although this stage is very weak, but in some images. The result can be helpful (and possible the other way around).

System Model Cont… Binary Threshold good result

System Model Cont… Binary Threshold bad result

Retrieval Approach Query sequence Query Image Global Extraction Region Extraction Pre-process stage Find similarity with data set Display the result in ascending order

Retrieval Approach Cont… For finding similarity, I use Euclidean distance measure formula: Where: – p is the database image – q is the query image – P i is the database images i th features – Q i is the query’s i th features – n is the number of features – W” is the weight for i th feature W” i

Retrieval Approach Cont… w’ i is the weight of feature i from relevant images (σ i ) is the standard deviation of feature i from relevant images w’ t is the total weight of feature I w” t is the normalised weight

Retrieval Approach Cont… Gaussian Normalisation (for feature normalization): – d’(f i,f j ) is the similarity of image f i and f j, range in [-1, 1] – σ ij and μ ij are the standard deviation and mean of each feature respectively. – d”(f i,f j ) is to make d’(f i,f j ) in range [0, 1]

Experiment Result Go to Excel fileExcel file m1-m8 only use global features m3 uses colour avg, colour sigma and edge density m2 uses colour avg and colour sigma m8 uses colour sigma and edge density m9 use region features + m3

Conclusion Indexing depend on retrieval and vice versa No universal system / method for indexing or retrieval. We can try to develop something that robust. Indexing base on regional features give better result then global features. With more time, more result can be produced.

Reference Kompatsiaris, I., Triantafillou, E. and Strintzis, M. G., “Region-Based Color Image Indexing and Retrieval”, 2001 Parker, J. R., Behm, B., “Use of Multiple Algorithm in Image Content Searches”, International Conference on Information Technology: Coding and Computing (ITCC’04) Volume2 p.246. Smith, J. R., Chang, S., “Single Color Extraction and Image Query”, International Conference on Image Processing (ICIP-95), Washington, DC, Oct, Park, J. M., Looney, C. G., Chen, H. C., ”Fast Connected Component Labeling Algorithm Using A Divide and Conquer Technique”, Technical Report, 2000 Chiueh, T., "Content-Based Image Indexing," in Proceedings of International Very Large DataBase Conference, VLDB '94, Santiago, Chile, September, Gonzalez, R. C. and Woods, R. E., “Digital Image Processing”, 1993, Addison- Wesley Publishing Company, inc, 3 rd edition.