Organizing a spectral image database by using Self-Organizing Maps Research Seminar Oili Kohonen

Motivation? Image retrieval from conventional databases since 1990's... many efficient techniques have been developed However, efficient techniques for querying images from spectral image database does not exist. Due to the high amount of data in the case of spectral images, the efficient techniques will be needed.

Spectral imaging? Metameric imaging: cheap and practical way to achieve a color match. Spectral imaging: needed to achieve a color match for all observers across the changes in the illumination.

Principle of SOM: The Self-Organizing Map (SOM) algorithm: Is an unsupervised learning algorithm. Defines mapping from high-dimensional data into lower-dimensional data. SOM: Consists of arranged units (or neurons), which are represented by weight vectors. Units are connected to each other by neighborhood relation.

Principle of SOM: SOM Algorithm: begin Initialize the SOM for i = 1 : number of epochs take input vector x randomly from the training data; find the BMU for x; update the weight vectors of the map; decrease the learning rate & neighborhood function; end;

Principle of SOM: finding the BMU Mathematically the BMU is defined for input data vector, x, as follows: Euclidean distance is a typically used distance measure.

Principle of SOM: updating the weight vectors Learning rate: product of learning rate parameter & neighborhood function:

Principle of SOM: neighborhood function Neighborhood function h(t) has to fullfill the following two requirements: It has to be symmetric about the maximum point (BMU). It's amplitude has to decrease monotonically with an increasing distance from BMU. Gaussian function is a typical choice for h(t)

Principle of SOM: Lattice structure Lattice structures: hexagonal & rectangular

Searching Technique: Constructing histogram database Train SOM Find BMU for each pixel in an image Generate BMU-histogram & normalize it by the number of pixels in an image Repeat steps 2 & 3 for all images in a spectral image database Save histogram database with the information of SOM-map

Searching Technique: making a search Choose an image and generate its histogram. Calculate the distances between the generated histogram and the existing histogram database. Order images by these distances. The results of the search are shown to user as RGB-images

Searching techniques: One-dimensional SOM:

Searching techniques: Two-dimensional histogram-trained SOM

Distance Calculations: H1 & H2 are the compared histograms L1 & L2 are the indices of max. values| H3=(H1+H2)/2

Experiments: One-dimensional SOM for unweighted images One-dimensional SOM for images weighted by HVS-function Two-dimensional SOM From histogram data From spectral data Human Visual Sensitivity-function (Unweighted images) (Unweighted and weighted images)

The Used Database: 106 images: 61 components, spectral range from 400 nm to 700 nm at 5 nm interval.

Training of the SOMs: spectra were selected randomly from each image & epochs in ordering & fine tuning phases, respectively. Unit sizes: 50 – chosen empirically 49 – to have comparable results with 1D-SOM 14*14 map in the case of histogram-trained SOM

Results: 1d-SOM, Unweighted images Pure data Multiplied data The distance measure: Euclidean distance

Results: 1D, Unweighted images Energy K-L Peak DPD JD

Results: 1D, Weighted images Energy K-L Peak DPD JD

Conclusions I: The “structure” of the database is different for weighted and unweighted images. The “best” results were got by using euclidean distance and Jeffrey divergence. Importance of normalization?? * Better results with Euclidean distance & DPD * Worse results with Jeffrey divergence

Results: 2D, Unweighted spectral data Euclidean Energy K-L Peak DPD JD

Results: 2D, Weighted spectral data Euclidean Energy K-L Peak DPD JD

Conclusions II: In the case of two-dimensional SOM better results are achieved by using non-weighted images. When the weighted images are used, the use of 1D- SOM seems to be more reasonable.

Results: histogram-trained 2D-SOM Euclidean Energy K-L Peak DPD JD

Connections between images and histograms: non-weighted weighted

Past, Present & Future: Past: What you have seen so far... Present: Texture features in addition to color features Future: Testing the effect of different metrics in ordering and fine-tuning phases (during the training of SOM)

Questions: ? Thank you for not asking any... =)