1 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Singular Value Decompositions with applications to Singular Value Decompositions with applications to 1. Texture differentiation 2. Detection of an extraneous object in a texture environment 3. Segmentation of images 4. Locating eyes in facial images Alireza Tavakoli Targhi Institute for Studies in Theoretical Physics and Mathematics (IPM), Iran and Royal Institute of Technology (KTH), Sweden

2 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Introduction We propose new measures for texture classification based on a local version of Singular Value Decomposition (SVD). The proposed measures classify textures by their roughness and structure. Experimental results show that these measures are suitable for texture clustering and image segmentation and they are robust to changes in local lighting, orientation etc.

3 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Singular Value Decomposition A=U 1 *D*U 2 U i Orthogonal Matrix, D Diagonal Matrix with Diagonal Entries in descending order: d 1 >d 2 > …>=0 Overview of SVD

4 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran To find the SVD curve of an arbitrary row of the texture: 1. Scan a row with overlapping w*w windows W a, a=1,2,3,….. ; w ≈ Calculate the SVD Decomposition 3. W a =U 1,a *D a *U 2,a 4. As windows scan the image we obtain w curves (  i ) corresponding to diagonal entries d 1,a >d 2,a >….>d w,a SVD Curves

5 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Classification Diagonal entries reflect image characteristics.

6 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran The sizes of the first few coefficients are considerably larger than the remaining

7 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran SVD Curve Classifiers We introduce two measures, obtained from SVD curves, which capture some of the perceptual and conceptual features in an image. SVD Curve mean classifier SVD Curve distance classifier

8 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran SVD Curve Mean Classifier Our experiments show that the smaller coefficients d a,j, i.e., da,j with 23<j<32, are more representative of the structure of the texture and less dependent on inessential features. In practice, we set l = 22 and k = 10.

9 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran

10 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran SVD-Distance Classifier

11 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Detection of Extraneous Object: Detection of Extraneous Object:

12 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Even Small objects

13 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Interest Point Detector

14 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran The technique identifies the bug even the location of its legs

15 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Added two coins

16 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran No Differentiation

17 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran SVD Transform We slide a w*w window across the image. We slide a w*w window across the image. We identify each window by its upper left corner coordinates (x,y). We identify each window by its upper left corner coordinates (x,y). Let F be a function of w variables. Let F be a function of w variables. The SVD surface is the graph of the function (x,y) → the value of F on D (x,y ). The SVD surface is the graph of the function (x,y) → the value of F on D (x,y ). The SVD transform is the representation of the of the SVD surface as a 2D image. The SVD transform is the representation of the of the SVD surface as a 2D image.

18 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran SVD Transform

19 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Original Image SVD Transform SVD surface

20 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

21 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

22 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

23 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

24 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

25 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

26 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Application of Feature Vector II SVD Transform, Segmentation

27 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

28 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Sensitivity to Texture

29 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Sensitivity to Texture The Berkeley Segmentation Data Base. Computer Vision Group.

30 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD Transform

31 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation via SVD transform The Berkeley Segmentation Data Base. Computer Vision Group.

32 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran The Berkeley Segmentation Data Base. Computer Vision Group.

33 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Effect of change of a parameter The Berkeley Segmentation Data Base. Computer Vision Group.

34 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Effect of change of a parameter

35 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation (cont.)

36 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation (cont.)

37 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation (cont.)

38 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Segmentation (cont.)

39 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Images show the effect of substituting the diagonal part or the orthogonal parts from the SVD decomposition of an image into that of another image. ws=5 Understanding SVD

40 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Images show the effect of substituting the diagonal part or the orthogonal parts from the SVD decomposition of Lena into that of a randomly generated image. ws=5 Understanding SVD

41 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Understanding SVD Images show the effect of substituting the diagonal part or the orthogonal parts from the SVD decomposition of an image into that of another image. ws=32

42 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Images show the effect of substituting the diagonal part or the orthogonal parts from the SVD decomposition of Lena into that of a randomly generated image. ws=32 Understanding SVD

43 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Invariance relative to inversion These images are negatives of each other. Their SVD surfaces are identical.

44 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran An SVD Surface

45 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Detecting cracks and defects

46 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Detecting cracks and defects

47 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Detecting cracks and defects

48 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Detecting cracks and defects

49 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Theoretical Framework We do not have a definitive answer why SVD works to the extent that it does. However, on the basis of our experimentations we can assert the following: We do not have a definitive answer why SVD works to the extent that it does. However, on the basis of our experimentations we can assert the following: 1. The diagonal entries of SVD capture some features of an image which are not encoded by the correlations of nearby pixels in an image. 1. The diagonal entries of SVD capture some features of an image which are not encoded by the correlations of nearby pixels in an image. 2. This may explain why images constructed on the basis local correlations virtually never exhibit features similar to ones in real images. Images constructed on the basis of Markov random fields or similar procedures generally look very random. 2. This may explain why images constructed on the basis local correlations virtually never exhibit features similar to ones in real images. Images constructed on the basis of Markov random fields or similar procedures generally look very random. 3. As the size of the sliding window increases the importance of the orthogonal parts of SVD decomposition relative to the diagonal part increases. 3. As the size of the sliding window increases the importance of the orthogonal parts of SVD decomposition relative to the diagonal part increases.

50 International Workshop on Computer Vision April 26-30, 2004 Tehran,Iran Work in Progress We are in the process of applying SVD for material/surface classification. We are in the process of applying SVD for material/surface classification. The SVD transform can be applied to the test case of differentiating between a dog and a cat. The results are preliminary and require further tests. The SVD transform can be applied to the test case of differentiating between a dog and a cat. The results are preliminary and require further tests. SVD transforms are also being tested on movie images. SVD transforms are also being tested on movie images. Our methods are being tested on medical images by IRMA (Image Retrieval in Medical Applications) of Institut fuer Medizinische Informatik Universitaetklinikum der RWTH in Aachen, Germany. Our methods are being tested on medical images by IRMA (Image Retrieval in Medical Applications) of Institut fuer Medizinische Informatik Universitaetklinikum der RWTH in Aachen, Germany.