2. Faculty of Computational Mathematics and Cybernetics, Moscow State University NUS, Singapore, August 14, 2003 Andrey S. Krylov

3. Outline: Projection Method (Hermite series approach)Projection Method (Hermite series approach) ApplicationsApplications 1. 1.Image filtering and deblocking by projection filtering 2. 2.Image matching 3. 3.Texture matching 4. 4.Low-level methods for audio signal processing

4. The proposed methods is based on the features of Hermite functions. An expansion of signal information into a series of these functions enables one to perform information analysis of the signal and its Fourier transform at the same time. Hermite transform

5. B) They derivate a full orthonormal in system of functions. The Hermite functions are defined as: A)

6. Original image 2D decoded image by 45 Hermite functions at the first pass and 30 Hermite functions at the second pass Difference image (+50% intensity)

7. Original image 2D decoded image by 90 Hermite functions at the first pass and 60 Hermite functions at the second pass Difference image (+50% intensity)

8. Image filtering and deblocking by projection filtering

9. Original lossy JPEG image

10. Enhanced image

11. Difference image (Subtracted high frequency information)

12. Original image Zoomed in: Enhanced image

13. Scanned image Enhanced image

14. Zoomed in: Scanned image Enhanced image

15. Image matching

16. Information parameterization for image database retrieval Algorithm Image normalizing Graphical information parameterization Parameterized image retrieval _

17. Information parameterization for image database retrieval Algorithm Image normalizing Graphical information parameterization Parameterized image retrieval =+ HF componentLF component Normalized image Recovered image with the recovered image plane

18. Information parameterization for image database retrieval Algorithm Image normalizing Graphical information parameterization Parameterized image retrieval Database size Initial images format Normalized images format Number of parameterization coefficients Error rate Search time – 768 images (4.12Gb) – 1600x1200x24bit (5.5Mb) – 512x512x24bit (0.75Mb) – 32x32x3 – <0.14% – 4 sec. (for K7-750)

20. Image matching results

22. Texture matching

24. A method of obtaining the texture feature vectors Input function Fourier coefficients 1-D to 2-D expansion

25. A method of obtaining the texture feature vectors 1-D Hermite functions: 1-D to 2-D expanded Hermite functions:

26. Orientations

27. Localization problem Decomposition process is optimal, if localization segments of the input function and filtering functions are equal.

28. Feature vectors In this approach to get the feature vectors we consider the functions  n (x,y) where n1=0..64, and 6 energy coefficients are calculated as: E 1 = (  0 ) 2 + (  1 ) 2, E 2 = (  2 ) 2 +(  3 ) 2 + (  4 ) 2, E 3 = (  5 ) 2 +(  6 ) 2 +(  7 ) 2 +(  8 ) 2, … E 6 = (  33 ) 2 +(  34 ) 2 + … + (  63 ) 2 +(  64 ) 2, f(x,y) is the source image. Standard coding

29. Feature vectors E 1 = (  0 (1) ) 2 + (  1 (1) ) 2, E 2 = (  0 (2) ) 2 +(  1 (2) ) 2 + (  2 (2) ) 2 +(  3 (2) ) 2, … E 6 = (  0 (6) ) 2 +(  1 (6) ) 2 + … + (  62 (6) ) 2 +(  63 (6) ) 2, f(x,y) is the source image. Hierarchical coding

30. Feature vectors E 1 = (  0 (1) ) 2 + (  1 (1) ) 2, E 2 = (  0 (2) ) 2 +(  1 (2) ) 2 + (  2 (2) ) 2 +(  3 (2) ) 2, … E 6 = (  0 (6) ) 2 +(  1 (6) ) 2 + … + (  62 (6) ) 2 +(  63 (6) ) 2, f(x,y) is the source image. Hierarchical coding without subtractions

31. Feature vectors

32. abStandard coding Hierarchical coding Hierarchical coding without subtractions a1a20.0045050.0053490.003739 b1b20.0099050.0081600.007742 c1c20.0177780.0118480.009946 a3a40.0088070.0060470.002422 b3b40.0200750.0106670.006275 c3c40.1283220.1015910.080176 a1b10.4884200.4922380.491653 b1c10.3156440.3045970.305442

35. Image segmentation task: Brodatz textures

36. Image segmentation task

39. Texture parameterization using 2-D Hermite functions

40. Quasiperiod’s waveforms

41. Signal filtering Speaker indexing Speaker recognition using database Source separation Areas of Hermite transform application:

42. MusicSpeaker 1Speaker 22 speakers Audio sample

43. Quasiperiod waveform Hermite histogram

44. Speaker indexing

45. Speech/music separation Analysis of signal dynamics (distribution of amplitudes in time) Analysis of base tone distribution in time Currently implemented criterions: Result of speech/music separation:

46. Mix detection One speakerMix

47. References 1.Krylov A.S., Liakishev A.N., Numerical Projection Method For Inverse Fourier Transform and its Application, Numerical Functional Analysis and optimization, vol. 21 (2000) p. 205-216. 2.Krylov A.S., Kortchagine D.N., Projection filtering in image processing, Graphicon’2000 Conf. proceedings, p. 42-45, 2000. 3.Krylov A.S., Kortchagine D.N., Lukin A.S., Streaming Waveform Data Processing by Hermite Expansion for Text-Independent Speaker Indexing from Continuous Speech, Graphicon’2002 Conf. proceedings, p. 91-98, 2002. 4.Krylov A.S., Kutovoi A.V., Wee Kheng Leow, Texture Parameterization With Hermite Functions, Graphicon’2002 Conf. proceedings, p. 190-194, 2002. 5.Mohsen Najafi, Andrey Krylov, Danil Kortchagine Image Deblocking With 2-D Hermite Transform, Graphicon’2003 Conf. proceedings.