1. Invariants to convolution

2. Motivation – template matching

3. Image degradation models Space-variant Space-invariant  convolution

4. Two approaches Traditional approach: Image restoration (blind deconvolution) and traditional features Proposed approach: Invariants to convolution

5. Invariants to convolution for any admissible h

6. The moments under convolution Geometric/central Complex

7. PSF is centrosymmetric (N = 2) and has a unit integral Assumptions on the PSF supp(h)

8. Common PSF’s are centrosymmetric

9. Invariants to centrosymmetric convolution where (p + q) is odd What is the intuitive meaning of the invariants? “Measure of anti-symmetry”

10. Invariants to centrosymmetric convolution

12. Convolution invariants in FT domain

14. Relationship between FT and moment invariants where M(k,j) is the same as C(k,j) but with geometric moments

15. Template matching sharp image with the templates the templates (close-up)

16. Template matching a frame where the templates were located successfully the templates (close-up)

17. Template matching performance

18. Valid region Boundary effect in template matching

19. Invalid region - invariance is violated Boundary effect in template matching

20. zero-padding

21. N-fold rotation symmetry, N > 2 Axial symmetry Circular symmetry Gaussian PSF Motion blur PSF Other types of the blurring PSF The more we know about the PSF, the more invariants and the higher discriminability we get

22. Discrimination power The null-space of the blur invariants = a set of all images having the same symmetry as the PSF. One cannot distinguish among symmetric objects.

23. Recommendation for practice It is important to learn as much as possible about the PSF and to use proper invariants for object recognition

24. Combined invariants Convolution and rotation For any N

25. Robustness of the invariants

26. Satellite image registration by combined invariants

27. Registration algorithm Independent corner detection in both images Extraction of salient corner points Calculating blur-rotation invariants of a circular neighborhood of each extracted corner Matching corners by means of the invariants Refining the positions of the matched control points Estimating the affine transform parameters by a least-square fit Resampling and transformation of the sensed image

28. ( v1 1, v2 1, v3 1, … ) ( v1 2, v2 2, v3 2, … ) min distance(( v1 k, v2 k, v3 k, … ), ( v1 m, v2 m, v3 m, … )) k,m Control point detection

29. Control point matching

30. Registration result

31. Application in MBD

32. Combined blur-affine invariants Let I(μ00,…, μPQ) be an affine moment invariant. Then I(C(0,0),…,C(P,Q)), where C(p,q) are blur invariants, is a combined blur-affine invariant (CBAI).

33. Digit recognition by CBAI CBAI AMI

34. Documented applications of convolution and combined invariants Character/digit/symbol recognition in the presence of vibration, linear motion or out-of-focus blur Robust image registration (medical, satellite,...) Detection of image forgeries

35. Moment-based focus measure Odd-order moments  blur invariants Even-order moments  blur/focus measure If M(g1) > M(g2)  g2 is less blurred (more focused)

36. Other focus measures Gray-level variance Energy of gradient Energy of Laplacian Energy of high-pass bands of WT

37. Desirable properties of a focus measure Agreement with a visual assessment Unimodality Robustness to noise Robustness to small image changes

38. Agreement with a visual assessment

39. Robustness to noise

40. Sunspots – atmospheric turbulence

41. Moments are generally worse than wavelets and other differential focus measures because they are too sensitive to local changes but they are very robust to noise.

43. Invariants to elastic deformations

44. How can we recognize objects on curved surfaces...

45. Moment matching Find the best possible fit by minimizing the error and set

46. The bottle

48. Orthogonal moments Motivation for using OG moments Stable calculation by recurrent relations Easier and stable image reconstruction - set of orthogonal polynomials

49. Numerical stability How to avoid numerical problems with high dynamic range of geometric moments?

50. Standard powers

51. Orthogonal polynomials Calculation using recurrent relations

52. Two kinds of orthogonality Moments (polynomials) orthogonal on a unit square Moments (polynomials) orthogonal on a unit disk

53. Moments orthogonal on a square is a system of 1D orthogonal polynomials

54. Common 1D orthogonal polynomials Legendre Chebyshev Gegenbauer Jacobi or (generalized) Laguerre <0,∞) Hermite(-∞,∞)

55. Legendre polynomials Definition Orthogonality

56. Legendre polynomials explicitly

57. Legendre polynomials in 1D

58. Legendre polynomials in 2D

59. Legendre polynomials Recurrent relation

60. Legendre moments

61. Moments orthogonal on a disk Radial part Angular part

62. Moments orthogonal on a disk Zernike Pseudo-Zernike Orthogonal Fourier-Mellin Jacobi-Fourier Chebyshev-Fourier Radial harmonic Fourier

63. Zernike polynomials Definition Orthogonality

64. Zernike polynomials – radial part in 1D

65. Zernike polynomials – radial part in 2D

66. Zernike polynomials

67. Zernike moments Mapping of Cartesian coordinates x,y to polar coordinates r,φ: Whole image is mapped inside the unit disk Translation and scaling invariance

68. Zernike moments

69. Rotation property of Zernike moments The magnitude is preserved, the phase is shifted by ℓθ. Invariants are constructed by phase cancellation

70. Zernike rotation invariants Phase cancellation by multiplication Normalization to rotation

71. Image reconstruction Direct reconstruction from geometric moments Solution of a system of equations Works for very small images only For larger images the system is ill-conditioned

72. Image reconstruction by direct computation 12 x 12 13 x 13

73. Image reconstruction Reconstruction from geometric moments via FT

74. Image reconstruction via Fourier transform

75. Image reconstruction Image reconstruction from OG moments on a square Image reconstruction from OG moments on a disk (Zernike)

76. Image reconstruction from Legendre moments

77. Image reconstruction from Zernike moments Better for polar raster

78. Image reconstruction from Zernike moments

79. Reconstruction of a noise-free image

80. Reconstruction of a noisy image

82. Summary of OG moments OG moments are used because of their favorable numerical properties, not because of theoretical contribution OG moments should be never used outside the area of orthogonality OG moments should be always calculated by recurrent relations, not by expanding into powers Moments OG on a square do not provide easy rotation invariance Even if the reconstruction from OG moments is seemingly simple, moments are not a good tool for image compression