Invariants to convolution

Motivation – template matching

Image degradation models Space-variant Space-invariant  convolution

Two approaches Traditional approach: Image restoration (blind deconvolution) and traditional features Proposed approach: Invariants to convolution

Invariants to convolution for any admissible h

The moments under convolution Geometric/central Complex

PSF is centrosymmetric (N = 2) and has a unit integral Assumptions on the PSF supp(h)

Common PSF’s are centrosymmetric

Invariants to centrosymmetric convolution where (p + q) is odd What is the intuitive meaning of the invariants? “Measure of anti-symmetry”

Invariants to centrosymmetric convolution

Convolution invariants in FT domain

Relationship between FT and moment invariants where M(k,j) is the same as C(k,j) but with geometric moments

Template matching sharp image with the templates the templates (close-up)

Template matching a frame where the templates were located successfully the templates (close-up)

Template matching performance

Valid region Boundary effect in template matching

Invalid region - invariance is violated Boundary effect in template matching

zero-padding

N-fold rotation symmetry, N > 2 Axial symmetry Circular symmetry Gaussian PSF Motion blur PSF Other types of the blurring PSF The more we know about the PSF, the more invariants and the higher discriminability we get

Discrimination power The null-space of the blur invariants = a set of all images having the same symmetry as the PSF. One cannot distinguish among symmetric objects.

Recommendation for practice It is important to learn as much as possible about the PSF and to use proper invariants for object recognition

Combined invariants Convolution and rotation For any N

Robustness of the invariants

Satellite image registration by combined invariants

Registration algorithm Independent corner detection in both images Extraction of salient corner points Calculating blur-rotation invariants of a circular neighborhood of each extracted corner Matching corners by means of the invariants Refining the positions of the matched control points Estimating the affine transform parameters by a least-square fit Resampling and transformation of the sensed image

( v1 1, v2 1, v3 1, … ) ( v1 2, v2 2, v3 2, … ) min distance(( v1 k, v2 k, v3 k, … ), ( v1 m, v2 m, v3 m, … )) k,m Control point detection

Control point matching

Registration result

Application in MBD

Combined blur-affine invariants Let I(μ00,…, μPQ) be an affine moment invariant. Then I(C(0,0),…,C(P,Q)), where C(p,q) are blur invariants, is a combined blur-affine invariant (CBAI).

Digit recognition by CBAI CBAI AMI

Documented applications of convolution and combined invariants Character/digit/symbol recognition in the presence of vibration, linear motion or out-of-focus blur Robust image registration (medical, satellite,...) Detection of image forgeries

Moment-based focus measure Odd-order moments  blur invariants Even-order moments  blur/focus measure If M(g1) > M(g2)  g2 is less blurred (more focused)

Other focus measures Gray-level variance Energy of gradient Energy of Laplacian Energy of high-pass bands of WT

Desirable properties of a focus measure Agreement with a visual assessment Unimodality Robustness to noise Robustness to small image changes

Agreement with a visual assessment

Robustness to noise

Sunspots – atmospheric turbulence

Moments are generally worse than wavelets and other differential focus measures because they are too sensitive to local changes but they are very robust to noise.

Invariants to elastic deformations

How can we recognize objects on curved surfaces...

Moment matching Find the best possible fit by minimizing the error and set

The bottle

Orthogonal moments Motivation for using OG moments Stable calculation by recurrent relations Easier and stable image reconstruction - set of orthogonal polynomials

Numerical stability How to avoid numerical problems with high dynamic range of geometric moments?

Standard powers

Orthogonal polynomials Calculation using recurrent relations

Two kinds of orthogonality Moments (polynomials) orthogonal on a unit square Moments (polynomials) orthogonal on a unit disk

Moments orthogonal on a square is a system of 1D orthogonal polynomials

Common 1D orthogonal polynomials Legendre Chebyshev Gegenbauer Jacobi or (generalized) Laguerre <0,∞) Hermite(-∞,∞)

Legendre polynomials Definition Orthogonality

Legendre polynomials explicitly

Legendre polynomials in 1D

Legendre polynomials in 2D

Legendre polynomials Recurrent relation

Legendre moments

Moments orthogonal on a disk Radial part Angular part

Moments orthogonal on a disk Zernike Pseudo-Zernike Orthogonal Fourier-Mellin Jacobi-Fourier Chebyshev-Fourier Radial harmonic Fourier

Zernike polynomials Definition Orthogonality

Zernike polynomials – radial part in 1D

Zernike polynomials – radial part in 2D

Zernike polynomials

Zernike moments Mapping of Cartesian coordinates x,y to polar coordinates r,φ: Whole image is mapped inside the unit disk Translation and scaling invariance

Zernike moments

Rotation property of Zernike moments The magnitude is preserved, the phase is shifted by ℓθ. Invariants are constructed by phase cancellation

Zernike rotation invariants Phase cancellation by multiplication Normalization to rotation

Image reconstruction Direct reconstruction from geometric moments Solution of a system of equations Works for very small images only For larger images the system is ill-conditioned

Image reconstruction by direct computation 12 x x 13

Image reconstruction Reconstruction from geometric moments via FT

Image reconstruction via Fourier transform

Image reconstruction Image reconstruction from OG moments on a square Image reconstruction from OG moments on a disk (Zernike)

Image reconstruction from Legendre moments

Image reconstruction from Zernike moments Better for polar raster

Image reconstruction from Zernike moments

Reconstruction of a noise-free image

Reconstruction of a noisy image

Summary of OG moments OG moments are used because of their favorable numerical properties, not because of theoretical contribution OG moments should be never used outside the area of orthogonality OG moments should be always calculated by recurrent relations, not by expanding into powers Moments OG on a square do not provide easy rotation invariance Even if the reconstruction from OG moments is seemingly simple, moments are not a good tool for image compression