1. Image Restoration Digital Image Processing Instructor: Dr. Cheng-Chien LiuCheng-Chien Liu Department of Earth Sciences National Cheng Kung University Last updated: 10 October 2003 Chapter 6

2. Introduction  Image restoration Use objective criteria and prior knowledge cf. image enhancement  subjective criteria  Two cases need image restoration Degradation  gray value altered Distortion  pixel shifted  Geometric restoration (image registration)  Aerial photographs

3. Geometric restoration  Source of geometric distortion Lens (Fig 6.1) Irregular movement (Fig 6.2)  Two-stage operation Spatial transformation  x ^ = O x (x, y) = c 1 x + c 2 y + c 3 xy + c 4 y ^ = O y (x, y) = c 5 x + c 6 y + c 7 xy + c 8  Four tie points  c 1, …, c 8 Grey level interpolation  Simple way: g(x ^, y ^ ) =  x ^ +  y ^ +  x ^ y ^ +   Fig 6.3  Example 6.1

4. Linear degradation  Output image g( ,  ) =  -    -   f(x, y) h(x, y, ,  ) dxdy The point spread function: h(x, y, ,  )  Shift invariant h(x, y, ,  ) = h(x, y,  - x,  - y) g( ,  ) =  -    -   f(x, y) h(x, y,  - x,  - y) dxdy  g depends on the relative position rather than actual position G(u, v) = F(u, v) H(u, v)  For discrete images g(i, j) =  k=1 N  l=1 N f(k, l)h(k, l, i, j) g = H f

5. The point spread function H  Problems of image restoration: g = H f Given the degraded image g, recover the original undegraded image f Obtain the information of H  From the knowledge of the physical process  e.g.diffraction, atmospheric turbulence, motion, …  From some known objects on the image  Example 6.2 Expression of blurred image  Example 6.3 Derive H for the blurred image

6. The point spread function H (cont.)  Example 6.4 Calculate H for the blurred image  Example 6.5 Derive H for the degradation process of accelerating motion  Example 6.6 Asymptotic solution of Example 6.5  Example 6.7 Application of Example 6.6

7. The point spread function H (cont.)  Example 6.8 Calculate H from a bright straight line  Example 6.9 Calculate H from an edge  Example 6.10 Calculate H from an image device

8. Straightforward solution  If H is known F(u, v) = G(u, v) / H(u, v) F(u, v)  f(u, v)  However Straightforward solution  unacceptable poor results  H(u, v) = 0 at some points  G = 0  0/0  undetermined  If there is a small amount of noise  G  0, even if H = 0  For additive noise: G(u, v) = F(u, v) H(u, v) + N(u, v)  F(u, v) = G(u, v) / H(u, v) - N(u, v) / H(u, v)  If H(u, v)  0  N(u, v) / H(u, v)   (amplified noise)

9. Straightforward solution (cont.)  Avoiding the amplification of noise Windowed version of the filter 1 / H  F(u, v) = M(u, v) G(u, v) - M(u, v) N(u, v) where M(u, v) = 1 / H(u, v) for u 2 + v 2   0 2 M(u, v) = 1 for u 2 + v 2 >  0 2  Where  0 is chosen so that all zeroes of H(u, v) are excluded Other windowing filters are also valid  Example 6.11 Application of inverse filtering to restore a motion blurred image

10. Indirect solution – Wiener filter  Formal expression of the problem of IR To identify f(r) which minimizes e 2  E{[f(r) - f(r)] 2 }  Where f(r) is an estimate of the original undegraded image f(r) Shift invariant assumption  g(r) =  -    -   h(r  - r΄) f(r΄)dr΄ + v(r)  Where g(r), f(r) and h(r) are random fields, v(r) is noise field  Solution  find the Wiener filter If no imposed condition  conditional expectation  simulated annealing  beyond our scope Constraint: f(r) is a linear function of g(r)  f(r) =  -    -   m(r  r΄) g(r΄)dr΄  we decide (B6.1)  f(r) =  -    -   m(r - r΄) g(r΄)dr΄  if the random fields are homogeneous  Identify the Wiener filter m(r) with which to convolve g(r΄)  f(r)

11. Fourier transfer of the Wiener filter  M(u, v) = F{m(r)} = S fg (u, v) / S gg (u, v) Proof in B6.3 S fg (u, v) is the cross-spectral density of f and g S gg (u, v) is the spectral density of g  Extra assumption: f(r) and v(r) are uncorrelated E{v(r)} = 0  E{f(r)v(r)} = E{f(r)}E{v(r)} = 0

12. Fourier transfer of the Wiener filter (cont.)  Create S gf g(r) =  -    -   h(r  - r΄) f(r΄)dr΄ + v(r) R gf (s) = E{g(r)f(r - s)} =  -    -   h(r  - r΄) E{f(r΄)f(r - s)}dr΄ + E{f(r - s)v(r)} =  -    -   h(r  - r΄) R ff (r΄ - r + s)dr΄ S gf (u, v) = H * (u, v)S ff (u, v)  (B6.4) S gg (u, v) = S ff (u, v)|H(u, v)| 2 + S vv (u, v)  (B6.4)  M(u, v) M = H * S ff / [S ff |H| 2 + S vv ] M = (1/H) |H| 2 / [|H| 2 + S vv /S ff ]

13. Fourier transfer of the Wiener filter (cont.)  Noise If there is no noise  S vv (u, v) = 0  M = 1/H  So the linear least square error approach simply determines a correction factor with which the inverse transfer function of the degradation process has to be multiplied before it is used as a filter, so that the effect of noise is taken care of. Assumption  White noise: S vv (u, v) = constant = S vv (0, 0) =  -    -   R vv (x, y)dxdy  Ergodic noise: R vv (x, y) can be obtained from a single pure noise image i.e. when f(x, y) = 0

14. Fourier transfer of the Wiener filter (cont.)  B6.1 If m(r - r΄) satisfies E{[f(r) -  -    -   m(r  r΄) g(r΄)dr΄]g(s)} = 0, then it minimizes e 2  E{[f(r) - f(r)] 2 }  Example 6.12 g(r) =  -    -   h(t  - r) f(t)dt  G(u, v) = H * (u, v) F(u, v)  B6.2 Wiener-Khinchine theorem: R ff (u, v) = |F fg (u, v)| 2  B6.3 M(u, v) = F{m(r)} = S fg (u, v) / S gg (u, v)

15. Fourier transfer of the Wiener filter (cont.)  B6.4 S gg (u, v) = S ff (u, v)|H(u, v)| 2 + S vv (u, v)  Example 6.13 Apply Wiener filtering to restore a motion blurred image

16. Problems of the straightforward solution  Straightforward solution g = Hf Including noise: g = Hf + v Inversion: f = H -1 g – H -1 v  H is an N 2  N 2 matrix  f, g and v are N 2  1 vectors Problems  f is very sensitive to v (Example 6.14)  Formidable task to inverse an N 2  N 2 matrix

17. Circulant matrix  Definition The circulant matrix D (Eq. 6.78)  Each column of a matrix can be obtained from the precious one by shifting all elements one place and putting the last element at the top The block circulant matrix (Eq. 6.77)  Dw(k) = (k)w(k) (k) are the eigenvalues of D  (k)  d(0) + d(M-1)exp[2  jk/M] + d(M-2)exp[2  j2k/M] + … + d(1)exp[2  j(M-1)k/M] w(k) are the eigenvectors of D  w(k)  [1, exp[2  jk/M], exp[2  j2k/M], …, exp[2  j(M-1)k/M]] T

18. Inversion of the circulant matrix  Inversion of D D = W  W -1  W is formed by having the eigenvectors of D as columns  W -1 (k, j) = (1/M)exp[-2  j/M ki] (Example 6.15)   is a diagonal matrix with the eigenvalues alone its diagonal. D -1 = (W  W -1 ) -1 = (W -1 ) -1  -1 W -1 = W  W -1 Example 6.16: A case of M = 3 Example 6.17: A case of M = 4 Example 6.18:  W  W N  W N  W -1 = Z  W N -1  W N -1  W N (k, n) = (N) -1/2 exp[2  j/N kn]  W N -1 (k, n) = (N) -1/2 exp[-2  j/N kn]  Kronecker product 

19. Inverting H – Overcome one problem of the straightforward solution  H is block circulant g = H f g(i, j) =  k=0 N-1  l=0 N-1 h(k, l, i, j) f(k, l) For a shift invariant point spread function g(i, j) =  k=0 N-1  l=0 N-1 f(k, l) h(i-k, j-l)  Diagonalize H H = W  W -1 (B 6.5)  W N (k, n) = (N) -1/2 exp[2  j/N kn]  W N -1 (k, n) = (N) -1/2 exp[-2  j/N kn]   (k, i) = NH(k mod N, [k/N]) if i = k   (k, i) = 0 if i  k  H( , ) = (1/N)  x=0 N-1  y=0 N-1 h(x,y)e -2  j(  x/N+ y/N)

20. Inverting H – Overcome one problem of the straightforward solution (cont.)  Transpose H H T = W   W -1 (B 6.6)   * means the complex conjugate of   Example 6.19: Laplacian at a pixel position  2 f(i, j) = f(i-1, j) + f(i, j-1) + f(i+1, j) + f(i, j +1) - 4f(i, j)  Example 6.20: Identify L to estimate  2 f(i, j)  Example 6.21: Apply the Eq. of  2 f(i, j)  L  Example 6.22:

21. Constrained matrix inversion filter – Overcome another problem