CS654: Digital Image Analysis Lecture 21: Image Restoration

Recap of Phase 1 Image: acquisition, digitization Geometric Transformations: Interpolation techniques Image Transforms (spatial to frequency domain) Image Enhancement (spatial and frequency domain)

Outline of Lecture 21 Image restoration vs. enhancement What is restoration Image restoration model Continuous, discrete formulation Point spread function Noise

Image restoration It refers to the minimization or removal of the known degradations in an image. De-blurring, noise filtering, correction of geometric distortion etc. Original image Blurred input image Restored image

Restoration vs. Enhancement

Causes of Common degradation Sensor noise (poor illumination, long exposer) Improper focusing (out of focus image) Geometric restoration Lense Irregular movement of the sensor Atmospheric turbulence …

Assumptions Degradation function must be a linear system The system is homogeneous The system is shift invariant

Linear position invariant degradation 𝑔 𝑥,𝑦 =𝐻 𝑓 𝑥,𝑦 +𝜂(𝑥,𝑦) 𝑔 𝑥,𝑦 =𝐻 𝑓 𝑥,𝑦 Assume 𝜂 𝑥,𝑦 =0 𝑯 is a Linear system 𝐻 𝑎 𝑓 1 𝑥,𝑦 +𝑏 𝑓 2 (𝑥,𝑦) =𝑎𝐻[ 𝑓 1 𝑥,𝑦 ]+𝑏𝐻[ 𝑓 2 𝑥,𝑦 ] If 𝑎=𝑏=1; Additivity property If 𝑓 2 (𝑥,𝑦)=0; Homogeneity property 𝑯 is a Position Invariant 𝐻 𝑓(𝑥−𝛼,𝑦−𝛽) =𝑔 𝑥−𝛼,𝑦−𝛽 ]

Image formation in continuous domain 𝑓 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝛿 𝑥−𝛼,𝑦−𝛽 𝑑𝛼𝑑𝛽 𝑔 𝑥,𝑦 =𝐻[𝑓 𝑥,𝑦 ]=𝐻 −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝛿 𝑥−𝛼,𝑦−𝛽 𝑑𝛼𝑑𝛽 𝑔 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝐻 𝑓 𝛼,𝛽 𝛿 𝑥−𝛼,𝑦−𝛽 𝑑𝛼𝑑𝛽 Additivity property 𝑔 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝐻 𝛿 𝑥−𝛼,𝑦−𝛽 𝑑𝛼𝑑𝛽 Homogeneity property 𝒉 𝒙,𝒚,𝜶,𝜷 =𝑯 𝜹 𝒙−𝜶,𝒚−𝜷 Impulse response of H

Image formation in continuous domain 𝑔 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝐻 𝛿 𝑥−𝛼,𝑦−𝛽 𝑑𝛼𝑑𝛽 𝑔 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝒉 𝒙,𝒚,𝜶,𝜷 𝑑𝛼𝑑𝛽 A linear system is characterized by its impulse response 𝑔 𝑥,𝑦 = −∞ ∞ −∞ ∞ 𝑓 𝛼,𝛽 𝒉 𝒙−𝜶,𝒚−𝜷 𝑑𝛼𝑑𝛽 Position Invariant

Point Spread Function scene image Optical System Ideally, the optical system should be a Dirac delta function. Optical System point source point spread function However, optical systems are never ideal. Point spread function of Human Eyes

PSF “A Point source” 𝒉 𝒙,𝒚,𝜶,𝜷 𝐼 1 ( 𝑥 1 , 𝑦 1 ) 𝐼 2 ( 𝑥 2 , 𝑦 2 )

Point Spread Function normal vision myopia hyperopia Astigmatism Images by Richmond Eye Associates

Discrete formulation 𝑔 𝐻 𝑓 𝑔 𝑥 = 𝑚=0 𝑀−1 𝑓 𝑚 ℎ 𝑥−𝑚 ;0≤𝑥≤𝑀−1 1-D case: Matrix notation: 𝑔(0) ⋮ 𝑔(𝑀−1) = ℎ(0) … ℎ(−𝑀+1) ⋮ ⋱ ⋮ ℎ(𝑀−1) ⋯ ℎ(0) 𝑓(0) ⋮ 𝑓(𝑀−1) 𝑔 𝐻 𝑓

Circulant Matrix Assume 𝐻 to be periodic, with periodicity 𝑀 𝐻= ℎ 0 ℎ 𝑀−1 ℎ 𝑀−2 … ℎ 1 ℎ 1 ℎ 0 ℎ(𝑀−1) … ℎ(2) … … … … … ℎ(𝑀−1) ℎ(𝑀−2) ℎ(𝑀−3) … ℎ(0) Each row vector is rotated one element to the right relative to the preceding row vector

Extension to 2-D 𝑔=𝐻𝑓+𝜂 𝑔 𝑥,𝑦 = 𝑚=0 𝑀−1 𝑛=0 𝑁−1 𝑓 𝑚,𝑛 ℎ(𝑥−𝑚,𝑦−𝑛) 𝑔 𝑥,𝑦 = 𝑚=0 𝑀−1 𝑛=0 𝑁−1 𝑓 𝑚,𝑛 ℎ(𝑥−𝑚,𝑦−𝑛) 𝑓 𝑥,𝑦 and ℎ(𝑥,𝑦) are of dimension 𝑀×𝑁 Matrix notation: 𝑔=𝐻𝑓+𝜂 𝑓: Vector of dimension 𝑀𝑁 𝜂: Vector of dimension 𝑀𝑁 𝐻:Matrix of dimension 𝑀𝑁×𝑀𝑁

A model restoration process 𝑓(𝑥,𝑦) 𝐻(𝑥,𝑦) ∗ 𝜂(𝑥,𝑦) + 𝑔 𝑥,𝑦 = 𝑔 𝑥,𝑦 → 𝑓 (𝑥,𝑦) 𝑓 (𝑥,𝑦)≅𝑓(𝑥,𝑦) Target

Noise models Statistical behavior of the grey-level values Can be modeled as a random variable with a specific PDF Gaussian noise Rayleigh noise Gamma noise Exponential noise Uniform noise Impulse (salt & pepper) noise

Gaussian noise The PDF of a Gaussian noise is given by p(z) z

Rayleigh noise p(z) and a z The PDF of a Rayleigh noise is given by The mean and variance are given and a z Application areas: MRI images, Underwater images

Gamma noise p(z) K and z The PDF of a Gamma noise is given by The mean and variance are given and z

Exponential noise p(z) a and z The PDF of a Exponential noise is given by p(z) a The mean and variance are given and z Note: It is a special case of Gamma PDF, with b=1.

Uniform noise p(z) and z a b The PDF of a Uniform noise is given by All noise is present within this interval The mean and variance are given and z a b

Application of Uniform noise b Quantization Predictive coding

Impulse (salt-and-pepper) noise The PDF of a (bipolar) impulse noise is given by p(z) z a b

Summary of different noise models

Thank you Next Lecture: Image Restoration