Image Restoration – Degradation Model and General Approaches

Image enhancement vs. restoration Degradation model Image Restoration - 1 Outline Image enhancement vs. restoration Degradation model Noise only Linear, space-invariant General approaches Inverse filters Wiener filters Constrained least squares filtering

Image Restoration - 1 Image enhancement vs. restoration Image enhancement : process image so that the result is more suitable for a specific application, is largely a subjective process. Image restoration : recover image from distortions to its original image, is largely an objective process.

Apply the inverse process to recover the original image Image Restoration - 1 General approaches Model the degradation Apply the inverse process to recover the original image The degradation process is modeled as a degradation function with an additive noise term.

Degradation models : noise only Image Restoration - 1 Degradation models : noise only Noise models Spatial characteristics (independent or dependent) Intensity ( distribution, spectrum) Uniform, Gaussian, Rayleigh, Gamma (Erlang), Exponential, impulse Correlation with the image (additive, multiplicative) De-noising Spatial filtering Frequency domain filtering The principal sources of noise arise during image acquisition and/or transmission.

Image Restoration - 1 Noise models – examples

Image Restoration - 1 Noise models – examples

Degradation models : noise only Image Restoration - 1 Degradation models : noise only Noise simulation (Random number/signal generator) 盛文，焦晓丽. 雷达系统建模与仿真导论. 国防工业出版社，2006 The principal sources of noise arise during image acquisition and/or transmission.

Degradation models : noise only Image Restoration - 1 Degradation models : noise only 高斯分布随机变量的近似产生方法 中心极限定理：n个均值为 、方差为 的随机变量的和服从均值为 、方差为 的近似正态分布 取n个[0，1]区间上的均匀分布随机变量，则以下随机变量服从均值为0、方差为1的近似标准正态分布。

Image Restoration - 1 Noise models – periodic noise

Estimation of noise parameters Image Restoration - 1 De-noising Estimation of noise parameters By spectrum inspection: for periodic noise By test image: mean, variance and histogram shape, if imaging system is available By small patches, if only image is available De-noising Spatial filtering ( for additive noise) Mean filters Order-statistics filters Adaptive filters Frequency domain filtering (for periodic noise)

De-noising – Gaussian noise example Image Restoration - 1 De-noising – Gaussian noise example Arithmetic mean filter Geometric mean filter

Order-statistics filters Image Restoration - 1 De-noising Order-statistics filters Median filter Good for impulse noise reduction with less blurring Max filter Find the brightest points Min filter Find the darkest points Midpoint filter Combines order statistics and averaging, works best for randomly distributed noise.

Image Restoration - 1 De-noising – Salt & Pepper noise example

Image Restoration - 1 De-noising – Salt & Pepper noise example

De-noising – Adaptive filters Image Restoration - 1 De-noising – Adaptive filters Adaptive local noise reduction filter Adaptive median filter Homework: read pp.241-243

Image Restoration - 1 De-noising – Adaptive local filter

Image Restoration - 1 De-noising – Periodic noise example

Image Restoration - 1 De-noising – evaluation PSNR Visual perception

De-noising – evaluation example Image Restoration - 1 De-noising – evaluation example ImageDenoising.m

Degradation models: linear vs. non-linear Image Restoration - 1 Degradation models: linear vs. non-linear Many types of degradation can be approximated by linear, space invariant processes Can take advantages of the mature techniques developed for linear systems Non-linear and space variant models are more accurate Difficult to solve Unsolvable

Linear, space-invariant degradations Image Restoration - 1 Linear, space-invariant degradations Sampling theorem --> Linearity, additivity --> Linearity, homogeneity --> Space-invariant --> (convolution integral)

Image Restoration - 1 Linear, space-invariant degradations (cont’) Point Spread Function: Linear, space-invariant degradation model:

Estimating degradation function - 1 Image Restoration - 1 Estimating degradation function - 1 Estimation by image observation Degradation system H is completely characterized by its impulse response Select a small section from the degraded image Reconstruct an unblurred image of the same size The degradation function can be estimated by By ignoring the noise term, G(u,v) = F(u,v)H(u,v). If F(u,v) is the Fourier transform of point source (impulse), then G(u,v) is approximates H(u,v).

Estimating degradation function - 2 Image Restoration - 1 Estimating degradation function - 2 Estimation by experimentation Point spread function (PSF) Used in optics The impulse becomes a point of light The impulse response is commonly referred to as the PSF

Image Restoration - 1 Estimating degradation function - 3 Estimation by modeling – atmospheric turbulence

Image Restoration - 1 Estimating degradation function - 3 Estimation by modeling – linear motion blurring

The regularization theory Image Restoration - 1 Different approaches Classical approaches Inverse filter Weiner filter Algebraic approaches The regularization theory

Image Restoration - 1 Inverse filtering Degradation model This expression tells us that even if we know the degradation function we cannot recover the undegraded image exactly because of the random noise, whose Fourier transform is not known. Image restoration is an ill-posed problem. When H(u,v) is very small, the noise term dominates the restoration result.

Inverse filtering - examples Image Restoration - 1 Inverse filtering - examples This expression tells us that even if we know the degradation function we cannot recover the undegraded image exactly because of the random noise, whose Fourier transform is not known. Image restoration is an ill-posed problem. When H(u,v) is very small, the noise term dominates the restoration result.

Image Restoration - 1 Wiener filtering In most images, adjacent pixels are highly correlated, while the gray level of widely separated pixels are only loosely correlated. Therefore, the autocorrelation function of typical images generally decreases away from the origin. Power spectrum of an image is the Fourier transform of its autocorrelation function, therefore we can argue that the power spectrum of an image generally decreases with frequency. Typical noise sources have either a flat power spectrum or one that decreases with frequency more slowly than typical image power spectrum. Therefore, the expected situation is for the signal to dominate the spectrum at low frequencies, while the noise dominates the high frequencies.

Image Restoration - 1 Wiener filtering (cont’) Degradation model Wiener filter

Image Restoration - 1 Wiener filtering - example

Image Restoration - 1 Wiener filtering - example

Image Restoration - 1 Wiener filtering - problems The power spectra of the undegraded image and noise must be known. Weights all errors equally regardless of their location in the image, while the eye is considerably more tolerant of errors in the dark areas and high-gradient areas in the image. In minimizing the mean square error, Wiener filter also smooth the image more than the eye would prefer.

Image Restoration - 1 Constrained Least Squares Filtering Only the mean and variance of the noise is required The degradation model in vector-matrix form The objective function

Image Restoration - 1 Constrained Least Squares Filtering The solution

Image Restoration - 1 Constrained Least Squares Filtering - example