Download presentation
Presentation is loading. Please wait.
1
Digital Image Processing
Image Restoration Matrix Formulation 5/3/2019 Duong Anh Duc - Digital Image Processing
2
Matrix Formulation of Image Restoration Problem
1-D Case: We will consider the 1-D version first, for simplicity: g(m) = f(m)*h(m) + (m) We will assume that the arrays f and h have been zero-padded to be of size M, where M length(f) + length(h) - 1. Henceforth, we will not explicitly mention the zero-padding. The degradation equation: can be written in matrix-vector form as follows: g = Hf + n, where 5/3/2019 Duong Anh Duc - Digital Image Processing
3
Matrix Formulation of Image Restoration Problem
5/3/2019 Duong Anh Duc - Digital Image Processing
4
Matrix Formulation of Image Restoration Problem
However, since the arrays f and h are zero-padded, we can equivalently set: Notice that the (second) matrix H is circulant; i.e., each row of H is a circular shift of the previous row. 5/3/2019 Duong Anh Duc - Digital Image Processing
5
Matrix Formulation of Image Restoration Problem
Example: A = length of array f = 3 B = length of array h = 2 M ³ A + B – 1 = 4, say M = 4. 5/3/2019 Duong Anh Duc - Digital Image Processing
6
Matrix Formulation of Image Restoration Problem
5/3/2019 Duong Anh Duc - Digital Image Processing
7
Matrix Formulation of Image Restoration Problem
Notice that H1f = H2f. Indeed 5/3/2019 Duong Anh Duc - Digital Image Processing
8
Matrix Formulation of Image Restoration Problem
Henceforth, we will use H = H2 , so that we can apply properties of circulant matrices to H. 5/3/2019 Duong Anh Duc - Digital Image Processing
9
Matrix Formulation of Image Restoration Problem
2-D Case: Suppose g, f, h, are MN arrays (after zero-padding). The degradation equation can be written in matrix-vector format as follows: g = Hf + n, where 5/3/2019 Duong Anh Duc - Digital Image Processing
10
Matrix Formulation of Image Restoration Problem
5/3/2019 Duong Anh Duc - Digital Image Processing
11
Matrix Formulation of Image Restoration Problem
Note that H is a MNMN block-circulant matrix with MM blocks. 5/3/2019 Duong Anh Duc - Digital Image Processing
12
Matrix Formulation of Image Restoration Problem
Each block Hj is itself an NN circulant matrix. Indeed, the matrix Hj is a circulant matrix formed from the j-th row of array h(m,n): 5/3/2019 Duong Anh Duc - Digital Image Processing
13
Matrix Formulation of Image Restoration Problem
Given the degradation equation: g = Hf + n, our objective is to recover f from observation g. We will assume that the array h(m,n) (usually referred to as the blurring function) and statistics of the noise h(m,n) are known. The problem becomes very complicated if array h(m,n) is unknown and this case is usually referred to as blind restoration or blind deconvolution. 5/3/2019 Duong Anh Duc - Digital Image Processing
14
Matrix Formulation of Image Restoration Problem
Notice that, even when there is no noise; i.e. h(m,n) = 0, or the values of h(m,n) were exactly known, and matrix H is invertible, computing ^f = H-1(g-n) directly would not be practical. Example: Suppose M = N = 256. Therefore MN = and H would be a by matrix to be inverted! 5/3/2019 Duong Anh Duc - Digital Image Processing
15
Matrix Formulation of Image Restoration Problem
Naturally, direct inversion of H would not be feasible. But H has several useful properties; in particular: H is block circulant. H is usually sparse (has very few non-zero entries). We will exploit these properties to obtain ˆf more efficiently. In particular, we will derive the theoretical solutions to the restoration problem using matrix algebra. However, when it comes to implementing the solution, we can resort to the Fourier domain, thanks to the properties of circulant matrices. 5/3/2019 Duong Anh Duc - Digital Image Processing
16
Constrained least squares filtering (restoration)
Recall that the knowledge of blur function h(m, n) is essential to obtain a meaningful solution to the restoration problem. Often, knowledge of h(m, n) is not perfect and subject to errors. One way to alleviate sensitivity of the result to errors in h(m, n) is to base optimality of restoration on a measure of smoothness, such as the second derivative of the image. 5/3/2019 Duong Anh Duc - Digital Image Processing
17
Constrained least squares filtering (restoration)
We will approximate the second derivative (Laplacian) by a matrix Q. Indeed, we will first formulate the constrained restoration problem and obtain its solution in terms of a general matrix Q. 5/3/2019 Duong Anh Duc - Digital Image Processing
18
Constrained least squares filtering (restoration)
Later different choices of matrix Q will be considered, each giving rise to a different restoration filter. Suppose Q is any matrix (of appropriate dimension). In constrained image restoration, we choose ^f to minimize ||Qˆf||2, subject to the constraint, ||g-Hˆf||2= ||n||2. (Recall the degradation equation g=Hˆf +n g-Hˆf = n.) 5/3/2019 Duong Anh Duc - Digital Image Processing
19
Constrained least squares filtering (restoration)
Introduction of matrix Q allows considerable flexibility in the design of appropriate restoration filters (we will discuss specific choices of Q later). So our problem is formulated as follows: min ||Qˆf||2 subject to ||g-Hˆf||2= ||n||2 or ||g-Hˆf||2- ||n||2=0 5/3/2019 Duong Anh Duc - Digital Image Processing
20
A brief review of matrix differentiation
Suppose and f(x1,x2) is a function of two variables. Then 5/3/2019 Duong Anh Duc - Digital Image Processing
21
A brief review of matrix differentiation
f(x1,x2) = ||Ax-b||2 = (Ax-b)T (Ax-b) for some matrix A and some vector b , then where superscript T denotes matrix transpose. 5/3/2019 Duong Anh Duc - Digital Image Processing
22
A brief review of matrix differentiation
Recall from calculus that such a constrained minimization problem can be solved by means of Lagrange multipliers. We need to minimize the augmented objective function J(ˆf): J(ˆf) = ||Qˆf||2+(||g-Hˆf||2- ||n||2), where a is a Lagrange multiplier. We set the derivative of J(ˆf) with respect to ˆf to zero. J(ˆf) = 2QT Qˆf-2HT(g-Hˆf) = 0 (QT Q + HTH)ˆf = HTg 5/3/2019 Duong Anh Duc - Digital Image Processing
23
A brief review of matrix differentiation
Therefore, ˆf = (QT Q + HTH)-1HTg = (QT Q/ + HTH)-1HTg = (QT Q + HTH)-1HTg where = 1/ is chosen to satisfy the constraint ||g-Hˆf||2= ||n||2. We will now use the above formulation to derive a number of restoration filters. 5/3/2019 Duong Anh Duc - Digital Image Processing
24
Pseudo-inverse Filtering
The pseudo-inverse filter tries to avoid the pitfalls of applying an inverse filter in the presence of noise. Consider the constrained restoration solution, ^f = (QT Q + HTH)-1HTg with Q = I. This gives, ^f = (I + HTH)-1HTg 5/3/2019 Duong Anh Duc - Digital Image Processing
25
Pseudo-inverse Filtering
It can be implemented in the Fourier domain by the following equation: ^F(u,v) = R(u,v)G(u,v), where The parameter g is a constant to be chosen. Note that g = 0 gives us back the inverse filter. For g > 0, the denominator of R(u, v) is strictly positive and the pseudo-inverse filter is well defined. 5/3/2019 Duong Anh Duc - Digital Image Processing
26
Pseudo-inverse Filtering example
Added Zero-mean Gaussian noise with variance s2 = 0.003 f(m,n) g(m,n) MSE = 0.01 5/3/2019 Duong Anh Duc - Digital Image Processing
27
Pseudo-inverse Filtering
^f(x,y) g = MSE = 0.075 g = MSE = 5/3/2019 Duong Anh Duc - Digital Image Processing
28
Pseudo-inverse Filtering
^f(x,y) g = MSE = g = 0 (Inv. Filter) MSE = 2.7 5/3/2019 Duong Anh Duc - Digital Image Processing
29
Minimum Mean Square Error (Wiener) Filter
This is a restoration technique based on the statistics (mean and correlation) of the image and noise. We consider each element of f and n as random variables. Define the correlation matrices 5/3/2019 Duong Anh Duc - Digital Image Processing
30
Minimum Mean Square Error (Wiener) Filter
The matrices Rf and Rn are real and symmetric, with all eigenvalues being non-negative. The 2D-DFT of the correlations Rf and Rn are called the power spectra and are denoted by Sf(u,v) and Sn(u,v) respectively. 5/3/2019 Duong Anh Duc - Digital Image Processing
31
Minimum Mean Square Error (Wiener) Filter
Recall the constrained restoration solution given by ^f = (QT Q + HTH)-1HTg Choose matrix Q such that QT Q = Rf-1Rn In a sense, we are trying to minimize the noise-to-signal ratio. The constrained restoration is then given by ^f = (HTH + Rf-1Rn)-1HTg 5/3/2019 Duong Anh Duc - Digital Image Processing
32
Minimum Mean Square Error (Wiener) Filter
This can be implemented using DFT as 5/3/2019 Duong Anh Duc - Digital Image Processing
33
Minimum Mean Square Error (Wiener) Filter
Here Sf(u,v) = E(|F(u,v)|2) is the power spectral density of the image f(m, n) and Sh(u,v) = E(|N(u,v)|2) is the power spectral density of the noise h(m, n). The restoration filter R(u,v) is called the parametric Wiener filter, with parameter g Special cases: g = 1: Wiener Filter g = 0: Inverse Filter g 0,1: Parametric Wiener Filter 5/3/2019 Duong Anh Duc - Digital Image Processing
34
Minimum Mean Square Error (Wiener) Filter
According to the constrained restoration filter derived earlier, parameter g should be chosen to satisfy ||g-Hˆf||2= ||n||2. However, choice of g = 1 yields an optimal filter in the sense of minimizing the error function e2= E{[f(m,n)-^f(m,n)]2}. In other words, setting g = 1 yields a statistically optimal restoration. Implementation of the parametric Wiener filter requires knowledge of the image and noise power spectra Sf(u,v) and Sn(u,v). In particular, we need the so called signal-to-noise ratio (SNR) r(u,v) = Sf(u,v)/Sn(u,v) 5/3/2019 Duong Anh Duc - Digital Image Processing
35
Minimum Mean Square Error (Wiener) Filter
This is not always available and a simple approximation is to replace r(u,v) by a constant r. In this case, the Wiener filter is given by Note that as (no noise), the Wiener filter tends to the inverse filter. 5/3/2019 Duong Anh Duc - Digital Image Processing
36
Duong Anh Duc - Digital Image Processing
Wiener Filter example f(m,n) 5/3/2019 Duong Anh Duc - Digital Image Processing
37
Duong Anh Duc - Digital Image Processing
Wiener Filter example = 25.9dB 5/3/2019 Duong Anh Duc - Digital Image Processing
38
Duong Anh Duc - Digital Image Processing
Wiener Filter example = 15.9dB 5/3/2019 Duong Anh Duc - Digital Image Processing
39
Duong Anh Duc - Digital Image Processing
Wiener Filter example = 5.9dB 5/3/2019 Duong Anh Duc - Digital Image Processing
40
Parametric Wiener Filter example (effect of parameter g)
g(m,n) ^f(m,n), g = 0.01 ^f(m,n), g = 0.1 5/3/2019 Duong Anh Duc - Digital Image Processing
41
Parametric Wiener Filter example (effect of parameter g)
^f(m,n), g = 1 ^f(m,n), g = 5 ^f(m,n), g = 50 5/3/2019 Duong Anh Duc - Digital Image Processing
42
Parametric Wiener Filter example (effect of parameter g)
Small values of g result in better “blur removal” and poor noise filtering. Large values of g result in poor “blur removal” and better noise filtering. 5/3/2019 Duong Anh Duc - Digital Image Processing
43
Constrained Least Squares Restoration
Recall the constrained restoration problem: min ||Q^f ||2 subject to ||g-Hˆf||2 - ||n||2 = 0 or ||g-Hˆf||2= ||n||2 Its solution ^f = (HTH + QT Q )-1HTg depends on the choice of Q. 5/3/2019 Duong Anh Duc - Digital Image Processing
44
Constrained Least Squares Restoration
One possibility is to formulate a criterion of optimality (choice of Q) that is based on a measure of smoothness (minimize “roughness” or oscillatory behavior of the solution). This is normally done by choosing Q to represent a second derivative of the image. 5/3/2019 Duong Anh Duc - Digital Image Processing
45
Constrained Least Squares Restoration
Consider the 1-D case: A discrete approximation of the second derivative at a point x = mDx can be obtained as follows: 5/3/2019 Duong Anh Duc - Digital Image Processing
46
Constrained Least Squares Restoration
In a discrete formulation with Dx = 1, this can be written as Therefore, we seek an estimate ˆf of f which is smooth in the sense that it minimizes the above “roughness measure.” 5/3/2019 Duong Anh Duc - Digital Image Processing
47
Constrained Least Squares Restoration
This can be formulated in our standard matrix notation as follows: min ||C^f ||2 = {^fTCTC^f} subject to ||g-Hˆf||2 - ||n||2 = 0 or ||g-Hˆf||2= ||n||2 where (recall zero-padding) 5/3/2019 Duong Anh Duc - Digital Image Processing
48
Constrained Least Squares Restoration
or equivalently is a “smoothing matrix” and ^f is a vector representing the restored image. 5/3/2019 Duong Anh Duc - Digital Image Processing
49
Constrained Least Squares Restoration
In the 2D case (with Dx = Dy = 1 ), we have 5/3/2019 Duong Anh Duc - Digital Image Processing
50
Constrained Least Squares Restoration
The roughness measure can then be written as 5/3/2019 Duong Anh Duc - Digital Image Processing
51
Constrained Least Squares Restoration
This can be formulated in our standard matrix notation as follows: min ||C^f ||2 = {^fTCTC^f} subject to ||g-Hˆf||2 - ||n||2 = 0 or ||g-Hˆf||2= ||n||2 where (recall zero-padding): 5/3/2019 Duong Anh Duc - Digital Image Processing
52
Constrained Least Squares Restoration
is a “smoothing matrix” and ˆf is a vector representing the restored image. 5/3/2019 Duong Anh Duc - Digital Image Processing
53
Constrained Least Squares Restoration
Notice that C is a block circulant matrix. As before, the solution to the above optimization problem is given by ^f = (HTH + CT C )-1HTg 5/3/2019 Duong Anh Duc - Digital Image Processing
54
Constrained Least Squares Restoration
Using properties of the block circulant matrix C, we get the following implementation of this filter: 5/3/2019 Duong Anh Duc - Digital Image Processing
55
Constrained Least Squares Restoration
Here P(u,v) is the 2D-DFT of matrix p(m,n), after appropriate zeropadding. Compare this with the parametric Wiener filter: no power spectrum information is required in the constrained leastsquares restoration! 5/3/2019 Duong Anh Duc - Digital Image Processing
56
Constrained Least Squares Restoration
However, for the new filter to be optimal, the parameter g must be chosen to satisfy the constraint ||g-Hˆf||= ||n||. Define the residual vector r = g-Hˆf = g - H (HTH + CT C )-1HTg Therefore, we need to choose g such that ||r|| = ||n|| It can be shown that the function ()= rTr = ||r||2 is a monotonically increasing function of g. We want to adjust g so that () = ||r||2 = ||n||2 ± a for some accuracy factor a. 5/3/2019 Duong Anh Duc - Digital Image Processing
57
Constrained Least Squares Restoration
Since () is monotonically increasing, this can be accomplished by the following procedure: 1. Specify an initial value of g = g0 2. For k = 1, 2 , …, compute ^fk = (HTH + kCT C )-1HTg. This can be done using a frequency domain implementation: ^Fk(u,v) = Rk(u,v)G(u,v), where 5/3/2019 Duong Anh Duc - Digital Image Processing
58
Constrained Least Squares Restoration
Compute 3. If (k)<||n||2–a, k+1=k+b, set k=k+1, return to step 2 If (k)>||n||2+a, k+1=k-b, set k=k+1, return to step 2 Otherwise, STOP (current ^fk or ^Fk(u,v) is the restored image and gk is the optimal choice of parameter g). 5/3/2019 Duong Anh Duc - Digital Image Processing
59
Constrained Least Squares Restoration
Implementation of this procedure requires knowledge of ||n||2, which denotes the strength of noise. If are the noise mean and variance, respectively, then 5/3/2019 Duong Anh Duc - Digital Image Processing
60
Constrained Least Squares Restoration
Therefore which can be computed from knowledge of the mean and variance of noise. 5/3/2019 Duong Anh Duc - Digital Image Processing
61
Constrained LS Example
f(m,n) g(m,n) 5/3/2019 Duong Anh Duc - Digital Image Processing
62
Constrained LS Example
^f(m,n), = 0.01 ^f(m,n), = 0.442 ^f(m,n), = 1 5/3/2019 Duong Anh Duc - Digital Image Processing
63
Duong Anh Duc - Digital Image Processing
Geometric Distortion 5/3/2019 Duong Anh Duc - Digital Image Processing
64
Gray-level Interpolation
5/3/2019 Duong Anh Duc - Digital Image Processing
65
Duong Anh Duc - Digital Image Processing
Example 5/3/2019 Duong Anh Duc - Digital Image Processing
66
Duong Anh Duc - Digital Image Processing
Example 5/3/2019 Duong Anh Duc - Digital Image Processing
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.