1. ELE 488 Fall 2006 Image Processing and Transmission
9/28/06
ELE 488 Fall 2006 Image Processing and Transmission
Edge Map
Laplacian
Median Filter
Filtering Images in Frequency Domain
Image Restoration
There are many other examples not touched here.
But with these examples, we can say something about why image processing.
We saw some examples of enhancement, which makes the image looks better, or show more details. We saw some examples of extracting information from the images, both still images and video. We saw the same image have vastly different file sizes. This has to do with image coding and data compression. We also saw video encoding can be used to make it less vulnerable to transmission error.
Image is information. How to store this kind of information, distribute it are important issued. A related topic is the digital library, which presents many challenges. Book, index, search, etc.

2. Gradient, 1st-order Derivatives, and Edges
Edge: where luminance changes abruptly
For binary image
Take black pixels with immediate white neighbors as edge pixel
For continuous-tone image
Find luminance gradient
partial derivatives along two orthogonal directions
gives the direction with highest rate of changes
If gradient is larger than a threshold => edge
To represent edge
Edge map ~ edge intensity & directions
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

3. The Gradient of Function f(x,y)
The gradient of a smooth function f(x,y) at (x,y) is
derivative of f in the x-direction, partial derivative of f w.r.t. x
derivative of f in the y-direction, partial derivative of f w.r.t. y
2-vector pointing in the direction of greatest increase of f fr
rate of change of f in the direction h f .

4. The Discrete Gradient
The gradient of a real-valued discrete function f(m,n) at (j.k) is
The symmetric rate of change of f in the m-direction at (j,k)
The symmetric rate of change of f in the n-direction at (j,k)
(Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)

5. Averaging GVF
gradient is computed using high pass filters (discrete differentiation)  will amplify noise.
So average neighboring gradient vectors using a spatial filtering on each of two gradient components.
The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e.,
Can compute gradient and average, or
Average pixels and then compute gradient.
Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?

6. Averaging the GVF
Spatial averaging filter perpendicular to direction of discrete derivative
discrete derivative filter in horizontal direction
Combination of the two masks gives single “averaged gradient mask” in horizontal direction.

7. Common GVF Masks
Roberts
Prewitt
Sobel -1 1 -1 -2 1 2 1 -1 -1 1 -1 -2 1 2 1 -1
Apply both masks to image and combine results to determine magnitude and angle (if desired).
Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise

8. Averaged GVF and Edges Large gradient magnitude Edge:
Normal to direction of gradient
Small gradient magnitude
No edge
Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge?
Scaled & averaged gradient of apple image plotted at every 3rd pixel.

9. Example: Edge Detection from GVF
Threshold magnitude of the GVF:
Above threshold:
edge pixel
Below threshold:
no edge

10. Sobel GVF
Sobel
Prewitt

11. Examples of Edge Detectors
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
Examples of Edge Detectors
Quantize edge intensity to 0/1:
set a threshold
white pixel denotes strong edge
Roberts
Prewitt
Sobel

12. Edges and Second Derivatives in 1-D
“edges” are at points of local maximum and minimum slope of f(x). Here at x=1 and x=3.
“edges” are at local maxima and minima of df/dx. Here at x=1 and x=3. Threshold the magnitude of the derivative…gives local region of edge.
“edges” are at points where the second derivative passes through zero. Here at x=1 and x=3.
From 1D to 2D

13. 2D: Derivatives of f(x,y)

14. Example: Gaussian

15. Derivatives
Red lines indicate zero level sets

16. Difference Operators (along rows, columns similar)
Many choices:
More generally:

17. Averaging Across Rows

18. Common Difference Masks
Roberts
Prewitt
Sobel
Prewitt and Sobel masks are separable. They spatially average across 3 rows (or columns).

19. Example
Comparison of Sobel operation with actual derivative of Gaussian.
Note scale difference of approx. 5.3
The error (after adjusting for scale & the edge effect) is shown below.

20. Second Differences Examples:
In this case, change the 00 pixel of the second dm mask to make the result symmetric
Examples:

21. Example: Second Difference Sobel
Do the same for the Sobel column difference operator.

22. Example: 2nd Diff Sobel
Comparison of Sobel operation with actual second derivative of Gaussian.
Note scale difference of approx
The error (after adjusting for scale and edge effect) is shown below.

23. A 3x3 Discrete Laplacians
First difference. This operator is rotationally symmetric for 90 deg rotations.
Roberts. This operator is rotationally symmetric for 90 deg rotations.
Composite. This operator is rotationally symmetric for 90 deg rotations.

24. The 5x5 Sobel Laplacian
Sobel. This operator is also rotationally symmetric for 90 deg rotations. As expected, sum of all entries is 0.

25. Zero Contour of 3x3 Roberts Laplacian

26. Zero Contour of Sobel Laplacian

27. Zero Contour of a 17x17 Laplacian

28. Spatial Masks Spatial Mask Examples: Finite, 2-D, real valued, array.
One element is designated as (0,0).
Used to compute the weighted sum of an input image, where weights are specified by the mask.
Usually has small support 2x2, 3x3, 5x5, 7x7
Examples:
These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.
1/4 1
1/9 -1
1/8
1/2

29. Use Averaging to Suppressing Noise
Image with additive noise x(m,n) + N(m,n)
Averaging over window of Nw pixels:
v(m,n) = (1/Nw)  x(m-k, n-j) + (1/Nw)  N(m-k, n-j)
Noise variance reduced by a factor of Nw SNR improved by a factor of Nw if x(m,n) is constant in local window
Limit window size to avoid blurring
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
1/4 1
1/9 -1
1/8
1/2

30. Coping with Salt-and-Pepper Noise
Matlab Image Toolbox Guide Fig.10-12, 10-13
Coping with Salt-and-Pepper Noise
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

31. UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
Median Filtering
Median of pixel values in a window of size Nw
Median is middle value of all pixel values if Nw is odd, and is the average of two middle values if Nw is even.
Need to order data
“Salt-and-Pepper” noise
Isolated white or black pixels spread randomly over the image
Spatial averaging filter can cause blur
Median filtering
Take median value over a small window as output
Note: Median { x(m) + y(m) }  Median {x(m)} + Median {y(m)}
nonlinear
Commonly used window sizes
3x3, 5x5, 7x7
5x5 “+”–shaped window
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

32. Other Ways of Image Sharpening
Combine sharpening with histogram equalization
UMCP ENEE408G Slides (created by M.Wu © 2002)

33. Image Processing Pointwise processing Spatial filtering
Each output pixel depends only on one input pixel
Histogram modification
Spatial filtering
Each output pixel may depend on more than one pixels
Masking, linear filtering, edge detection,
Linear filtering can be carried out in frequency domain
Extending 1D processing in frequency domain to 2D

34. Signal Processing in Time and Frequency Domain
Processor
LTI system
output signal y[n]
input signal x[n]
time domain: y[n] = Σ h[k] x[n – k] , h[n] impulse response
Fourier transform: X(ω) = Σx[n] e–jnω. = FT{x[n]}
Inverse Fourier transform: x[k] = (1/2π)∫ X(ω) ejnω dω = IFT {X(ω)}
Frequency domain: Y(ω) = X(ω) H(ω),
where Y(ω) = FT{ y[n] } , H(ω) = FT{ h[n] }

35. Discrete Fourier Transform
FT: X(ω) = Σx[n] e–jnω IFT: x[k] = (1/2π)∫ X(ω) ejnω dω .
DFT: X[k] = Σ0≤n≤M–1 x[n] e–jn2π/M.
IDFT: x[n] = (1/M)Σ0≤k≤M–1 X[k] e jk2π/M.
Fast Fourier Transform (FFT)

36. 2D Signal Processing in Time and Frequency Domain
Processor
LTI system
output signal y[m,n]
input signal x[m,n]
time domain: y[m,n] = Σ h[k,l] x[m – k ,n – l] , h[m,n] impulse response
Fourier transform: X(ω,λ) = Σx[m,n] e–j(mω+nλ) = FT{x[m,n]}
Inverse Fourier transform: x[k,l] = (1/2π)∫ X(ω) ej(mω+nλ) dωdλ = IFT {X(ω)}
Frequency domain: Y(ω ,λ) = X(ω ,λ) H(ω ,λ),
where Y(ω ,λ) = FT{ y[m,n] } , H(ω ,λ) = FT{ h[m,n] }

37. 2D Discrete Fourier Transform
FT: X(ω ,λ) = Σx[n] e–j(nω +nλ) .
IFT: x[k,l] = (1/2π)2∫ ∫ X(ω ,λ) e –j(nω +nλ) dω dλ .
DFT: X[k,l] = Σ0≤m≤M–1 Σ0≤n≤N–1 x[m,n] e–j(m2π/M+n2π/N) .
IDFT: x[m,n] = (1/MN) Σ0≤k≤M–1Σ0≤n≤N–1 X[k,l] ej(k2π/M+l2π/M) .
Fast Fourier Transform (FFT)
Separable

38. Review of Random Signals

39. Random Signal (cont)

40. 2D Random Signal

41. 2D Random Signal (cont)

42. UMCP ENEE631 Slides (created by M.Wu © 2001)
Image Restoration
UMCP ENEE631 Slides (created by M.Wu © 2001)
From Matlab ImageToolbox Documentation pp12-4

43. Imperfection in Image CapturingH
u(n1, n2 )
v(n1, n2 )
N(n1, n2 )
Blurring ~ linear spatial-invariant filter model w/ additive noise
Impulse response h(n1, n2) & H(1, 2)
Point Spread Function (PSF) ~ positive I/O
If h(n1, n2) = (n1, n2), no blur
Linear translational motion blur caused
by local average along motion direction
Uniform out-of-focus blur caused by
local average in a circular neighborhood
Atomspheric turbulence blur, etc.
UMCP ENEE631 Slides (created by M.Wu © 2001)

44. Fourier Transform of PSF for Common Distortions
UMCP ENEE631 Slides (created by M.Wu © 2001)
From Bovik’s Handbook Sec.3.5 Fig.2&3