ELE 488 Fall 2006 Image Processing and Transmission 9/28/06 ELE 488 Fall 2006 Image Processing and Transmission 09-28-06 Edge Map Laplacian Median Filter Filtering Images in Frequency Domain Image Restoration There are many other examples not touched here. But with these examples, we can say something about why image processing. We saw some examples of enhancement, which makes the image looks better, or show more details. We saw some examples of extracting information from the images, both still images and video. We saw the same image have vastly different file sizes. This has to do with image coding and data compression. We also saw video encoding can be used to make it less vulnerable to transmission error. Image is information. How to store this kind of information, distribute it are important issued. A related topic is the digital library, which presents many challenges. Book, index, search, etc.

Gradient, 1st-order Derivatives, and Edges Edge: where luminance changes abruptly For binary image Take black pixels with immediate white neighbors as edge pixel For continuous-tone image Find luminance gradient partial derivatives along two orthogonal directions gives the direction with highest rate of changes If gradient is larger than a threshold => edge To represent edge Edge map ~ edge intensity & directions UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

The Gradient of Function f(x,y) The gradient of a smooth function f(x,y) at (x,y) is derivative of f in the x-direction, partial derivative of f w.r.t. x derivative of f in the y-direction, partial derivative of f w.r.t. y 2-vector pointing in the direction of greatest increase of f fr rate of change of f in the direction h f .

The Discrete Gradient The gradient of a real-valued discrete function f(m,n) at (j.k) is The symmetric rate of change of f in the m-direction at (j,k) The symmetric rate of change of f in the n-direction at (j,k) (Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)

Averaging GVF gradient is computed using high pass filters (discrete differentiation)  will amplify noise. So average neighboring gradient vectors using a spatial filtering on each of two gradient components. The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e., Can compute gradient and average, or Average pixels and then compute gradient. Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?

Averaging the GVF Spatial averaging filter perpendicular to direction of discrete derivative discrete derivative filter in horizontal direction Combination of the two masks gives single “averaged gradient mask” in horizontal direction.

Common GVF Masks Roberts Prewitt Sobel -1 1 -1 -2 1 2 1 -1 -1 1 -1 -2 1 2 1 -1 Apply both masks to image and combine results to determine magnitude and angle (if desired). Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise

Averaged GVF and Edges Large gradient magnitude Edge: Normal to direction of gradient Small gradient magnitude No edge Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge? Scaled & averaged gradient of apple image plotted at every 3rd pixel.

Example: Edge Detection from GVF Threshold magnitude of the GVF: Above threshold: edge pixel Below threshold: no edge

Sobel GVF Sobel Prewitt

Examples of Edge Detectors UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) Examples of Edge Detectors Quantize edge intensity to 0/1: set a threshold white pixel denotes strong edge Roberts Prewitt Sobel

Edges and Second Derivatives in 1-D “edges” are at points of local maximum and minimum slope of f(x). Here at x=1 and x=3. “edges” are at local maxima and minima of df/dx. Here at x=1 and x=3. Threshold the magnitude of the derivative…gives local region of edge. “edges” are at points where the second derivative passes through zero. Here at x=1 and x=3. From 1D to 2D

2D: Derivatives of f(x,y)

Example: Gaussian

Derivatives Red lines indicate zero level sets

Difference Operators (along rows, columns similar) Many choices: More generally:

Averaging Across Rows

Common Difference Masks Roberts Prewitt Sobel Prewitt and Sobel masks are separable. They spatially average across 3 rows (or columns).

Example Comparison of Sobel operation with actual derivative of Gaussian. Note scale difference of approx. 5.3 The error (after adjusting for scale & the edge effect) is shown below.

Second Differences Examples: In this case, change the 00 pixel of the second dm mask to make the result symmetric Examples:

Example: Second Difference Sobel Do the same for the Sobel column difference operator.

Example: 2nd Diff Sobel Comparison of Sobel operation with actual second derivative of Gaussian. Note scale difference of approx. 28.24 The error (after adjusting for scale and edge effect) is shown below.

A 3x3 Discrete Laplacians First difference. This operator is rotationally symmetric for 90 deg rotations. Roberts. This operator is rotationally symmetric for 90 deg rotations. Composite. This operator is rotationally symmetric for 90 deg rotations.

The 5x5 Sobel Laplacian Sobel. This operator is also rotationally symmetric for 90 deg rotations. As expected, sum of all entries is 0.

Zero Contour of 3x3 Roberts Laplacian

Zero Contour of Sobel Laplacian

Zero Contour of a 17x17 Laplacian

Spatial Masks Spatial Mask Examples: Finite, 2-D, real valued, array. One element is designated as (0,0). Used to compute the weighted sum of an input image, where weights are specified by the mask. Usually has small support 2x2, 3x3, 5x5, 7x7 Examples: These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc. 1/4 0 1 1 1/9 -1 0 1 -1 1/8 1/2

Use Averaging to Suppressing Noise Image with additive noise x(m,n) + N(m,n) Averaging over window of Nw pixels: v(m,n) = (1/Nw)  x(m-k, n-j) + (1/Nw)  N(m-k, n-j) Noise variance reduced by a factor of Nw SNR improved by a factor of Nw if x(m,n) is constant in local window Limit window size to avoid blurring UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) 1/4 0 1 1 1/9 -1 0 1 -1 1/8 1/2

Coping with Salt-and-Pepper Noise Matlab Image Toolbox Guide Fig.10-12, 10-13 Coping with Salt-and-Pepper Noise UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002) Median Filtering Median of pixel values in a window of size Nw Median is middle value of all pixel values if Nw is odd, and is the average of two middle values if Nw is even. Need to order data “Salt-and-Pepper” noise Isolated white or black pixels spread randomly over the image Spatial averaging filter can cause blur Median filtering Take median value over a small window as output Note: Median { x(m) + y(m) }  Median {x(m)} + Median {y(m)} nonlinear Commonly used window sizes 3x3, 5x5, 7x7 5x5 “+”–shaped window UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)

Other Ways of Image Sharpening Combine sharpening with histogram equalization UMCP ENEE408G Slides (created by M.Wu © 2002)

Image Processing Pointwise processing Spatial filtering Each output pixel depends only on one input pixel Histogram modification Spatial filtering Each output pixel may depend on more than one pixels Masking, linear filtering, edge detection, Linear filtering can be carried out in frequency domain Extending 1D processing in frequency domain to 2D

Signal Processing in Time and Frequency Domain Processor LTI system output signal y[n] input signal x[n] time domain: y[n] = Σ h[k] x[n – k] , h[n] impulse response Fourier transform: X(ω) = Σx[n] e–jnω. = FT{x[n]} Inverse Fourier transform: x[k] = (1/2π)∫ X(ω) ejnω dω = IFT {X(ω)} Frequency domain: Y(ω) = X(ω) H(ω), where Y(ω) = FT{ y[n] } , H(ω) = FT{ h[n] }

Discrete Fourier Transform FT: X(ω) = Σx[n] e–jnω. IFT: x[k] = (1/2π)∫ X(ω) ejnω dω . DFT: X[k] = Σ0≤n≤M–1 x[n] e–jn2π/M. IDFT: x[n] = (1/M)Σ0≤k≤M–1 X[k] e jk2π/M. Fast Fourier Transform (FFT)

2D Signal Processing in Time and Frequency Domain Processor LTI system output signal y[m,n] input signal x[m,n] time domain: y[m,n] = Σ h[k,l] x[m – k ,n – l] , h[m,n] impulse response Fourier transform: X(ω,λ) = Σx[m,n] e–j(mω+nλ) = FT{x[m,n]} Inverse Fourier transform: x[k,l] = (1/2π)∫ X(ω) ej(mω+nλ) dωdλ = IFT {X(ω)} Frequency domain: Y(ω ,λ) = X(ω ,λ) H(ω ,λ), where Y(ω ,λ) = FT{ y[m,n] } , H(ω ,λ) = FT{ h[m,n] }

2D Discrete Fourier Transform FT: X(ω ,λ) = Σx[n] e–j(nω +nλ) . IFT: x[k,l] = (1/2π)2∫ ∫ X(ω ,λ) e –j(nω +nλ) dω dλ . DFT: X[k,l] = Σ0≤m≤M–1 Σ0≤n≤N–1 x[m,n] e–j(m2π/M+n2π/N) . IDFT: x[m,n] = (1/MN) Σ0≤k≤M–1Σ0≤n≤N–1 X[k,l] ej(k2π/M+l2π/M) . Fast Fourier Transform (FFT) Separable

Review of Random Signals

Random Signal (cont)

2D Random Signal

2D Random Signal (cont)

UMCP ENEE631 Slides (created by M.Wu © 2001) Image Restoration UMCP ENEE631 Slides (created by M.Wu © 2001) From Matlab ImageToolbox Documentation pp12-4

Imperfection in Image Capturing H u(n1, n2 ) v(n1, n2 ) N(n1, n2 ) Blurring ~ linear spatial-invariant filter model w/ additive noise Impulse response h(n1, n2) & H(1, 2) Point Spread Function (PSF) ~ positive I/O If h(n1, n2) = (n1, n2), no blur Linear translational motion blur caused by local average along motion direction Uniform out-of-focus blur caused by local average in a circular neighborhood Atomspheric turbulence blur, etc. UMCP ENEE631 Slides (created by M.Wu © 2001)

Fourier Transform of PSF for Common Distortions UMCP ENEE631 Slides (created by M.Wu © 2001) From Bovik’s Handbook Sec.3.5 Fig.2&3