Chapter 5 Neighborhood Processing Introduction Filters (LPF, HPF, Gaussian Filter, …) Region of interest processing SCCS 476

Basic Image Processing Operations Neighborhood processing process the pixel with its neighbors Point operations process according to the pixel’s value alone. SCCS 476

Neighborhood Processing 3x3 Mask Output derives from multiplying all elements in the mask by corresponding elements in the neighborhood and adding together all these products. SCCS 476

Filter A rule or procedure for processing an image Combination of mask and function Goal: separating/attenuating a desired component of an observed image Type: Linear (function), Nonlinear (function) Low-pass filter (LPF), High-pass filter (HPF), Band-pass filter (BPF) SCCS 476

Steps of Linear Spatial Filtering Position the mask over the current pixel. Form all products of filter elements with the corresponding elements of the neighborhood. Add all products. Other names for mask: filter, mask, filter mask, kernel, template or window, etc. SCCS 476

Linear Spatial Filter: Example (1) 1/9 2 15 10 20 10 Average Filter 10 10 15 10 5 20 10 10 15 Find the output intensity of the blue pixel. SCCS 476

Linear Spatial Filter: Example (2) 15 10 20 1/9 1/9 1/9 1/9 10 1/9 15 1/9 1/9 1/9 1/9 20 10 10 Multiply the number in the filter’s element with the corresponding pixel’s intensity. SCCS 476

Linear Spatial Filter: Example (3) 15/9 + 10/9 + 20/9 + 0/9 + 10/9 + 15/9 + 20/9 + 10/9 + 10/9 = 12.22 15/9 10/9 20/9 0/9 10/9 15/9 20/9 10/9 10/9 Output intensity of blue pixel = 12.22 Add all products for output. Spatial filtering is spatial convolution. SCCS 476

Correlation Sum of the product of mask and intensity on each point. SCCS 476

Example: Correlation (1) 3 2 4 5 10 20 10 15 5 5 10 15 10 Kernel 20 10 20 20 15 5 10 5 10 10 20 15 10 SCCS 476

Example: Correlation (2) 101 +152 +103 +102 +203 +154 +53 +04 +05 = 225 10 15 10 1 2 3 10 2 20 3 15 4 3 4 5 5 Multiply and sum all products. SCCS 476

Exercise: Correlation 1 3 2 4 5 Kernel 1 10 Find the correlation of the green kernel to the above image. SCCS 476

Convolution Sum of the response on each point SCCS 476

Example: Convolution (1) 5 3 4 2 1 10 20 10 15 5 5 10 15 10 Reverse Kernel!! 20 10 20 20 15 5 10 5 10 1 3 2 4 5 10 20 15 10 Kernel SCCS 476

Example: Convolution (2) 105 +154 +103 +104 +203 +152 +53 +02 +01 = 285 10 15 10 5 4 3 10 4 20 3 15 2 3 2 1 5 Multiply and sum all products. SCCS 476

Exercise: Convolution 1 3 2 4 5 Kernel 1 10 Find the convolution of the green kernel to the above image. SCCS 476

Notation A linear filter is represented as a matrix, e.g., the 3 x 3 averaging filter. 1/9 SCCS 476

Notation (2) Edges of the Image There are a number of different approaches to dealing with this problem. Ignore the edges: The mask is applied to all pixels except the edges and results in an output image that is smaller than the original. If the mask is very large, a significant amount of information may be lost. Pad with zeros: We assume that all necessary values outside the image are zero. It will return an output image of the same size as the original, but may have the effect of introducing unwanted artifacts, e.g., edges, around the image. SCCS 476

Filtering in MATLAB Command: filter2 Syntax: filter2(filter, image, shape); filter2(filter, image); shape: ‘same’: pad edge with zeros. Size unchanged. (default) ‘valid’: apply mask only to inside pixel. Size smaller. ‘full’: pad edge with zeros and applying the filter at all places on and around the image where the mask intersects the image matrix. Size larger. SCCS 476

Filter Construction in MATLAB Command: fspecial Syntax: fspecial(type, parameter); fspecial(type); type: type of the filter ‘average’ : average filter ‘gaussian’ : Gaussian filter ‘laplacian’ : Laplacian filter … parameter: parameter of the filter (size, sigma, …). Default varies among filter. Try!!! SCCS 476

Separable Filters Some filters can be implemented by the successive application of two simple filters. SCCS 476

Separable Filters (2) Speed up the processing by filtering in one axis at a time. working with 2D n x n matrix requires n2, O(n2), multiplications, and n2 – 1, O(n2), addition for each pixel. working with 1D n x 1 matrix twice requires 2n, O(n), multiplications, and 2n – 2, O(n), additions for each pixel. SCCS 476

Examples of Separable Filter Average filter Laplacian filter SCCS 476

Filter on Frequency Domain Low-pass filter (LPF): filter that allows only the low-frequency components and reduces or eliminates the high-frequency components. E.g. Gaussian, average High-pass filter (HPF): filter that allows only the high-frequency components and reduces or eliminates the low-frequency components. E.g. Laplacian, Prewitt, Sobel SCCS 476

Frequency Components Spatial data (intensity) transformed by Fourier transform. Simplified version: high-frequency indicates the abrupt changes in intensity  edges. low-frequency indicates the intensity smoothness  uniform region. SCCS 476

Example: HPF Kernel Input image 1 -2 4 10 20 10 15 5 5 10 15 10 20 10 20 10 20 15 5 Kernel 10 5 10 10 20 15 10 Input image SCCS 476

Example: HPF (2) Filtered Image 10 20 10 15 5 5 20 -10 20 -40 25 15 5 20 -10 20 -40 25 15 5 10 5 -35 10 10 20 15 10 Filtered Image SCCS 476

Computing Consideration Filter may lead to the value outside [0,255] Solution 1: Make negative values positive (use absolute value)  good when there are few negative values and the negative values are close to zero Solution 2: Clip values. Values larger than 255 become 255 and values less than 0 become 0.  not good if there are many values outside the range. Solution 3: Scaling transformation. Rescale the range to [0,255] by pixel transform. Suppose gL and gH are the lowest and the highest values. SCCS 476

Rescaling Intensity 1. Map gL to 0. 2. Map gH to 255. Rescaled value (y) 1. Map gL to 0. 255 2. Map gH to 255. 3. Interpolate for the remaining intensity. Filtered value (x) gL gH SCCS 476

Rescaling: MATLAB Manual: >> gH = max(filtered_image(:)); >> gL = min(filtered_image(:)); >> scaled = (image – gmin)/(gmax – gmin); Note: No need to rescale to 255 because intensity range for double image is [0,1]. SCCS 476

Rescaling: MATLAB (2) Command: mat2gray Syntax: mat2gray(double_image); What this command do? scale the value in double_image to displayable value. Output is double type. minimum value is mapped to 0. maximum value is mapped to 1. SCCS 476

Gaussian Filter 1D Gaussian filter: 2D Gaussian filter: SCCS 476 http://www-mmdb.iai.uni-bonn.de/lehre/BIT/ss03_DSP_Vorlesung/matlab_demos/ http://upload.wikimedia.org/wikipedia/su/thumb/3/38/Gaussian-pdf.png/300px-Gaussian-pdf.png SCCS 476

Benefits of Gaussian Filter They are mathematically very well behaved. The Fourier transform of a Gaussian filter is another Gaussian. There are rotationally symmetric, so are very good starting points for some edge-detection algorithms. They are separable in x and y axes. This can lead to very fast implementations. The convolution of two Gaussians is another Gaussian. SCCS 476

Gaussian Filter (2) Effect of Gaussian filter = blurring larger leads to more blur.  average filter Construction of Gaussian filter: command: fspecial(‘gaussian’, size, gamma); size : size of the filter [row column], default [3 3] gamma : , default 0.5. SCCS 476

Gaussian Filter: MATLAB >>gaussian1 = fspecial(‘gaussian’, [5 5], 5); Create the 55 Gaussian filter with the  value of 5. >>gaussian2 = fspecial(‘gaussian’, 3, 0.75);  Create the 33 Gaussian filter with the  value of 0.75. SCCS 476

Effect of Gaussian Filter  = 0.5  = 2  = 5 Filter size = 5  5 SCCS 476

Effect of Gaussian Filter (2) Filter size = 3  3 Filter size = 5  5  = 5 SCCS 476

Edge Sharpening Also known as edge enhancement, edge crispening, unsharp masking. Process to make the edge slightly sharper and crisper. E.g. linear edge sharpening, unsharp masking, high boost filtering SCCS 476

Linear Edge Sharpening High pass filter (HPF) Increase the edge power Example: SCCS 476

Unsharp Masking Original Image Scale for Display Blurred with LPF with a < 1 SCCS 476

Unsharp Masking: MALAB >> f = fspecial(‘average’);  Create LPF (E.g. average filter) >> xf = filter2(f,x);  Filter input image x by average filter. >> xu = double(x) – xf/1.5;  Substract the low passed components. >> imshow (xu/70)  Scale intensity for better viewing. SCCS 476

Unsharp Filter [unsharp_image] = [input] – (a [filter]  [input]) = (I* – a [filter])  [input] = [unsharp_filter]  [input] I* : matrix whose center member is 1 and the others are zero. E.g. for 3  3 matrix I* = SCCS 476

Unsharp Filter (2) For 33 matrix, after some rearranging term: MATLAB use this format for the unsharp filter with the default  of 0.2. SCCS 476

Unsharp Filter: MATLAB Command: fspecial Syntax: fspecial(‘unsharp’, alpha); alpha : alpha value for unsharp filter Size of the filter is fixed to 3  3 SCCS 476

Effect of Unsharp Masking BEFORE AFTER http://ise.hansung.ac.kr/jun/DI/Chapter-5.htm SCCS 476

High-Boost Filtering [high_boost] = A [input] – [low_passed] Original Image Scale with A>1 Scale for Display Blurred with LPF Amplification factor. A=1, high boost = HPF [high_boost] = A [input] – [low_passed] SCCS 476

High Boost Filter (2) high_boost = A input – low_pass = A input – (input - high_pass) = (A – 1)  input + high_pass Rescale high_boost by w. w high_boost = w (A – 1)  input + w high_pass Best result when w (A – 1) = 1.  w = 1/(A – 1) SCCS 476

High Boost Filter (3) General high boost filter: Another version: Best result when 3/5  A  5/6 Used in dark image. Boost the intensity of the original image. SCCS 476

Effect of High Boost Filter BEFORE AFTER A = 5/6 SCCS 476

Non-linear Filter A non-linear filter is obtained by a nonlinear function of the grayscale values in the mask. E.g. rank-order filters (minimum filter, maximum filter, median filter); the elements under the mask are ordered. MATLAB command: nlfilter A faster alternative: colfilt (rearranges the image into column) Good for removing impulse-like noise Better tradeoff between smoothing and retention of fine image details. SCCS 476

MATLAB Command: nlfilter Syntax: nlfilter(Image,[m,n],func); Image : input image [m n] : size of the neighborhood func : function to be applied to the neighborhood E.g. >> cmed = nlfilter(image, [3,3], ‘median(x(:))’); 33 sliding box Median function SCCS 476

MATLAB Command: colfilt Columnwise neighborhood operation Syntax: colfilt(im, [m n], block_type, func) im : input image [m n] : size that image matrix to be arranged into block_type : how to rearrange im matrix. func : function to be applied to the neighborhood E.g. >> cmed = colfilt(image, [3,3], ‘sliding’, @median); SCCS 476

Block_type in colfilt ‘distinct’: ‘sliding’ “Rearranges each distinct m-by-n block in the image A into a column of B. im2col pads A with 0's, if necessary, so its size is an integer multiple of m-by-n. If A = [A11 A12; A21 A22], where each Aij is m-by-n, then B = [A11(:) A12(:) A21(:) A22(:)].” ‘sliding’ “Converts each sliding m-by-n block of A into a column of B, with no zero padding. B has m*n rows and contains as many columns as there are m-by-n neighborhoods of A. SCCS 476

Rearrangement by ‘distinct’ Rearrange to column vector B = im2col(A,[m n],’distinct’); a1 a2 a3 a4 a5 a6 a7 a8 a9 a1 Input Image a2 a3 a4 a5 a6 a7 a8 a9 Pad this area with 0 mn Block A B SCCS 476

Rearrangement by ‘sliding’ B = im2col(A,[m n],’sliding’); a0 Input Image a1 a2 an an+1 amn mn Block A B SCCS 476

Rank-Order Filter Sort the intensities within the mask. Choose the intensity at ith position as output. Min. filter 10 20 10 15 5 5 10 10 13 13 5 5 20 11 11 20 20 20 15 15 5 Median filter Sort intensity 10 8 8 10 10 20 15 10 SCCS 476 Max. filter

Maximum Filter Output pixel is the maximum intensity of the pixels within the mask. (find brightest point) BEFORE AFTER Image corrupted by pepper noise SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/min_max.html

Minimum Filter Output pixel is the minimum intensity of the pixels within the mask. (find darkest point) BEFORE AFTER Image corrupted by salt noise SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/max_min.html

Median Filter Output pixel is the mid-intensity of the pixels within the mask (the median intensity). SCCS 476 www.ee.siue.edu/~sumbaug/ CVIPbook_PPLec/Chapter%209b.ppt

Effect of Median Filter BEFORE AFTER 3x3 Kernel SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/

Rank-Order Filtering: MATLAB Command: ordfilt2 Syntax: ordfilt2(image, order, domain); image : input image order : which order of the sorted intensity (minimum to maximum value) taken as output domain : matrix indicating the neighborhood. 1 : pixels in the neighbor. 0 : pixels not in the neighbor E.g. cmin = ordfilt2(image, 1, ones(3,3)); SCCS 476

Geometric Mean Filter x(i, j): pixel’s intensity M : filter mask |M| : size of the mask SCCS 476

Effect of Geometric Mean Filter BEFORE AFTER Image corrupted by Gaussian noise with variance = 300, mean = 0 SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html

Bad Effect of Geometric Mean Filter BEFORE AFTER Image corrupted by pepper noise with probability = 0.4 SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/geometric.html

Alpha-Trimmed Mean Filter Output is the mean of the data after removing the first d/2 and the last d/2 ordered data. d =2 10 20 10 15 5 Trim the data by 2. (1 from the top. 1 from the bottom.) 5 10 13 5 5 8 20 11 20 20 15 5 10 Sort intensity Output = average intensity of the remaining data. = 9.5 11 10 8 10 13 10 20 15 10 15 20 SCCS 476

Effect of Alpha-Trimmed Mean Filter BEFORE AFTER Image corrupted by salt-and-pepper noise with variance = 200, mean = 0 Trim size = 2, mask size =1 SCCS 476 http://www.ee.siue.edu/~cvip/CVIPtools_demos/RESTORATION/alpha.html

Region of Interest Processing Process image only in the predefined area. The predefined area is called “Region Of Interest” (ROI). Function is the same as before but applied to only ROI. SCCS 476

ROI Selection: MATLAB Command: roipoly Syntax: roipoly(im); roipoly(im, [x0 x1 …xm], [y0 y1 …ym]; im : input image (x0, y0), (x1, y1), …, (xm, ym) : coordinate of the polygon covering region of interest. E.g. roi = roipoly(im, [60 100 100 60], [40 40 80 80]); SCCS 476

Mask for ROI Binary image with the white (1) indicating ROI. Mask coordinate: (222, 21) (272, 21) (300, 75) (270, 121) (221, 121) (191, 75) SCCS 476 http://www.mathworks.com/access/helpdesk/help/toolbox/images/roipoly.html

ROI Filtering: MATLAB Command: roifilt2 Syntax: roifilt2(filter, image, roi); filter : 2D linear filter image : input image roi : region of interest (1: ROI, 0: not ROI) SCCS 476

Effect of ROI Filtering ROI at this coin Unsharp masking in ROI. SCCS 476 http://www.mathworks.com/access/helpdesk/help/toolbox/images/roifilt2.html