1. Chapter 5 Neighborhood Processing
Introduction
Filters (LPF, HPF, Gaussian Filter, …)
Region of interest processing
SCCS 476

2. Basic Image Processing Operations
Neighborhood processing
process the pixel with its neighbors
Point operations
process according to the pixel’s value alone.
SCCS 476

3. Neighborhood Processing
3x3 Mask
Output derives from multiplying all elements in the mask by corresponding elements in the neighborhood and adding together all these products.
SCCS 476

4. Filter A rule or procedure for processing an image
Combination of mask and function
Goal: separating/attenuating a desired component of an observed image
Type:
Linear (function), Nonlinear (function)
Low-pass filter (LPF), High-pass filter (HPF), Band-pass filter (BPF)
SCCS 476

5. Steps of Linear Spatial Filtering
Position the mask over the current pixel.
Form all products of filter elements with the corresponding elements of the neighborhood.
Add all products.
Other names for mask:
filter, mask, filter mask, kernel, template or window, etc.
SCCS 476

6. Linear Spatial Filter: Example (1)
1/9 2 15 10 20 10
Average Filter 10 10 15 10 5 20 10 10 15
Find the output intensity of the blue pixel.
SCCS 476

7. Linear Spatial Filter: Example (2)15 10 20
1/9
1/9
1/9
1/9 10
1/9 15
1/9
1/9
1/9
1/9 20 10 10
Multiply the number in the filter’s element with the corresponding pixel’s intensity.
SCCS 476

8. Linear Spatial Filter: Example (3)
15/9 + 10/9 + 20/9 + 0/ /9 + 15/9 + 20/9 + 10/9 + 10/9
= 12.22
15/9
10/9
20/9
0/9
10/9
15/9
20/9
10/9
10/9
Output intensity of blue pixel
= 12.22
Add all products for output.
Spatial filtering is spatial convolution.
SCCS 476

9. Correlation Sum of the product of mask and intensity on each point.
SCCS 476

10. Example: Correlation (1)3 2 4 5 10 20 10 15 5 5 10 15 10
Kernel 20 10 20 20 15 5 10 5 10 10 20 15 10
SCCS 476

11. Example: Correlation (2)
101 +152 +103 +102 +203 +154 +53 +04 +05
= 225 10 15 10 1 2 3 10 2 20 3 15 4 3 4 5 5
Multiply and sum all products.
SCCS 476

12. Exercise: Correlation1 3 2 4 5
Kernel 1 10
Find the correlation of the green kernel to the above image.
SCCS 476

13. Convolution
Sum of the response on each point
SCCS 476

14. Example: Convolution (1)5 3 4 2 1 10 20 10 15 5 5 10 15 10
Reverse Kernel!! 20 10 20 20 15 5 10 5 10 1 3 2 4 5 10 20 15 10
Kernel
SCCS 476

15. Example: Convolution (2)
105 +154 +103 +104 +203 +152 +53 +02 +01
= 285 10 15 10 5 4 3 10 4 20 3 15 2 3 2 1 5
Multiply and sum all products.
SCCS 476

16. Exercise: Convolution1 3 2 4 5
Kernel 1 10
Find the convolution of the green kernel to the above image.
SCCS 476

17. Notation
A linear filter is represented as a matrix, e.g., the 3 x 3 averaging filter.
1/9
SCCS 476

18. Notation (2) Edges of the Image
There are a number of different approaches to dealing with this problem.
Ignore the edges: The mask is applied to all pixels except the edges and results in an output image that is smaller than the original. If the mask is very large, a significant amount of information may be lost.
Pad with zeros: We assume that all necessary values outside the image are zero. It will return an output image of the same size as the original, but may have the effect of introducing unwanted artifacts, e.g., edges, around the image.
SCCS 476

19. Filtering in MATLAB Command: filter2
Syntax: filter2(filter, image, shape);
filter2(filter, image);
shape:
‘same’: pad edge with zeros. Size unchanged. (default)
‘valid’: apply mask only to inside pixel. Size smaller.
‘full’: pad edge with zeros and applying the filter at all places on and around the image where the mask intersects the image matrix. Size larger.
SCCS 476

20. Filter Construction in MATLAB
Command: fspecial
Syntax: fspecial(type, parameter);
fspecial(type);
type: type of the filter
‘average’ : average filter
‘gaussian’ : Gaussian filter
‘laplacian’ : Laplacian filter …
parameter: parameter of the filter (size, sigma, …). Default varies among filter. Try!!!
SCCS 476

21. Separable Filters
Some filters can be implemented by the successive application of two simple filters.
SCCS 476

22. Separable Filters (2)
Speed up the processing by filtering in one axis at a time.
working with 2D n x n matrix requires n2, O(n2), multiplications, and n2 – 1, O(n2), addition for each pixel.
working with 1D n x 1 matrix twice requires 2n, O(n), multiplications, and 2n – 2, O(n), additions for each pixel.
SCCS 476

23. Examples of Separable Filter
Average filter
Laplacian filter
SCCS 476

24. Filter on Frequency Domain
Low-pass filter (LPF): filter that allows only the low-frequency components and reduces or eliminates the high-frequency components.
E.g. Gaussian, average
High-pass filter (HPF): filter that allows only the high-frequency components and reduces or eliminates the low-frequency components.
E.g. Laplacian, Prewitt, Sobel
SCCS 476

25. Frequency Components
Spatial data (intensity) transformed by Fourier transform.
Simplified version:
high-frequency indicates the abrupt changes in intensity  edges.
low-frequency indicates the intensity smoothness  uniform region.
SCCS 476

26. Example: HPF Kernel Input image 1 -2 4 10 20 10 15 5 5 10 15 10 20 1020 10 20 15 5
Kernel 10 5 10 10 20 15 10
Input image
SCCS 476

27. Example: HPF (2) Filtered Image 10 20 10 15 5 5 20 -10 20 -40 25 15 520
-10 20
-40 25 15 5 10 5
-35 10 10 20 15 10
Filtered Image
SCCS 476

28. Computing Consideration
Filter may lead to the value outside [0,255]
Solution 1: Make negative values positive (use absolute value)  good when there are few negative values and the negative values are close to zero
Solution 2: Clip values. Values larger than 255 become 255 and values less than 0 become 0.  not good if there are many values outside the range.
Solution 3: Scaling transformation. Rescale the range to [0,255] by pixel transform. Suppose gL and gH are the lowest and the highest values.
SCCS 476

29. Rescaling Intensity 1. Map gL to 0. 2. Map gH to 255.
Rescaled value (y)
1. Map gL to 0.
255
2. Map gH to 255.
3. Interpolate for the remaining intensity.
Filtered value (x) gL gH
SCCS 476

30. Rescaling: MATLAB Manual: >> gH = max(filtered_image(:));
>> gL = min(filtered_image(:));
>> scaled = (image – gmin)/(gmax – gmin);
Note: No need to rescale to 255 because intensity range for double image is [0,1].
SCCS 476

31. Rescaling: MATLAB (2) Command: mat2gray
Syntax: mat2gray(double_image);
What this command do?
scale the value in double_image to displayable value. Output is double type.
minimum value is mapped to 0.
maximum value is mapped to 1.
SCCS 476

32. Gaussian Filter 1D Gaussian filter: 2D Gaussian filter: SCCS 476
SCCS 476

33. Benefits of Gaussian Filter
They are mathematically very well behaved. The Fourier transform of a Gaussian filter is another Gaussian.
There are rotationally symmetric, so are very good starting points for some edge-detection algorithms.
They are separable in x and y axes. This can lead to very fast implementations.
The convolution of two Gaussians is another Gaussian.
SCCS 476

34. Gaussian Filter (2) Effect of Gaussian filter = blurring
larger leads to more blur.
 average filter
Construction of Gaussian filter:
command: fspecial(‘gaussian’, size, gamma);
size : size of the filter [row column], default [3 3]
gamma : , default 0.5.
SCCS 476

35. Gaussian Filter: MATLAB
>>gaussian1 = fspecial(‘gaussian’, [5 5], 5);
Create the 55 Gaussian filter with the  value of 5.
>>gaussian2 = fspecial(‘gaussian’, 3, 0.75);
 Create the 33 Gaussian filter with the  value of 0.75.
SCCS 476

36. Effect of Gaussian Filter
 = 0.5
 = 2
 = 5
Filter size = 5  5
SCCS 476

37. Effect of Gaussian Filter (2)
Filter size = 3  3
Filter size = 5  5
 = 5
SCCS 476

38. Edge Sharpening
Also known as edge enhancement, edge crispening, unsharp masking.
Process to make the edge slightly sharper and crisper.
E.g. linear edge sharpening, unsharp masking, high boost filtering
SCCS 476

39. Linear Edge Sharpening
High pass filter (HPF)
Increase the edge power
Example:
SCCS 476

40. Unsharp Masking Original Image Scale for Display Blurred with LPF
with a < 1
SCCS 476

41. Unsharp Masking: MALAB
>> f = fspecial(‘average’);
 Create LPF (E.g. average filter)
>> xf = filter2(f,x);
 Filter input image x by average filter.
>> xu = double(x) – xf/1.5;
 Substract the low passed components.
>> imshow (xu/70)
 Scale intensity for better viewing.
SCCS 476

42. Unsharp Filter [unsharp_image] = [input] – (a [filter]  [input])
= (I* – a [filter])  [input]
= [unsharp_filter]  [input]
I* : matrix whose center member is 1 and the others are zero.
E.g. for 3  3 matrix I* =
SCCS 476

43. Unsharp Filter (2) For 33 matrix, after some rearranging term:
MATLAB use this format for the unsharp filter with the default  of 0.2.
SCCS 476

44. Unsharp Filter: MATLAB
Command: fspecial
Syntax: fspecial(‘unsharp’, alpha);
alpha : alpha value for unsharp filter
Size of the filter is fixed to 3  3
SCCS 476

45. Effect of Unsharp Masking
BEFORE
AFTER
SCCS 476

46. High-Boost Filtering [high_boost] = A [input] – [low_passed] Original
Image
Scale
with A>1
Scale for
Display
Blurred with
LPF
Amplification factor. A=1, high boost = HPF
[high_boost] = A [input] – [low_passed]
SCCS 476

47. High Boost Filter (2) high_boost = A input – low_pass
= A input – (input - high_pass)
= (A – 1)  input + high_pass
Rescale high_boost by w.
w high_boost = w (A – 1)  input + w high_pass
Best result when w (A – 1) = 1.
 w = 1/(A – 1)
SCCS 476

48. High Boost Filter (3) General high boost filter: Another version:
Best result when 3/5  A  5/6
Used in dark image.
Boost the intensity of the original image.
SCCS 476

49. Effect of High Boost Filter
BEFORE
AFTER
A = 5/6
SCCS 476

50. Non-linear Filter
A non-linear filter is obtained by a nonlinear function of the grayscale values in the mask.
E.g. rank-order filters (minimum filter, maximum filter, median filter); the elements under the mask are ordered.
MATLAB command: nlfilter
A faster alternative: colfilt (rearranges the image into column)
Good for removing impulse-like noise
Better tradeoff between smoothing and retention of fine image details.
SCCS 476

51. MATLAB Command: nlfilter
Syntax: nlfilter(Image,[m,n],func);
Image : input image
[m n] : size of the neighborhood
func : function to be applied to the neighborhood
E.g.
>> cmed = nlfilter(image, [3,3], ‘median(x(:))’);
33 sliding box
Median function
SCCS 476

52. MATLAB Command: colfilt
Columnwise neighborhood operation
Syntax: colfilt(im, [m n], block_type, func)
im : input image
[m n] : size that image matrix to be arranged into
block_type : how to rearrange im matrix.
func : function to be applied to the neighborhood
E.g.
>> cmed = colfilt(image, [3,3],
SCCS 476

53. Block_type in colfilt ‘distinct’: ‘sliding’
“Rearranges each distinct m-by-n block in the image A into a column of B. im2col pads A with 0's, if necessary, so its size is an integer multiple of m-by-n. If A = [A11 A12; A21 A22], where each Aij is m-by-n, then B = [A11(:) A12(:) A21(:) A22(:)].”
‘sliding’
“Converts each sliding m-by-n block of A into a column of B, with no zero padding. B has m*n rows and contains as many columns as there are m-by-n neighborhoods of A.
SCCS 476

54. Rearrangement by ‘distinct’
Rearrange to column vector
B = im2col(A,[m n],’distinct’); a1 a2 a3 a4 a5 a6 a7 a8 a9 a1
Input Image a2 a3 a4 a5 a6 a7 a8 a9
Pad this area with 0
mn
Block A B
SCCS 476

55. Rearrangement by ‘sliding’
B = im2col(A,[m n],’sliding’); a0
Input Image a1 a2 an
an+1
amn
mn
Block A B
SCCS 476

56. Rank-Order Filter Sort the intensities within the mask.
Choose the intensity at ith position as output.
Min. filter 10 20 10 15 5 5 10 10 13 13 5 5 20 11 11 20 20 20 15 15 5
Median filter
Sort intensity 10 8 8 10 10 20 15 10
SCCS 476
Max. filter

57. Maximum Filter
Output pixel is the maximum intensity of the pixels within the mask. (find brightest point)
BEFORE
AFTER
Image corrupted by pepper noise
SCCS 476

58. Minimum Filter
Output pixel is the minimum intensity of the pixels within the mask. (find darkest point)
BEFORE
AFTER
Image corrupted by salt noise
SCCS 476

59. Median Filter
Output pixel is the mid-intensity of the pixels within the mask (the median intensity).
SCCS 476
CVIPbook_PPLec/Chapter%209b.ppt

60. Effect of Median Filter
BEFORE
AFTER
3x3 Kernel
SCCS 476

61. Rank-Order Filtering: MATLAB
Command: ordfilt2
Syntax: ordfilt2(image, order, domain);
image : input image
order : which order of the sorted intensity (minimum to maximum value) taken as output
domain : matrix indicating the neighborhood.
1 : pixels in the neighbor.
0 : pixels not in the neighbor
E.g. cmin = ordfilt2(image, 1, ones(3,3));
SCCS 476

62. Geometric Mean Filter x(i, j): pixel’s intensity M : filter mask
|M| : size of the mask
SCCS 476

63. Effect of Geometric Mean Filter
BEFORE
AFTER
Image corrupted by Gaussian noise with variance = 300, mean = 0
SCCS 476

64. Bad Effect of Geometric Mean Filter
BEFORE
AFTER
Image corrupted by pepper noise with probability = 0.4
SCCS 476

65. Alpha-Trimmed Mean Filter
Output is the mean of the data after removing the first d/2 and the last d/2 ordered data.
d =2 10 20 10 15 5
Trim the data by 2.
(1 from the top.
1 from the bottom.) 5 10 13 5 5 8 20 11 20 20 15 5 10
Sort intensity
Output = average intensity of the remaining data.
= 9.5 11 10 8 10 13 10 20 15 10 15 20
SCCS 476

66. Effect of Alpha-Trimmed Mean Filter
BEFORE
AFTER
Image corrupted by salt-and-pepper noise with variance = 200, mean = 0
Trim size = 2, mask size =1
SCCS 476

67. Region of Interest Processing
Process image only in the predefined area.
The predefined area is called “Region Of Interest” (ROI).
Function is the same as before but applied to only ROI.
SCCS 476

68. ROI Selection: MATLAB Command: roipoly Syntax: roipoly(im);
roipoly(im, [x0 x1 …xm], [y0 y1 …ym];
im : input image
(x0, y0), (x1, y1), …, (xm, ym) : coordinate of the polygon covering region of interest.
E.g. roi = roipoly(im, [ ], [ ]);
SCCS 476

69. Mask for ROI Binary image with the white (1) indicating ROI.
Mask coordinate: (222, 21) (272, 21) (300, 75)
(270, 121) (221, 121) (191, 75)
SCCS 476

70. ROI Filtering: MATLAB Command: roifilt2
Syntax: roifilt2(filter, image, roi);
filter : 2D linear filter
image : input image
roi : region of interest (1: ROI, 0: not ROI)
SCCS 476

71. Effect of ROI Filtering
ROI at this coin
Unsharp masking in ROI.
SCCS 476