1. CIS 595 Image ENHANCEMENT SPATIAL DOMAIN
in the
SPATIAL DOMAIN
Dr. Rolf Lakaemper

2. Most of these slides base on the book Digital Image Processing
by Gonzales/Woods

3. Spatial Filtering
Spatial Filtering

4. Operation on the set of ‘neighborhoods’ N(x,y) of each pixel
Spatial Filtering
Spatial Filtering:
Operation on the set of ‘neighborhoods’ N(x,y) of each pixel 6 8 12
200
(Operator: sum) 6 8 2
226 12
200 20 10

5. N(p) = {(x,y): |x-xP|=1, |y-yP| = 1}
Spatial Filtering
Neighborhood of a pixel p at position x,y is a
set N(p) of pixels defined relative to p.
Example 1:
N(p) = {(x,y): |x-xP|=1, |y-yP| = 1} P Q

6. More examples of neighborhoods:
Spatial Filtering
More examples of neighborhoods: P P P P P P

7. The most prominent neighborhoods are the
Spatial Filtering
Usually neighborhoods are used which are close to discs, since properties of the eucledian metric are often useful.
The most prominent neighborhoods are the
4-Neighborhood and the 8-Neighborhood P P

8. We will define spatial filters on the
Spatial Filtering
We will define spatial filters on the
8-Neighborhood and their bigger relevants. P P P N8
N24
N48

9. Spatial Filtering
Index system for N8: n1 n2 n3 n4 n5 n6 n7 n8 n9

10. P = i ni Motivation: what happens to P if we apply the
Spatial Filtering
Motivation: what happens to P if we apply the
following formula:
P = i ni n1 n2 n3 n4
n5=P n6 n7 n8 n9

11. P = i ai ni with ai given by:
Spatial Filtering
What happens to P if we apply this formula:
P = i ai ni
with ai given by:
a1=1
a2=1
a3=1
a4=1
a5=4
a6=1
a7=1
a8=1
a9=1

12. Linear Image Filters Spatial Filtering
Linear operations calculate the resulting value in the output image pixel f(i,j) as a linear combination of brightness in a local neighborhood of the pixel h(i,j) in the input image.
This equation is called discrete convolution:
Function w is called a convolution kernel or a filter mask. In our case it is a rectangle of size (2a+1)x(2b+1).

13. Spatial Filtering

14. Spatial Filtering
Exercise: Compute the 2-D linear convolution of the following two signal X with mask w. Extend the signal X with 0’s where needed.

15. Lets have a look at different values of ai and their effects !
Spatial Filtering
Lets have a look at different values of ai and
their effects !
This MATLAB
program creates an
interesting output:
% Description: given an image 'im', % create 12 filtered versions using % randomly designed filters
for i=1:12
a=rand(7,7); % create a % random 7x % filter-matrix
a=a*2 - 1; % range: -1 to 1
a=a/sum(a(:)); % normalize
im1=conv2(im,a);% Filter
subplot(4,3,i);
imshow(im1/max(im1(:)));
end

16. Spatial Filtering

17. Different effects of the previous slides included:
Spatial Filtering
Different effects of the previous slides included:
Blurring / Smoothing
Sharpening
Edge Detection
All these effects can be achieved using different coefficients.

18. (Sometimes also referred to as averaging or lowpass-filtering)
Spatial Filtering
Blurring / Smoothing
(Sometimes also referred to as averaging or lowpass-filtering)
Average the values of the center pixel and its neighbors:
Purpose:
Reduction of ‘irrelevant’ details
Noise reduction
Reduction of ‘false contours’ (e.g. produced by zooming)

19. Blurring / Smoothing 1 1 1 1 1 1 * 1/9 1 1 1 Apply this scheme
Spatial Filtering
Blurring / Smoothing 1 1 1 1 1 1
* 1/9 1 1 1
Apply this scheme
to every single pixel !

20. Example 2: Weighted average
Spatial Filtering
Example 2:
Weighted average 1 2 1 2 4 2
* 1/16 1 2 1

21. The general formula for weighted average:
Spatial Filtering
Basic idea:
Weigh the center point the highest, decrease weight by distance to center.
The general formula for weighted average:
P = i ai ni / i ai
Constant value, depending on mask, not on image !

22. Blurring using different radii (=size of neighborhood)
Spatial Filtering
Blurring using different
radii (=size of
neighborhood)

23. Spatial Filtering
EDGE DETECTION

24. Spatial Filtering
EDGE DETECTION
Purpose:
Preprocessing
Sharpening

25. What are edges in an image?
Spatial Filtering
What are edges in an image?
Edges are those places in an image that correspond to object boundaries.
Edges are pixels where image brightness changes abruptly.
Brightness vs. Spatial Coordinates

26. Motivation: Derivatives
Spatial Filtering
Motivation: Derivatives

27. First and second order derivative
Spatial Filtering
First and second order derivative

28. First and second order derivative
Spatial Filtering
First and second order derivative
1st order generally produces thicker edges
2nd order shows stronger response to detail
1st order generally response stronger to gray level step
2nd order produce double (pos/neg) response at step change

29. Definition of 1 dimensional discrete 1st order derivative:
Spatial Filtering
Definition of 1 dimensional discrete 1st order derivative:
dF/dX = f(x+1) – f(x)
The 2nd order derivative is the derivative of the 1st order derivative…

30. F(x+1)-F(x) – (F(x)-F(x-1))
Spatial Filtering
The 2nd order derivative is the derivative of the 1st order derivative…
F(x-1)
F(x)
F(x+1)
derive
F(x)-F(x-1)
F(x+1)-F(x)
F(x+2)-F(x+1)
F(x+1)-F(x) – (F(x)-F(x-1))
F(x+1)-F(x) – (F(x)-F(x-1))= F(x+1) -2F(x)+F(x-1)

31. One dimensional 2nd derivative in x direction:
Spatial Filtering
One dimensional 2nd derivative in x direction:
F(x+1)-F(x) – (F(x)-F(x-1))= F(x+1) -2F(x) +F(x-1) 1 -2 1
One dimensional 2nd derivative, direction y: 1 -2 1

32. TWO dimensional 2nd derivative (derivatives are linear !):
Spatial Filtering
TWO dimensional 2nd derivative (derivatives are linear !): 1 1 + = 1 -2 1 -2 1 -4 1 1 1
This mask is called the ‘LAPLACIAN’
(remember calculus ?)

33. Different variants of the Laplacian
Spatial Filtering
Different variants of the Laplacian

34. Effect of the Laplacian… (MATLAB)
Spatial Filtering
Effect of the Laplacian… (MATLAB)
im = im/max(im(:));
% create laplacian
L=[1 1 1;1 -8 1; 1 1 1];
% Filter !
im1=conv2(im,L);
% normalize and show
im1=(im1-min(im1(:))) / (max(im1(:))-min(im1(:)));
imshow(im1);

35. Spatial Filtering

36. Spatial Filtering
A Quick Note
Matlab’s image processing toolbox provides edge function to find edges in an image:
I = imread('rice.tif');
BW1 = edge(I,'prewitt');
BW2 = edge(I,'canny');
imshow(BW1)
figure, imshow(BW2)
Edge function supports six different edge- finding methods: Sobel, Prewitt, Roberts, Laplacian of Gaussian, Zero-cross, and Canny.

37. Edges detected by the Laplacian can be used to sharpen
Spatial Filtering
Edges detected by the Laplacian can be used to sharpen
the image ! +

38. Spatial Filtering

39. Sharpening can be done in 1 pass:
Spatial Filtering
Sharpening can be done in 1 pass: -1 -1 + = -1 4 -1 1 -1 5 -1 -1 -1
LAPLACIAN
Original Image
Sharpened Image

40. Sharpening in General:
Spatial Filtering
Sharpening in General:
Unsharp Masking
and
High Boost Filtering

41. Basic Idea (unsharp masking):
Spatial Filtering
Basic Idea (unsharp masking):
Subtract a BLURRED version of an image from the image itself !
Fsharp = F – Fblurred

42. - = Variation: emphasize original image (high-boost filtering):
Spatial Filtering
Variation: emphasize original image
(high-boost filtering):
Fsharp = a*F – Fblurred , a>=1 -1 1 - = -1
a-1 -1 a 1 1 1 -1 1

43. Different Examples of High-Boost Filters:
Spatial Filtering
Different Examples of High-Boost Filters: -1
a=1 -1 -1 -1 -1
a=6 -1 5 -1 -1 -1
a=1e7+1 -1
1e7 -1 -1
Laplacian + Image !

44. Spatial Filtering

45. Enhancement using the First Derivative:
Spatial Filtering
Enhancement using the First Derivative:
The Gradient
Definition: 2 dim. column vector,
(f) = [Gx ; Gy], G is 1st order derivative
MAGNITUDE of gradient:
Mag((f))= SQRT(Gx2 + Gy2)

46. For computational reasons the magnitude is often approximated by:
Spatial Filtering
For computational reasons the magnitude is often approximated by:
Mag ~ abs(Gx) + abs(Gy)

47. Mag ~ abs(Gx) + abs(Gy) “Sobel Operators”
Spatial Filtering
Mag ~ abs(Gx) + abs(Gy) -1 1 -1 -2 -1 -2 2 -1 1 1 2 1
“Sobel Operators”

48. Sobel Operators are a pair of operators ! Effect of Sobel Filtering:
Spatial Filtering
Sobel Operators are a pair of operators !
Effect of Sobel Filtering:

49. Sobel Operators are a pair of operators !
Spatial Filtering
Sobel Operators are a pair of operators !

50. Remember the result some slides ago: First and second order derivative
Spatial Filtering
Remember the result some slides ago:
First and second order derivative
1st order generally produces thicker edges
2nd order shows stronger response to detail
1st order generally response stronger to gray level step
2nd order produce double (pos/neg) response at step change

51. In practice, multiple filters are combined to enhance images
Spatial Filtering
In practice, multiple filters are combined to enhance images

52. Spatial Filtering
…continued

53. Spatial Filtering

54. Non – Linear Filtering: Order-Statistics Filters
Spatial Filtering
Non – Linear Filtering:
Order-Statistics Filters
Median
Min
Max

55. Spatial Filtering
Median:
The median M of a set of values is such that half the values in the set are less than (or equal to) M, and half are greater (or equal to) M. 1 2 3 3 4 5 6 6 6 7 8 9 9

56. Most important properties of the median:
Spatial Filtering
Most important properties of the median:
less sensible to noise than mean
an element of the original set of values
needs sorting (O(n log(n))

57. Spatial Filtering
Median vs. Mean

58. Spatial Filtering
Min and Max 2 5 7 3 2 5 7 3 3 2 2 3
MIN 3 4 2 3 3 4 8 3 3 4 8 3 3 3 3 3 3 3 3 3
MAX 2 5 7 3 3 8 2 3 3 4 8 3 3 3 3 3

59. Basics of Morphological Filtering
Spatial Filtering
Basics of Morphological Filtering

60. Spatial Filtering
Dilation:

61. Spatial Filtering
Erosion:

62. Spatial Filtering
opening:

63. Spatial Filtering
closing:

64. Opening on binary images:
Spatial Filtering
Opening on binary images:
for i=2:2:18
se = strel('disk',i);
imo = imopen(im1,se);
subplot(3,3,(i)/2);
imshow(imo);
s=sprintf('disksize: %d',i);
title(s);
end

65. Opening on gray images:
Spatial Filtering
Opening on gray images:

66. Closing on binary images:
Spatial Filtering
Closing on binary images:
for i=2:2:18
se = strel('disk',i);
imo = imclose(im1,se);
subplot(3,3,(i)/2);
imshow(imo);
s=sprintf('disksize: %d',i);
title(s);
end

67. Closing on gray images:
Spatial Filtering
Closing on gray images:

68. Closing o Opening & Opening o Closing
Spatial Filtering
Closing o Opening & Opening o Closing

69. Spatial Filtering
Morphological Edges:
Dilation - Original

70. (Exercise: surveillance camera)
Spatial Filtering
(Exercise: surveillance camera)

71. …end of Operations in Spatial Domain.
Spatial Filtering
…end of Operations in Spatial Domain.
And now to something completely different ( really ?)

72. The
FREQUENCY Domain

73. Some of these slides base on the textbook Digital Image Processing
by Gonzales/Woods
Chapter 4

74. We called this the SPATIAL domain.
Frequency Domain
So far we processed the image ‘directly’, i.e. the transformation was a function of the image itself.
We called this the SPATIAL domain.
So what’s the FREQUENCY domain ?

75. Let’s first forget about images, and look at SOUND.
SOUND: 1 dimensional function of changing (air-)pressure in time
Pressure
Time t

76. Sound
SOUND: if the function is periodic, we perceive it as sound with a certain frequency (else it’s noise). The frequency defines the pitch.
Pressure
Time t

77. The AMPLITUDE of the curve defines the VOLUME
Sound
The AMPLITUDE of the curve defines the VOLUME

78. The SHAPE of the curve defines the sound character
String
Flute
Brass

79. Sound
How can the
SHAPE
of the curve be defined ?

80. Sound
Listening to an orchestra, you can distinguish between different instruments, although the sound is a
SINGLE FUNCTION !
Flute
Brass
String

81. Sound
If the sound produced by an orchestra is the sum of different instruments, could it be possible that there are BASIC SOUNDS, that can be combined to produce every single sound ?

82. The answer (Charles Fourier, 1822):
Sound
The answer (Charles Fourier, 1822):
Any function that periodically repeats itself can be expressed as the sum of sines/cosines of different frequencies, each multiplied by a different coefficient

83. (Don’t take this remark seriously, please)
Sound
Or differently:
Since a flute produces a sine-curve like sound, a (huge) group of (outstanding) talented flautists could replace a classical orchestra.
(Don’t take this remark seriously, please)

84. The sine-curve is defined by:
1D Functions
A look at SINE / COSINE
The sine-curve is defined by:
Frequency (the number of oscillations between 0 and 2*PI)
Amplitude (the height)
Phase (the starting angle value)
The constant y-offset, or DC (direct current)

85. The general sine-shaped function:
1D Functions
The general sine-shaped function:
f(t) = A * sin(t + ) + c
Amplitude
Frequency
Phase
Constant offset
(usually set to 0)

86. What happens if we add sine and cosine ?
1D Functions
Remember Fourier:
…A function…can be expressed as the sum of sines/cosines…
What happens if we add sine and cosine ?

87. (with A=sqrt(a^2+b^2) and tan  = b/a)
1D Functions
a * sin(t) + b * cos(t)
= A * sin(t + )
(with A=sqrt(a^2+b^2) and tan  = b/a)
Adding sine and cosine of the same frequency yields just another sine function with different phase and amplitude, but same frequency.
Or: adding a cosine simply shifts the sine function left/right and stretches it in y-direction. It does NOT change the sine-character and frequency of the curve.

88. Any function that periodically repeats itself…
1D Functions
Remember Fourier, part II:
Any function that periodically repeats itself…
=> To change the shape of the function, we must add sine-like functions with different frequencies.

89. This applet shows the result: Applet: Fourier Synthesis
1D Functions
This applet shows the result:
Applet: Fourier Synthesis

90. 1D Functions
What did we do ?
Choose a sine curve having a certain frequency, called the base-frequency
Choose sine curves having an integer multiple frequency of the base-frequency
Shift each single one horizontally using the cosine-factor
Choose the amplitude-ratio of each single frequency
Sum them up

91. FOURIER SYNTHESIS, FOURIER COEFFICIENTS
1D Functions
This technique is called the
FOURIER SYNTHESIS,
the parameters needed are the sine/cosine ratios of each frequency.
The parameters are called the
FOURIER COEFFICIENTS

92. f(x)= a0/2 + k=1..n akcos(kx) + bksin(kx)
1D Functions
As a formula:
f(x)= a0/2 + k=1..n akcos(kx) + bksin(kx)
Fourier Coefficients

93. The set of ak, bk TOTALLY defines the CURVE synthesized !
1D Functions
Note:
The set of ak, bk TOTALLY defines the CURVE synthesized !
We can therefore describe the SHAPE of the curve or the CHARACTER of the sound by the (finite ?) set of FOURIER COEFFICIENTS !

94. 1D Functions
Examples for curves, expressed by the sum of sines/cosines (the FOURIER SERIES):

95. f(x) = ½ - 1/pi * n 1/n *sin (n*pi*x)
1D Functions
SAWTOOTH Function
Freq.
sin
cos 1 2
1/2 3
1/3 4
1/4
f(x) = ½ - 1/pi * n 1/n *sin (n*pi*x)

96. f(x) = 4/pi * n=1,3,5 1/n *sin (n*pi*x)
1D Functions
SQUARE WAVE Function
Freq.
sin
cos 1 3
1/3 5
1/5 7
1/7
f(x) = 4/pi * n=1,3,5 1/n *sin (n*pi*x)

97. 1D Functions
What does the set of FOURIER COEFFICIENTS tell about the character of the shape ?

98. Steep slopes introduce HIGH FREQUENCIES.
1D Functions
Result:
Steep slopes introduce HIGH FREQUENCIES.

99. Motivation for Image Processing:
1D Functions
Motivation for Image Processing:
Steep slopes showed areas of high contrast…
…so it would be nice to be able to get the set of FOURIER COEFFICIENTS if an arbitrary (periodically) function is given.
(So far we talked about 1D functions, not images, this was just a motivation)

100. 1D Functions
The Problem now:
Given an arbitrary but periodically 1D function (e.g. a sound), can you tell the FOURIER COEFFICIENTS to construct it ?

101. The answer (Charles Fourier):
1D Functions
The answer (Charles Fourier):
YES.

102. F(u)=1/Mx=0..M-1 f(x)e-i2ux/M
1D Functions
With:
eix=cos(x)+ i sin(x)
The coefficients F(u) are given by (continous):
F(u)=f(x)e-i2ux dx
Discrete (M=number of samples):
F(u)=1/Mx=0..M-1 f(x)e-i2ux/M

103. Note the similarity between analysis and synthesis:
1D Functions
Note the similarity between analysis and synthesis:
Analysis:
F(u)=1/Mx=0..M-1 f(x)e-i2ux/M dx
(with u=0..M-1)
Synthesis:
f(x)=u=0..M-1 F(u)ei2ux/M
(with x=0..M-1)

104. 1D Functions
We don’t want to explain the mathematics behind the answer here, but simply use the MATLAB Fourier Transformation Function.

105. Input: A vector, representing the discrete function
1D Functions
MATLAB - function fft:
Input: A vector, representing the discrete function
Output: The Fourier Coefficients as vector of imaginary numbers, scaled

106. s=sin(x)+cos(x) + sin(2*x) + 0.3*cos(2*x); f=fft(s);
1D Functions
Example:
x=0:2*pi/(2047):2*pi;
s=sin(x)+cos(x) + sin(2*x) + 0.3*cos(2*x);
f=fft(s);
cos
1.3 i i
-0.4 +
1.6i
sin
Freq. 0
Freq. 1
Freq. 2
Freq. 3

107. Transformation: t(a) = 2*a / length(result vector)
1D Functions
1.3 i i i Fr Re Im
1.3 1
1026.2
1022.8 2
310.1
1022.1 Fr Re Im ~0 1 ~1 2
~0.3
Transformation: t(a) = 2*a / length(result vector)

108. real(F(k+1)) * 2 / L imag(F(k+1)) * 2 / L
1D Functions
The fourier coefficients are given by:
F=fft(function)
L=length(F); %this is always = length(function)
Coefficient for cosine, frequency k-times the base frequency:
real(F(k+1)) * 2 / L
Coefficient for sine, frequency k-times the base frequency:
imag(F(k+1)) * 2 / L
Since: a * sin(t) + b * cos(t) = A * sin(t + )
the Amplitude is given by: A=sqrt(a^2+b^2),
The Phase by: tan  = b/a

109. An application using the Fourier Transform:
1D Functions
An application using the Fourier Transform:
Create an autofocus system for a digital camera
We did this already, but differently !
(MATLAB DEMO)

110. Shape Database using Fourier Features
1D Functions
Second application:
Shape Database using Fourier Features
(revisited)

111. Some Results
Top 9
query

112. Some Results
Top 9
query

113. 2D Functions
From Sound to Images:
2D Fourier Transform

114. Extend the base functions to 2 dimensions:
2D Functions
The idea:
Extend the base functions to 2 dimensions:
fu(x) = sin(ux) 
fu,v(x,y) = sin(ux + vy)

115. The base function, direction x: u=1, v=0
2D Functions
Some examples:
The base function, direction x: u=1, v=0 x y

116. The base function, direction y: u=0, v=1
2D Functions
The base function, direction y: u=0, v=1

117. 2D Functions
u=2, v=0

118. 2D Functions
u=0, v=2

119. 2D Functions
u=1, v=1

120. 2D Functions
u=1, v=2

121. 2D Functions
u=1, v=3

122. 2D Functions
As in 1D, the 2D image can be phase-shifted by adding a weighted cosine function:
fu,v(x,y) = ak sin(ux + vy) + bk cos(ux + vy) + =

123. frequency-multiples (u,v):
2D Functions
As basic functions, we get a pair of sine/cosine functions for every pair of
frequency-multiples (u,v): u
sin
sin
sin
sin
cos
cos
cos
cos v
sin
sin
sin
sin
cos
cos
cos
cos
sin
sin
sin
sin
cos
cos
cos
cos

124. 2D Functions
Every single sin/cos function gets a weight, which is the Fourier Coefficient: u a a a a b b b b v a a a a b b b b a a a a b b b b

125. The set of weights defines the 2D function.
2D Functions
Summing all basic functions with their given weight gives a new function.
As in 1D:
Every 2D function can be synthesized using the basic functions with specific weights.
The set of weights defines the 2D function.

126. Summing basic functions of different frequencies:
2D Functions
Example:
Summing basic functions of different frequencies:

127. Summing basic functions of different frequencies:
2D Functions
Example:
Summing basic functions of different frequencies:

128. MATLAB Demo: Bear Reconstruction
2D Functions
MATLAB Demo: Bear Reconstruction