0. Chapter 5 – Image Pre-processing
5.1 Brightness Transformations
5.2 Geometric Transformations
5.3 Local Pre-processing
5.4 Image Restoration

1. Objectives of image pre-processing:
(a) Suppress image information
that is not relevant to later
work
(b) Enhancing information that
is useful for later analysis

2. Classes of Image Pre-processing Methods
(a) Brightness Transformations
(b) Geometric transformations
5.1 Brightness Transformations
Categorization:
(1) Point processing, Neighborhood processing
(2) Position invariant, Position variant
(3) Image Enhancement, Image Restoration
5-2

3. ○ Point Processing
Histogram Equalization

4. Transform function
e.g.,

6. Theorem: Let T be a differentiable strictly increasing
or strictly decreasing function.
Let r be a random variable having density
Let having density
Then, or

7. Proof: Let : the distribution functions of r and s
(a) T strictly increasing
(b) T strictly decreasing
5-7

8. Example: Let x: a random variable with uniform
distribution on (0, 1). Find the density
g of
Ans: Let Y : the distribution of y.
Since y is a positive random variable,
for
For ,
The density of y:
5-8

9. Let transform function be Then
Called equalization or linearization.
5-9

10. the transformation function T is
○ Example
Let
Since
the transformation function T is
5-10

11. i.e., is a uniform distribution
Since
From
i.e., is a uniform distribution
5-11

12. Discrete case:
Let , ,
Transformation:
5-12

13. ○ Example: L = 16, n = 360

15. ○ Examples:
5-15

16. 5-16

18. Specified Histogram Equalization
-- Specify the shape of the histogram that we wish
the processed image to have.
Histogram
equalization
Histogram
specification
Input image
5-18

19. 5-19

20. : gray levels of the input image I
Let
: gray levels of the input image I
: gray levels of the output image O
: the probability density function of r
that can be estimated from I
: the given specified probability density
function of z that we wish O to have
Let
and
Then
and
Both
are known
5-20

21. Given: input image (I), specification ( ) 1. Compute from I
Procedure:
Given: input image (I), specification ( )
1. Compute from I
2. Compute from
3. Compute from
4. Compute
5. Transform I into O by
5-21

22. Discrete case:
5-22

23. Example: Given image I of size 64 by 64 with 8 gray levels
Histogram of input image I:
Transformation function:
5-23

24. Transformation function:
Specified histogram:
Transformation function:
5-24

25. Inverse transformation function:
Output image O:
5-25

26. Histogram of output image O:
5-26

27. Input histogram Equalized histogram Output histogram
Specified histogram
5-27

28. 5.2. Geometric Transformations
Distorted grid image
Scene grid
Recovered grid image
A geometric transform is a vector function T defined by
Two steps: i) Pixel coordinate transformation
ii) Brightness interpolation
Applications: Remotely sensed image registration
Bird-view generation
Document skew
5-28

29. Remotely Sensed Image Registration
5-29

30. Bird’s-Eye View Image Generation
Blind areas around a vehicle
Window pillars
Height of vehicle
Summary of blind areas
Driver’s position
5-30

31. System Configuration Fish-eye camera with wide-angle lens Image Scene
5-31 31

32. 32
5-32

33. Experiments 33
5-33

34. Automatic Parking System
Four major tasks: (i) bird-view image generation
(ii) Parking space detection, (iii) path planning,
and (iv) automatic parking 34
5-34

35. Automatic parking 35
5-35

36. 5.2.1. Pixel Coordinate Transformations
Geometric distortion types :
a. variable distance, b. panoramic
c. skew, e. scale, f. perspective
Transformation model:
where
5-36

37. ◎ Image Enlargement Step 1: Zero interleave
5-37

38. (b) Bilinear interpolation
Step 2: Filling
(a) NN interpolation
(b) Bilinear interpolation
(c) Bicubic interpolation
5-38

39. (b) Bilinear interpolation
(a) NN interpolation
(b) Bilinear interpolation
5-39

40. Input
image
5-40

41. Polynomial transformation:
Bilinear transformation:
Affine transformation:
Rotation :
Scale change :
Skewing :
5-41

42. Example: Bilinear transform
Needs at least 4 pairs of corresponding points to
determine the parameters
5-42

43. Solve x by the least square error method.
5-43

44. 1. Detect salient points of images
Image Registration:
Steps:
1. Detect salient points of images
2. Determine the point correspondences between
the two images
3. Compute the parameters of the transformation
functions
5-44

45. 5.2.2. Brightness Interpolation
5-45

46. (a) Nearest-Neighbor Interpolation
(b) Linear Interpolation
5-46

47. 。 Bilinear Interpolation
5-47

48. ◎ Generalization ○ Interpolation function R
○ Examples:
5-48

49. ○ Substituting into
NN-interpolation
5-49

50. ○ Substituting into linear interpolation
5-50

51. ○ Cubic interpolation function
5-51

52. ○ Bi-cubic Interpolation
-- Apply cubic interpolation
first along the rows and
then down the columns
5-52

53. 5.3 Local (Neighborhood) Pre-Processing
-- Applies a function to a neighborhood of each pixel
-- Different functions  different objectives
e.g., noise removal (smoothing), edge detection,
corner detection
5-53

54. Neighborhood (window, mask)
Function + Window = Filter
5-54

55. Filtration
(Filtering)
Convolution:
5-55

56. Objective: noise removal
5.3.1 Image Smoothing
Objective: noise removal
Linear Smoothing Filters
1-D case:
Input data
Mean filter
Smoothed data
2-D case:

58. Gaussian Smoothing
1D:
2D:
Discrete case:

59. 。 Mean Filters (i) Arithmetic mean: (ii) Geometric mean:
(iii) Harmonic mean:
5-59

60. (iv) Contra-harmonic mean:
(v) Alpha-trimmed mean filter
i) Order elements,
ii) Trim off end elements
iii) Take mean
5-60

61. 。 Image Averaging Assume noise n(x,y) is Gaussian, uncorrelated
and has zero mean.
5-61

62. Non-linear Smoothing Filters
: mask elements
。 Maximum filter:
。 Minimum filter:
。 K-nearest neighbors (K-NN) mean filter
5-62

63. 。 Median filter
5-63

64. 5-64

65. 。 Smoothing by a rotating masker

66. Dispersion
5-66

67. 5.3.2 Edge Detectors -- Edges are important information for image
understanding
Origin of edges
Line drawing

68. Roof edge (crease edge) Smooth edge Line
Typical edge profiles:
Step edge (jump edge)
Ramp edge
Roof edge (crease edge)
Smooth edge
Line

69. ○ Derivatives

70. 2D case:
Gradient
Magnitude
Direction

71. 。 Prewitt filters Consider Horizontal filter: , Smooth filter: Combine
Vertical filter: , Smooth filter:

72. Input
Vertical
Horizontal
Edge image Binary image Thinning

73. 。Roberts operator: 。Sobel operator: 。Robinson operator:
。Kirsch operator:

74. 5.3.3 Zero-Crossings of Second Derivatives
Laplacian:
Laplace
operator:
Invariant under rotation (isotropic filter)

75. Step edge:
Ramp edge:

76. Zero crossing
0 + , + 0
0 －, － 0
+ －, － +

77. 。 Other Laplacian masks
。 Second derivatives are sensitive to noise
Example: Edge detection by taking zero crossings
after a Laplace filtering
Marr-Hildreth method
Smooth the input image using a Gaussian before
Laplace filtering

78. 。 Gaussian smooth + Laplace filtering = Laplacian of Gaussian (LOG):
Difference of Gaussian (DOG):
Mexican hat

80. Separable Filters Convolution: e.g., Laplacian filter n × n filter:
2 (n × 1) filters:
5-80

81. 5.4.3 Scale Space Filtering fewer noises, Larger scale 
less precise in location
Larger scale 
more noises,
more precise in location
Smaller scale 

82. 5.3.5 Canny Edge Detector Criteria: a. Low error rate of detection:
no missing and extra edges
b. Localization of edges: precise in edge position
c. Single response: one-pixel width edges
Step 1: Edge detection
(i) Horizontal direction
(ii) Vertical direction
5-82

83. Step 2: Non-maximum suppression For each pixel p, (i) Quantize to
(iii) Edge magnitude
Edge direction
Step 2: Non-maximum suppression
For each pixel p,
(i) Quantize to
0, 45, 90 or 135 degrees
(ii) Along
p is marked if its edge magnitude
is larger than both its two neighbors
p is ignored otherwise
5-83

84. Step 3: Hysteresis thresholding For each marked pixel p,
(i) If > or
(ii) If and p is adjacent to an
edge pixel
p is considered as an edge pixel
Step 4: Repeat steps (1) - (3) for ascending
Step 5: Synthesize edges at multiple scales
5-84

87. 5.3.7 Edges in Multi-Spectral Images
Methods:
Applied to individual components
Applied to combination of component images
(i) difference or (ii) ratio

88. 5.3.8 Pre-processing in the Frequency Domain
Fourier Transform
Spatial Domain
Frequency Domain
5-88

89. 5-89

90. Low pass
filtration
5-90

91. High Pass
Filtration
5-91

92. Band pass
filtration
5-92

93. Gaussian Frequency Filters
5-93

94. Spatial counterparts
Frequency
filters
Spatial
filters
5-94

95. Periodic noise removal
5-95

96. Butterworth frequency filters
5-96

97. Homomorphic filterings
5-97

98. 5.3.9 Line Detection Line Finding Operators
Reinforcement of Linear Structure Using Parameterized Relaxation Labeling J.S. Duncan & T. Birkholzer IEEE PAMI, vol. 14, no. 5, pp , 1992
5-98

99. Edge Reinforcement
(a) (c) (e) (g)
(b) (d) (f) (h)
5-99

100. 2. Edge Reinforcement with Thinning
(a) (c) (e) (g)
(b) (d) (f) (h)

101. 3. Bar Reinforcement

102. Detection of Corners
Basic idea: corners possess large curvatures
: approximates curvature
Harris corner detector
f: image, W: image patch
A corner point will have a high response of
for all
5-102

103. From Taylor
approximation
5-103

104. Both small: no edge and corner One large and one small: ridge
Harris matrix
Let
Both small: no edge and corner
One large and one small: ridge
Both large: corner
5-104

105. 5-105

106. 5-106

107. which is maximal in pixels with high contrast.
Moravec
Detector
which is maximal in pixels with high contrast.
Image function f(i,j) is approximated in the
neighborhood of pixel (i,j)
Zuniga-Haralick Detector
Kitchen-Rosenfeld Detector
5-107

108. 5.3.11. Maximally Stable External Regions
Harris corner detector can be invariant to rotation and
translation but variant to scale change and projective
transformation.
Maximally Stable External Regions (MSER) are
invariant to translation, rotation, similarity and
affine transformations.
To detect (MSER):
Maximal regions: union all connected components
of all frames of a sequence of thresholded I with
frame t corresponding to threshold t.
Minimal regions: obtained by inverting the intensity
of I and running the same process.
5-108

109. 5-109

110. 5-110

111. 5.4 Image Restoration
Objective: reconstruct or recover from degradation
(e.g., moving, distortion).
Idea: modeling the degradation
Degradation Model
Diagonalization of Circulant
and Block-Circulant Matrices
Inverse Filtering
Algebraic Approach to Restoration
Wiener Filter
5-111

112. 5.4.1 Degradation Model Mathematically,○
Mathematically,
Problem: Given g(x,y) and some knowledge
about degradation H and noise n, obtain
an approximation to f(x,y).
5-112

113. Assume Image: Degraded image: If H is linear, i.e.,
If H is homogeneous, i.e.,
5-113

114. If H is position invariant, i.e.,
Let
: PSF
If H is position invariant, i.e.,
In discrete case,
2-114

115. Consider 1D case,
In matrix form,
where
2-115

116. Since
is periodic,
: circulant
matrix
2-116

117. ◎ Diagonalization
Circulant Matrices
Define
2-117

118. e.g., k = 0
2-118

119. 2-119

120. For k = 1
2-120

121. 2-121

122. i.e., formed by the M eigenvectors of of H,
From
: a diagonal matrix
and
where
i.e., formed by the M eigenvectors of of H,
where * denotes conjugate transpose
2-122

123. : the DFT of
2-123

124. Block Circulant Matrices
Define
where
are N by N matrices and
2-124

125. The inverse matrix
Likewise,
where
is the DFT of
2-125

126. ◎ Effects of Diagonalization on the Degradation Model
1-D case:
From
and
2-126

127. : the DFT of : the DFT of f Similarly, : the DFT of g
is the DFT of sequence
2-127

128. Ignore the scale factor MN,
From
Including noise term,
2-D case:
Ignore the scale factor MN,
2-128

129. 5.4.3 Inverse Filtering Low-pass filtering: Constrained division:
5-129

130. 5.4.4 Algebraic Approach to Restoration
A. Unconstrained restoration
B. Constrained restoration
A. Unconstrained restoration
From
Find
s.t.
Let
2-130

131. B. Constrained restoration
where Q is a linear operator on f.
Using the method of Lagrange multipliers,
2-131

132. 5.4.5 Winer Filtering : correlation matrices of f and n
The ij-th element of
is given by
We hope noise-to-signal ratio
to be small.
: real symmetric matrices
For images, pixels within 20 to 30 pixels can
generally be correlated. A typical correlation
matrix has a bound of nonzero elements about
the main diagonal and zeros in the right upper
and left lower corner regions.
2-132

133. can be made to approximate block
circulant matrices and can be diagonalized by
Let
Substitute into
From
2-133

134. From
5-134

135. A and B are diagonal matrices derived from the
Ignore M, N
where
5-135

136. Parametric Wiener filter
Ideal inverse filter
(when no noise )
where k : constant
5-136

137. Different k’s
2-137

138. ○ Applications -- Motion Deblurring
Image f(x,y) undergoes planar motion
: the components of motion
T : the duration of exposure
Fourier transform,
5-138

139. 5-139

140. Suppose uniform linear motion:
Note H vanishes at u = n/a (n: an integer)
Restore image by the inverse or Wiener filter
5-140

141. ○ Atmospheric turbulence
○ Defocusing
○ Atmospheric turbulence
5-141