1. MATHEMATICS OF BINARY MORPHOLOGY APPLICATIONS IN Vision
and
APPLICATIONS IN Vision

3. In general; what is “Morphology”?
The science of form and structure
the science of form, that of the outer form, inner structure, and development of living organisms and their parts
about changing/counting regions/shapes
Among other applications it is used to pre- or post-process images
via filtering, thinning and pruning
Smooth region edges
create line drawing of face
Force shapes onto region edges
curve into a square
Count regions (granules)
number of black regions
Estimate size of regions
area calculations

4. What is Morphology in computer vision ?
Morphology generally concerned with shape and properties of objects.
Used for segmentation and feature extraction.
Segmentation = used for cleaning binary objects.
Two basic operations
erosion (opening)
dilation (closing)

5. Morphological operations and algebras
Different definitions in the textbooks
Different implementations in the image processing programs.
The original definition, based on set theory, is made by J. Serra in 1982.
Defined for binary images - binary operations (boolean, set-theoretical)
Can be used on grayscale images - multiple-valued logic operations

6. Morphological operations on a PC
Various but slightly different implementations in
Scion
Paint Shop Pro
Adope Photoshop
Corel Photopaint mm
Try them, it is a lot of fun

7. Mathematical Morphology - Set-theoretic representation for binary shapes

8. Binary Morphology
Morphological operators are used to prepare binary (thresholded) images for object segmentation/recognition
Binary images often suffer from noise (specifically salt-and-pepper noise)
Binary regions also suffer from noise (isolated black pixels in a white region). Can also have cracks, picket fence occlusions, etc.
Dilation and erosion are two binary morphological operations that can assist with these problems.

9. Goals of morphological operations:
1. Simplifies image data
2. Preserves essential shape characteristics
3. Eliminates noise
4. Permits the underlying shape to be identified and optimally reconstructed from their distorted, noisy forms

10. What is the mathematical morphology ?
1. An approach for processing digital image based on its shape
2. A mathematical tool for investigating geometric structure in image
The language of morphology is set theory.
Mathematical morphology is extension to set theory.

11. Importance of Shape in Processing and Analysis
Shape is a prime carrier of information in machine vision
For instance, the following directly correlate with shape:
identification of objects
object features
assembly defects

12. Binary Morphology

13. Shape Operators Set difference Set intersection
Shapes are usually combined by means of :
Set Union (overlapping objects):
Set Difference based on Set intersection (occluded objects):
Set difference
Set intersection

14. Morphological Operations based on combining base operations
The primary morphological operations are dilation and erosion
More complicated morphological operators can be designed by means of combining erosions and dilations
We will use combinations of union, complement, intersection, erosion, dilation, translation...
Let us illustrate them and explain how to combine

15. Libraries of Structuring Elements
Application specific structuring elements created by the user

16. Notation B X x -2 -1 0 1 2 -2 -1 0 1 2 y A special set : B y
A special set :
the structuring element
Origin at center in this case, but not necessarily centered nor symmetric X
No necessarily compact
nor filled
3*3 structuring element, see next slide how it works

17. Dilation

18. Explanation of Dilation
Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X. X
difference
dilation B

19. Notation for Dilation
Dilation : x = (x1,x2) such that if we center B on them, then the so translated B intersects X.
How to formulate this definition ?
1) Literal translation
Mathematical definition of dilation
2) Better : from Minkowski’s sum of sets
Another Mathematical definition of dilation uses the concept of Minkowski’s sum
B is ingeneral not the same as B

20. The Concept
of Minkowski Sum

21. Minkowski’s Sum Definition of Minkowski’s sum of sets S and B : l l

22. Another View at Dilation

23. Comparison of Dilation and Minkowski sum
Bx =
x and b are points
Minkowski sum

24. It is like dilation but we are not going around , we go only to top and to right

25. Dilation and Minkowski Set
Dilation and Minkowski Set are denoted by + or by 
No unified notation

26. Dilation is not the Minkowski’s sum

27. Dilation is not the Minkowski’s sumb b b b l
Dilation l
B is not the same as B

28. Dilation with other structuring elements

29. Dilation with other structuring elements

30. Dilation vs SE Erosion shrinks Dilation expands binary regions
Can be used to fill in gaps or cracks in binary images
structuring Element ( SE )
If the point at the origin of the structuring element is set in the underlying image, then all the points that are set in the SE are also set in the image
Basically, its like OR’ing the SE into the image

31. Dilation fills holes Fills in holes. Smoothes object boundaries.
Adds an extra outer ring of pixels onto object boundary, ie, object becomes slightly larger.

32. Main Applications of Dilation

33. Dilation example

34. Possible problems with Morphological Operators
Erosion and dilation clean image but leave objects either smaller or larger than their original size.
Opening and closing perform same functions as erosion and dilation but object size remains the same.

35. More Erode and Dilate Examples
Input Image
Dilated
Eroded
Made in Paint Shop Pro

36. Dilation explained pixed by pixel
Denotes origin of B i.e. its (0,0)
Denotes origin of A i.e. its (0,0) B • • • A

37. Dilation explained by shape of A
Shape of A repeated without shift B • • •
Shape of A repeated with shift A

38. Properties of Dilation
objects are light (white in binary)
Dilation does the following:
1. fills in valleys between spiky regions
2. increases geometrical area of object
3. sets background pixels adjacent to object's contour to object's value
4. smoothes small negative grey level regions

39. Structuring Element for Dilation
Length 6
Length 5

40. Structuring Element for Dilation

41. Structuring Element for Dilation
Single point in Image replaced with this in the Result

42. Structuring Element for Dilation

43. Definition of Dilation: Mathematically
Dilation is the operation that combines two sets using vector addition of set elements.
Let A and B are subsets in 2-D space. A: image undergoing analysis, B: Structuring element, denotes dilation

44. Dilation versus translation
Let A be a Subset of and
The translation of A by x is defined as:
The dilation of A by B can be computed as the union of translation of A by the elements of B
x is a vector

45. Dilation versus translation, illustrated
Element (0,0)
Shift vector (0,1)
Shift vector (0,0) • • • B •

46. Dilation using Union Formula
Center of the circle
This circle will create one point
This circle will create no point

47. Example of Dilation with various sizes of structuring elements
Pablo Picasso, Pass with the Cape, 1960

48. Mathematical Properties of Dilation
Commutative
Associative
Extensivity
Dilation is increasing
Illustrated in next slide

49. Illustration of Extensitivity of Dilation• • B •
Replaced with
Here 0 does not belong to B and A is not included in A B

50. More Properties of Dilation
Translation Invariance
Linearity
Containment
Decomposition of structuring element

51. Dilation The dilation operator takes two pieces of data as input
A binary image, which is to be dilated
A structuring element (or kernel), which determines the behavior of the morphological operation
Suppose that X is the set of Euclidean coordinates of the input image, and K is the set of coordinates of the structuring element
Let Kx denote the translation of K so that its origin is at x.
The DILATION of X by K is simply the set of all points x such that the intersection of Kx with X is non-empty

52. Dilation
Example: Suppose that the structuring element is a 3x3 square with the origin at its center 1
{ (-1,-1), (0,-1), (1,-1),
(-1,0), (0,0), (1,0),
( 1,1), (0,1), (1,1) }
K =
X =

53. Dilation
Example: Suppose that the structuring element is a 3x3 square with the origin at its center
Note: Foreground pixels are represented by a color and background pixels are empty

54. Dilation
Structuring element
Input
output

55. Dilation
output

56. Dilation
output

57. Dilation

58. Dilation

59. Erosion

60. Example of Erosion
Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.
difference
erosion

61. Minkowski’s substraction
Notation for Erosion
Erosion : x = (x1,x2) such that if we center B on them, then the so translated B is contained in X.
How to formulate this definition ?
1) Literal translation
Erosion
2) Better : from Minkowski’s substraction of sets
Minkowski’s substraction

62. Notation for Erosion Minkowski’s substraction of sets Erosion
BINARY MORPHOLOGY
Minkowski’s substraction of sets
Erosion

63. Erosion with other structuring elements

64. Erosion with other structuring elements
Did not belong to X  
When the new SE is included in old SE then a larger area is created

65. Common structuring elements shapes
= origin x y
disk
circle
segments 1 pixel wide
Note that here :
points

66. Problem in BINARY MORPHOLOGY
using Minkowski Sum
First we calculate the operation in parentheses to obtain a diamond

67. PROBLEM BINARY MORPHOLOGY
next we calculate the external operation to obtain a hexagon

68. ANOTHER EXAMPLE OF EROSION
Where d is a diameter
Problem :
<d/2
d/2 d

69. Implementation of dilation: very low computational cost
1 (or >0)
Logical or

70. Implementation of erosion: very low computational cost1
Logical and

71. More on Erosion

72. Erosion (Minkowski subtraction)
The contraction of a binary region (aka, region shrinking)
Use a structuring element on binary image data to produce a new binary image
Structuring elements (SE) are simply patterns that are matched in the image
It is useful to explain operation of erosion and dilation in different ways.

73. Typical Uses of Erosion
Removes isolated noisy pixels.
Smoothes object boundary.
Removes the outer layer of object pixels, ie, object becomes slightly smaller.

74. Properties of Erosion:
Erosion removes spiky edges
objects are light (white in binary)
decreases geometrical area of object
sets contour pixels of object to background value
smoothes small positive grey level regions

75. Erosion Example

76. Erosion explained pixel by pixel• • • B

77. Structuring Element in Erosion Example

78. How It Works?
During erosion, a pixel is turned on at the image pixel under the structuring element origin only when the pixels of the structuring element match the pixels in the image
Both ON and OFF pixels should match.
This example erodes regions horizontally from the right.

79. Structuring Element in Erosion Example

80. Structuring Element in Erosion Example

81. Structuring Element in Erosion Example

82. Structuring Element in Erosion Example

83. Structuring Element in Erosion Example

84. Structuring Element in Erosion Example

85. Mathematical Definition of Erosion
Erosion is the morphological dual to dilation.
It combines two sets using the vector subtraction of set elements.
Let denotes the erosion of A by B

86. Erosion in terms of other operations:
Erosion can also be defined in terms of translation
In terms of intersection
Observe that vector here is negative

87. Reminder - this was A •

88. Erosion illustrated in terms of intersection and negative translation
Observe negative translation
Because of negative shift the origin is here • • • •

89. Erosion formula and intuitive example
Center of B is here and adds a point
Center here will not add a point to the Result

90. Example of Erosions with various sizes of structuring elements
Pablo Picasso, Pass with the Cape, 1960

91. Properties of Erosion
Erosion is not commutative!
Extensivity
Erosion is dereasing
Chain rule
The chain rule is as sharp operator in Cube Calculus used in logic synthesis. There are more similarities of these algebras

92. Properties of Erosion Translation Invariance Linearity Containment
Decomposition of structuring element

93. Duality Relationship between erosion and dilation
Dilation and Erosion transformation bear a marked similarity, in that what one does to image foreground and the other does for the image background.
, the reflection of B, is defined as
Erosion and Dilation Duality Theorem
Observe negative value which is mirror image reflection of B
Similar but not identical to De Morgan rule in Boolean Algebra

94. Erosion as Dual of Dilation
Erosion is the dual of dilation
i.e. eroding foreground pixels is equivalent to dilating the background pixels.

95. Duality Relationship between erosion and dilation
Easily visualized on binary image
Template created with known origin
Template stepped over entire image
similar to correlation
Dilation
if origin == 1 -> template unioned
resultant image is large than original
Erosion
only if whole template matches image
origin = 1, result is smaller than original
Another look at duality

96. One more view at Erosion with examples
To compute the erosion of a binary input image by the structuring element
For each foreground pixel superimpose the structuring element
If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is
Otherwise, if any of the corresponding pixels in the image are background, however, the input pixel is set to background value

97. Erosion

98. Erosion example with dilation and negation
We want to calculate this
We dilate with negation

99. Erosion .. And we negate the result
We obtain the same thing as from definition

100. Morphological Operations in terms of more general neighborhoods
This exists in Matlab

101. Erode and Dilate in terms of more general neighborhoods
Yet another loook at Duality Relationship between erosion and dilation

102. Edge detection or Binary Contour

103. Boundary Extraction

104. Erode and Binary Contour in Matlab
Erosion can be used to find contour
Dilation can be also used for it - think how?

105. Edge detection This subtraction is set theoretical Dilate - original
Now you need to invert the image
There are more methods for edge detection

106. Opening & Closing
Opening and Closing are two important operators from mathematical morphology
They are both derived from the fundamental operations of erosion and dilation
They are normally applied to binary images

107. Open and Close Close = Dilate next Erode Open = Erode next Dilate
Original image
eroded
dilated
dilated
eroded
Open
Close

108. OPENING

109. OPENING OPENING Opening : also Supresses : small islands
ithsmus (narrow unions)
narrow caps
difference

110. Opening with other structuring elements

111. Open An erosion followed by a dilation It serves to eliminate noise
Does not significantly change an object’s size

112. Comparison of Opening and Erosion
Opening is defined as an erosion followed by a dilation using the same structuring element
The basic effect of an opening is similar to erosion
Tends to remove some of the foreground pixels from the edges of regions of foreground pixels
Less destructive than erosion
The exact operation is determined by a structuring element.

113. Opening Example Original
What combination of erosion and dilation gives:
cleaned binary image
object is the same size as in original
Original

114. Opening Example Cont Original Erode Dilate Erode original image.
Dilate eroded image.
Smooths object boundaries, eliminates noise (isolated pixels) and maintains object size.
Original
Erode
Dilate

115. One more example of Opening
Erosion can be used to eliminate small clumps of undesirable foreground pixels, e.g. “salt noise”
However, it affects all regions of foreground pixels indiscriminately
Opening gets around this by performing both an erosion and a dilation on the image

116. CLOSING

117. EXAMPLE OF CLOSING Closing : also Supresses : small lakes (holes)
channels (narrow separations)
narrow bays

118. Closing previous image with other structuring elements
BINARY MORPHOLOGY
With bigger rectangle like this
With smaller cross like this

119. Application : shape smoothing and noise filtering
Papilary lines recognition

120. segmentation of microstructures (Matlab Help)
Application :
segmentation of microstructures (Matlab Help)
opening
and with threshold
disk radius 6
closing
threshold
original
negated

121. PROPERTIES IN BINARY MORPHOLOGY
all of them are increasing :
opening and closing are idempotent :

122. EXTENSIVE VERSUS ANTI-EXTENSIVE OPERATIONS
dilation and closing are extensive operations
erosion and opening are anti-extensive operations:

123. DUALITIES OF MORPHOLOGICAL OPERATORS
duality of erosion-dilation, opening-closing,...

124. Decomposition of structuring elements
operations with big structuring elements can be done
by a succession of operations with small s.e’s

125. HIT-OR-MISS Hit-or-miss : Bi-phase structuring element “Hit” part
(white)
“Miss” part
(black)

126. HIT or MISS FOR ISOLATED POINTS
Looks for pixel configurations :
background
foreground
doesn’t matter

127. ISOLATED POINTS
isolated points at 4 connectivity

128. More examples on Closing

129. Close Dilation followed by erosion
Serves to close up cracks in objects and holes due to pepper noise
Does not significantly change object size

130. More examples of Closing
What combination of erosion and dilation gives:
cleaned binary image
object is the same size as in original
Original

131. More examples of Closing cont
Dilate original image.
Erode dilated image.
Smooths object boundaries, eliminates noise (holes) and maintains object size.
Original
Dilate
Erode

132. Closing as dual to Opening
Closing, like its dual operator opening, is derived from the fundamental operations of erosion and dilation.
Normally applied to binary images
Tends to enlarge the boundaries of foreground regions
Less destructive of the original boundary shape
The exact operation is determined by a structuring element.

133. Closing is opening in revers
Closing is opening performed in reverse.
It is defined simply as a dilation followed by an erosion using the same

134. One more example of Closing

135. Mathematical Definitions of Opening and Closing
Opening and closing are iteratively applied dilation and erosion
Opening
Closing

136. Relation of Opening and Closing
Difference is only in corners

137. Opening and Closing are idempotent
Their reapplication has not further effects to the previously transformed result

138. Properties of Opening and Closing
Translation invariance
Antiextensivity of opening
Extensivity of closing
Duality

139. Example of Openings with various sizes of structuring elements
Pablo Picasso, Pass with the Cape, 1960

140. Example of Closings with various sizes of structuring elements

141. Thinning
and Thickening

142. Thinning and Thickening
Thickenning :
Depending on the structuring elements (actually, series
of them), very different results can be achieved :
Prunning
Skeletons
Zone of influence
Convex hull
...

143. Prunning at 4 connectivity
Prunning at 4 connectivity : remove end points by a sequence of thinnings
This point is removed with dark green neighbors
1 iteration =

144. IDEMPOTENCE shown as a result of thinning
1st iteration
2nd iteration
3rd iteration: idempotence

145. Other thinning operations
background
doesn’t matter
foreground

146. USING EROSION TO FIND CONTOURS
Contours of binary regions :
erosion
difference
Contour found with larger mask

147. CONTOURS with different connectivity patterns
Important for perimeter computation.

148. Use of thickening: Convex hull
ii. Convex hull : union of thickenings, each up to idempotence
Original shaper
Thickening with first mask
Union of four thickenings

149. Example of using convex hull

150. iii. Skeleton : Maximal disk : disk centered at x, Dx, such that
Dx  X and no other Dy contains it .
Skeleton : union of centers of maximal disks.

151. PROBLEMS with skeletons
Instability : infinitessimal variations in the border of X
cause large deviations of the skeleton
not necessarily connex even though X connex
good approximations provided by thinning with
special series of structuring elements

152. Example of iterative thinning with 8 masks
1st
iteration

153. Example of iterative thinning with 8 masks
result of
1st iteration
2nd iteration reaches
idempotence

154. Thinning with thickening 20 iterations of thinning color white
Some sort of region clustering

155. Skeletonization for OCR
BINARY MORPHOLOGY
Application : skeletonization for OCR by graph matching
skeletonization
vectorization

156. skeletonization
Application : skeletonization for OCR by graph matching
and 3 rotations
Hit-or-Miss

157. Calculation of Geodesic zones of influence (GZI)
X set of n connex components {Xi}, i=1..n .
The zone of influence of Xi , Z(Xi) , is the set of points closer to some point of Xi than to a point of any other component.
Also, Voronoi partition.
Dual to skeleton.

158. Calculating and using Geodesic Zones of Influence
thr
erosion
7x7
and
GZI
opening
5x5

159. Calculating and using Geodesic Zones of Influence (cont)
thr
erosion
7x7
dist

160. Skeleton by Maximal Balls

161. Example – Morphological Processing of Handwritten Digits
thresholding
thinning
smoothing
opening

162. OF MORPHOLOGICAL OPERATIONS
PROGRAMMING
OF MORPHOLOGICAL OPERATIONS

163. USING LISP

164. USING LISP

165. USING LISP

166. USING LISP

167. USING LISP

168. Morphological Filtering

169. Morphological Filtering
Main idea of Morphological Filtering:
Examine the geometrical structure of an image by matching it with small patterns called structuring elements at various locations
By varying the size and shape of the matching patterns, we can extract useful information about the shape of the different parts of the image and their interrelations.
Combine set-theoretical and morphological operations:

170. Example 1: Morphological filtering
Noisy image will break down OCR systems
Noisy image
Clean original image

171. Morphological filtering (MF)
By applying MF, we increase the OCR accuracy!
Restored image

172. Rank Filter Median
Input
1 operation
2 operations

173. Postprocessing Opening followed by closing.
Removes noise and smoothes boundaries.

174. Postprocessing Opening followed by closing.
Removes noise and smoothes boundaries.

176. Grey Level Morphology

184. erosion
dilation

187. Change of histogram as a result of dilation

195. Removal of Border Objects
Marker is the border itself

196. Summary on Morphological Approaches
Mathematical morphology is an approach for processing digital image based on its shape
The language of morphology is set theory
The basic morphological operations are erosion and dilation
Morphological filtering can be developed to extract useful shape information
Methods can be extended to more values and more dimensions
Nice mathematics can be formulated - non-linear

197. Conclusion Segmentation separates an image into regions.
Use of histograms for brightness based segmentation.
Peak corresponds to object.
Height of peak corresponds to size of object.
If global image histogram is multimodal, local image region histogram may be bimodal.
Local thresholds can give better segmentation.

198. Conclusion Postprocessing uses morphological operators.
Same as convolution only use Boolean operators instead of multiply and add.
Erosion clears noise, makes smaller.
Dilation fills in holes, makes larger.
Postprocessing
Opening and closing to clean binary images.
Repeated erosion with special rule produces skeleton.

199. Problems 1 - 6
1. Write LISP or C++ program for dilation of binary images
2. Modify it to do erosions (few types)
3. Modify it to perform shift and exor operation and shift and min operation
4. Generalize to multi-valued algebra
5. Create a comprehensive theory of multi-valued morphological algebra and its algorithms (publishable).
6. Write a program for inspection of Printed Circuit Boards using morphological algebra.

200. Problem 7.
Electric Outlet Extraction has been done using a combination of Canny Edge Detection and Hough Transforms
Write a LISP program that will use only basic morphological methods for this application.

201. Image Processing for electric outlet, how?
Currently there are many, many ways to approach this problem
Segmentation
Edge Detection
DPC compression
FFT
IFFT
DFT
Thinning
Growing
Haar Transform
Hex Rotate
Alpha filtering
DPC compression
Perimeter
Fractal
Gaussian Filter
Band Pass Filter
Homomorphic Filtering
Contrast
Sharper
Least Square Restoration
Warping
Dilation

202. Create morphological equivalents of other image processing methods.
Image Processing, how?
Create morphological equivalents of other image processing methods.
New, publishable, use outlet problem as example to illustrate

203. Problem 8. Openings and Closings as examples.
The solution here is to follow up one operation with the other.
An opening is defined as an erosion operation followed by dilation using the same structuring element.
Similarly, a closing is dilation followed by erosion.
Define and implement other combined operations.

204. Problems
9. Generalize binary morphological algebra from 2 dimensional to 3 dimensional images. What are the applications.
10. Write software for 9.
11. Generalize your generalized multi-valued morphological algebra to 3 or more dimensions, theoretically, find properties and theorems like those from this lecture.
12. Write software for 11.

205. Problem 13
Mathematical morphology uses the concept of structuring elements to analyze image features.
A structuring element is a set of pixels in some arrangement that can extract shape information from an image.
Typical structuring elements include rectangles, lines, and circles.
Think about other structuring elements and their applications.

207. Morphological Operations: Matlab
BWMORPH Perform morphological operations on binary image.
BW2 = BWMORPH(BW1,OPERATION) applies a specific morphological operation to the binary image BW1.
BW2 = BWMORPH(BW1,OPERATION,N) applies the operation N times. N can be Inf, in which case the operation is repeated until the image no longer changes.
OPERATION is a string that can have one of these values:
'close' Perform binary closure (dilation followed by erosion)
'dilate' Perform dilation using the structuring elementones(3)
'erode' Perform erosion using the structuring elementones(3)
'fill' Fill isolated interior pixels (0's surrounded by1's)
'open' Perform binary opening (erosion followed bydilation)
'skel' With N = Inf, remove pixels on the boundariesof objects without allowing objects to break apart
demos/demo9morph/

208. Sources Shreekanth Mandayam D.A. Forsyth, University of New Mexico,
Qigong Zheng, Language and Media Processing Lab
Center for Automation Research
University of Maryland College Park
October 31, 2000
John Miller
Matt Roach
J. W. V. Miller and K. D. Whitehead
The University of Michigan-Dearborn
Spencer Lustor
Light Works Inc. C. Rössl, L. Kobbelt, H.-P. Seidel, Max-Planck Institute for, Computer Science, Saarbrücken, Germany
LBA-PC4
Howard Schultz
Shreekanth Mandayam
ECE Department
Rowan University
D.A. Forsyth, University of New Mexico

209. More recent Sources
Howie Choset
G.D. Hager,
Z. Dodds,
Dinesh Mocha