1. Computer and Robot Vision I
Chapter 2
Binary Machine Vision:
Thresholding and Segmentation
Presented by: 傅楸善 & 翁丞世
指導教授: 傅楸善 博士
Digital Camera and Computer Vision Laboratory
Department of Computer Science and Information Engineering
National Taiwan University, Taipei, Taiwan, R.O.C.

2. 2.1 Introduction
To detect the objects, the input image is often simplified.
binary value 1: considered part of object
binary value 0: background pixel
binary machine vision: generation and analysis of binary image
DC & CV Lab.
CSIE NTU

3. 2.2 Thresholding 𝐵 𝑟,𝑐 =1, 𝑖𝑓 𝐼 𝑟,𝑐 ≥𝑇 𝐵 𝑟,𝑐 =0, 𝑖𝑓 𝐼 𝑟,𝑐 <𝑇
r : row, c: column
I: grayscale intensity, B: binary intensity
T: intensity threshold
DC & CV Lab.
CSIE NTU

4. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

5. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

6. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

7. 2.2 Thresholding
The question of thresholding is a question of the automatic determination of the threshold value.
DC & CV Lab.
CSIE NTU

8. 2.2 Thresholding histogram ℎ 𝑚 =#{(𝑟,𝑐)|𝐼 𝑟,𝑐 =𝑚}
m spans each gray level value e.g
#: operator counts the number of elements in a set
DC & CV Lab.
CSIE NTU

9. 2.2 Thresholding 2 3 1 2-bits source image histogram of left image2 3 1
2-bits source image
histogram of left image
DC & CV Lab.
CSIE NTU

10. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

11. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

12. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

13. 2.2 Thresholding Bayonet Neill Concelman Connector (BNC)
The standard connector used to connect IEEE Base2 coaxial cable to an MAU(媒體附著單元 (Media attachment unit))
DC & CV Lab.
CSIE NTU

14. 2.2 Thresholding
DC & CV Lab.
CSIE NTU

15. 2.2 Thresholding Minimizes a criterion function
Minimize the within-group variance (Otsu’s method)
Minimize the Kullback information distance
DC & CV Lab.
CSIE NTU

16. 2.2.1 Minimizing Within-Group Variance
𝑃 1 , …, 𝑃 𝐼 : histogram probabilities of gray values 1 … I
𝑃 𝐼 = # 𝑟,𝑐 𝐼𝑛𝑡𝑒𝑛𝑠𝑖𝑡𝑦 𝑟,𝑐 =𝐼 𝑅×𝐶
𝑅×𝐶: the spatial domain of the image 𝑅: the number of row of the image
𝐶: the number of column of the image
DC & CV Lab.
CSIE NTU

17. 2.2.1 Minimizing Within-Group Variance
Observations
A group is a set of pixels with intensity homogeneity.
Homogeneity is measured by the use of variance
High homogeneity group has low variance
Low homogeneity group has high variance
Objective
Select a dividing score such that the weighted sum of the within-group variances is minimized.
DC & CV Lab.
CSIE NTU

18. 2.2.1 Minimizing Within-Group Variance
Given threshold t, divide histogram into two parts.
Part 1 is the left hand side of the histogram.
Part 2 is the right hand side of the histogram.
DC & CV Lab.
CSIE NTU

19. 2.2.1 Minimizing Within-Group Variance
Within-group variance 𝜎 𝑤 2 : weighted sum of group variances
𝜎 𝑊 2 = 𝑞 1 𝑡 𝜎 1 2 𝑡 + 𝑞 2 𝑡 𝜎 2 2 𝑡
Find 𝑡 ∗ which minimizes 𝜎 𝑊 2 𝑡
𝑡 ∗ =𝑎𝑟𝑔𝑚𝑖𝑛 𝜎 𝑊 2 (𝑡)
DC & CV Lab.
CSIE NTU

20. 2.2.1 Minimizing Within-Group Variance
Implementation
For i from 0 to 255:
Compute 𝑞 1 𝑡 , 𝑞 2 𝑡 , 𝜎 1 2 𝑡 , 𝜎 2 2 𝑡
Compute 𝜎 𝑤 2 𝑡 = 𝑞 1 𝑡 ∗ 𝜎 1 2 𝑡 + 𝑞 2 𝑡 ∗ 𝜎 2 2 𝑡
If 𝜎 𝑤 2 𝑡 ∗ > 𝜎 𝑤 2 𝑡 : 𝑡 ∗ =𝑡
DC & CV Lab.
CSIE NTU

21. 2.2.1 Minimizing Within-Group Variance
Speed up
𝜎 2 = 𝑖=0 𝑡 𝑖−𝜇 2 𝑃 𝑖 + 𝑖=𝑡+1 𝐼 𝑖−𝜇 2 𝑃 𝑖
= 𝑖=0 𝑡 𝑖− 𝜇 1 𝑡 + 𝜇 1 𝑡 −𝜇 2 𝑃 𝑖 + 𝑖=𝑡+1 𝐼 𝑖− 𝜇 2 𝑡 + 𝜇 2 𝑡 −𝜇 2 𝑃 𝑖
= 𝑖=0 𝑡 [ 𝑖− 𝜇 1 𝑡 𝑖− 𝜇 1 𝑡 ∗ 𝜇 1 𝑡 −𝜇 + 𝜇 1 𝑡 −𝜇 2 ]𝑃 𝑖 + 𝑖=𝑡+1 𝐼 [ 𝑖− 𝜇 2 𝑡 𝑖− 𝜇 2 𝑡 ∗ 𝜇 2 𝑡 −𝜇 + 𝜇 2 𝑡 −𝜇 2 ]𝑃 𝑖
Σ 𝑖− 𝜇 1 𝑡 ∗𝑃 𝑖 =0
Σ 𝑖− 𝜇 2 𝑡 ∗𝑃 𝑖 =0
DC & CV Lab.
CSIE NTU

22. 2.2.1 Minimizing Within-Group Variance
Speed up
𝜎 2 = 𝑖=0 𝑡 [ 𝑖− 𝜇 1 𝑡 𝜇 1 𝑡 −𝜇 2 ]𝑃 𝑖 + 𝑖=𝑡+1 𝐼 [ 𝑖− 𝜇 2 𝑡 𝜇 2 𝑡 −𝜇 2 ]𝑃 𝑖 𝑖=𝑡+1 𝐼 [ 𝑖− 𝜇 2 𝑡 𝜇 2 𝑡 −𝜇 2 ]𝑃 𝑖
= 𝑖=0 𝑡 𝑖− 𝜇 1 𝑡 2 𝑃 𝑖 + 𝜇 1 𝑡 −𝜇 2 𝑞 1 + 𝑖=𝑡+1 𝐼 𝑖− 𝜇 2 𝑡 2 𝑃 𝑖 + 𝜇 2 𝑡 −𝜇 2 𝑞 2 𝑖=𝑡+1 𝐼 𝑖− 𝜇 2 𝑡 2 𝑃 𝑖 + 𝜇 2 𝑡 −𝜇 2 𝑞 2
= 𝜎 1 2 ∗ 𝑞 1 + 𝜎 2 2 ∗ 𝑞 𝜇 1 𝑡 −𝜇 2 𝑞 𝜇 2 𝑡 −𝜇 2 𝑞 2
= 𝜎 𝑤 2 + 𝑞 1 𝑡 1− 𝑞 1 𝑡 [ 𝜇 1 𝑡 − 𝜇 2 𝑡 ]
𝜇= 𝑞 1 (𝑡)∗ 𝜇 1 (𝑡)+ 𝑞 2 (𝑡)∗ 𝜇 2 (𝑡)
𝑞 2 𝑡 =1− 𝑞 1 (𝑡)
DC & CV Lab.
CSIE NTU

23. 2.2.1 Minimizing Within-Group Variance
Speed up
𝜎 2 = 𝜎 𝑤 2 + 𝑞 1 𝑡 1− 𝑞 1 𝑡 [ 𝜇 1 𝑡 − 𝜇 2 𝑡 ]
𝑡 ∗ =𝑎𝑟𝑔𝑚𝑎𝑥 𝑞 1 (𝑡)[1− 𝑞 1 𝑡 ][ 𝜇 1 𝑡 − 𝜇 2 𝑡 ]
Fortunately, we can calculate 𝑞 1 𝑡 , 𝜇 1 𝑡 , 𝜇 2 𝑡 recursively.
constant
minimize
maximize
DC & CV Lab.
CSIE NTU

24. 2.2.1 Minimizing Within-Group Variance
Speed up
Fortunately, we can calculate 𝑞 1 𝑡 , 𝜇 1 𝑡 , 𝜇 2 𝑡 recursively.
𝑞 1 𝑡+1 = 𝑞 1 𝑡 +𝑃 𝑡+1
𝜇 1 𝑡+1 = 𝜇 1 (𝑡)∗ 𝑞 1 𝑡 + 𝑡+1 ∗𝑃 𝑡+1 𝑞 1 (𝑡+1)
𝜇 2 𝑡+1 = 𝜇− 𝜇 1 𝑡+1 ∗ 𝑞 1 𝑡+1 1− 𝑞 1 (𝑡+1)
DC & CV Lab.
CSIE NTU

25. 2.2.1 Minimizing Within-Group Variance
DC & CV Lab.
CSIE NTU

26. 2.2.2 Minimizing Kullback Information Distance
 Gaussian distribution
DC & CV Lab.
CSIE NTU

27. 2.2.2 Minimizing Kullback Information Distance
Assumption
The observations come from a weighted mixture of two Gaussians distributions.
Gaussian Distribution of Background
Gaussian Distribution of Object
DC & CV Lab.
CSIE NTU

28. 2.2.2 Minimizing Kullback Information Distance
DC & CV Lab.
CSIE NTU

29. 2.2.2 Minimizing Kullback Information Distance
Objective Formulation
Determine a threshold T that results in two Gaussian distributions which minimize Kullback divergence
P(I) : observed histogram distribution
f(I) : a mixture of Gaussian distributions determined by T
DC & CV Lab.
CSIE NTU

30. 2.2.2 Minimizing Kullback Information Distance
constant
minimize
DC & CV Lab.
CSIE NTU

31. 2.2.2 Minimizing Kullback Information Distance
DC & CV Lab.
CSIE NTU

32. 2.2.2 Minimizing Kullback Information Distance
DC & CV Lab.
CSIE NTU

33. 2.2.2 Minimizing Kullback Information Distance
Implementation
For i from 0 to 255:
Compute 𝑞 1 𝑡 , 𝑞 2 𝑡 , 𝜎 1 2 𝑡 , 𝜎 2 2 𝑡
Compute 𝐻(𝑡)
If 𝐻( 𝑡 ∗ )>𝐻(𝑡): 𝑡 ∗ =𝑡
DC & CV Lab.
CSIE NTU

34. 2.2.2 Minimizing Kullback Information Distance
DC & CV Lab.
CSIE NTU

35. 2.2.2 Minimizing Kullback Information Distance
Left dark line:
Within-Group Variance
(Otsu)
Right dark line:
Kullback Information
(Kittler-Illingworth)
DC & CV Lab.
CSIE NTU

36. 2.3 Connected Components Labeling
Description
Connected Components labeling is a grouping operation.
It performs the unit change from pixel to region or segment.
All pixels are given the same identifier
Have value binary 1
Connect to each other
Definition
DC & CV Lab.
CSIE NTU

37. 2.3 Connected Components Labeling
Terminology
label: unique name or index of the region
connected components labeling: a grouping operation
pixel property: position, gray level or brightness level
region property: shape, bounding box, position, intensity statistics
Definition
DC & CV Lab.
CSIE NTU

38. 2.3 Connected Components Labeling
Definition of Connected Component
Two pixels 𝑝 and 𝑞 belong to the same connected component 𝐶 if there is a sequence of 1-pixels ( 𝑝 0 , 𝑝 1 ,…, 𝑝 𝑛 ), where
𝑝 0 =𝑝
𝑝 𝑛 =𝑞
𝑝 𝑖−1 , 𝑝 𝑖 :𝑖=1,…,𝑛 are neighbor
Definition
DC & CV Lab.
CSIE NTU

39. 2.3.1 Connected Components Operators
To explain what neighborhood is
DC & CV Lab.
CSIE NTU

40. 2.3.1 Connected Components Operators1 1 2 3 1 2
source binary image
example
Connected components labeling with 4-adjacency
Connected components labeling with 8-adjacency
DC & CV Lab.
CSIE NTU

41. 2.3.2 Connected Components Algorithms
All the algorithms process a row of the image at a time
All the algorithms assign new labels to the first pixel of each component and attempt to propagate the label of a pixel to right or below neighbors
Principle & stop condition
DC & CV Lab.
CSIE NTU

42. 2.3.2 Connected Components Algorithms
What label should be assigned to pixel A?
How to keep track of the equivalence of two or more labels?
How to use the equivalence information to complete processing?
Issue
A應該放什麼標籤
怎麼保持追蹤等價的標籤
怎麼使用等價訊息去完成處理
DC & CV Lab.
CSIE NTU

43. 2.3.2 Connected Components Algorithms
An Iterative Algorithm
The Classical Algorithm
*A Space-Efficient Two-Pass Algorithm with a Local Equivalence Table
*An Efficient Run-Length Implementation of the Local Table Method
DC & CV Lab.
CSIE NTU

44. 2.3.3 An Iterative Algorithm
Implementation
Initialize each pixel with an unique label
Changed = True
While Changed == True
iteration of top-down followed by bottom-up passes
Changed = check is any label changed
DC & CV Lab.
CSIE NTU

45. 2.3.3 An Iterative Algorithm
Why can’t just one iterative
Jump to 35(ori) 36(after top-down)=>12與7的交界處
DC & CV Lab.
CSIE NTU

46. 2.3.4 The Classical Algorithm
performs label propagation as above
when two different labels propagate to the same pixel, the smaller label propagates and equivalence entered into table
resolve equivalence classes by DFS (Depth-First Search)
second pass performs a translation
Two top-down phase
First phase: assign label and establish equ_table
mid-phase: find equivalence classes
Second phase: chx label according to equ_label
DC & CV Lab.
CSIE NTU

47. 2.3.4 The Classical Algorithm
Example
Two top-down phase
First phase: assign label and establish equ_table
mid-phase: find equivalence classes
Second phase: chx label according to equ_label
{2=4}
{3=5}
{1=5}
DC & CV Lab.
CSIE NTU

48. 2.3.4 The Classical Algorithm
Example
Resolve equivalence
{2=4}
{3=5}
{1=5}
{2=4}
{1=3=5}
Two top-down phase
First phase: assign label and establish equ_table
mid-phase: find equivalence classes
Second phase: chx label according to equ_label
DC & CV Lab.
CSIE NTU

49. 2.3.4 The Classical Algorithm
Pros
More efficient, only two passes
Cons
Need more memory to store equivalence table
Two top-down phase
First phase: assign label and establish equ_table
mid-phase: find equivalence classes
Second phase: chx label according to equ_label
DC & CV Lab.
CSIE NTU

50. 2.3.5 A Space-Efficient Two-Pass Algorithm That Uses a Local Equivalence Table
Small table stores only equivalences from current preceding line
Relabel each line with equivalence labels when equivalence detected
Point2. alternation
DC & CV Lab.
CSIE NTU

51. 2.3.5 A Space-Efficient Two-Pass Algorithm That Uses a Local Equivalence Table
DC & CV Lab.
CSIE NTU

52. 2.4 Signature Segmentation and Analysis
Signature analysis perform a unit change from the pixel to the region or segment.
It was firstly used in character recognition.
The signature, which is a projection, is the histogram of the non-zero pixels of the masked image.
projections can be vertical, horizontal, diagonal, circular…
DC & CV Lab.
CSIE NTU

53. DC & CV Lab.
CSIE NTU

54. 2.4 Signature Segmentation and Analysis
General Signatures
Vertical Projection
Horizontal Projection
Diagonal Projection
ℎ 𝑟 =⋕# 𝑥,𝑦 𝑥,𝑦 =1,𝑥=𝑟
𝑣 𝑟 =⋕{ 𝑥,𝑦 | 𝑥,𝑦 =1,𝑦=𝑐}
DC & CV Lab.
CSIE NTU

55. 2.4 Signature Segmentation and Analysis
Steps
Thresholding: generate the binary image.
Projection Computation: compute the vertical, horizontal, or diagonal projections.
Projection Segmentation: divide the image into several segments or regions according to the signatures.
DC & CV Lab.
CSIE NTU

56. 2.4 Signature Segmentation and Analysis
DC & CV Lab.
CSIE NTU

57. 2.4 Signature Segmentation and Analysis
DC & CV Lab.
CSIE NTU

58. 2.4 Signature Segmentation and Analysis
segment the vertical and horizontal projections
treat each rectangular subimage as the image
DC & CV Lab.
CSIE NTU

59. 2.4 Signature Segmentation and Analysis
Magnetic ink character recognition
光學字母辨識
磁性墨水辨識
OCR: Optical Character Recognition
MICR: Magnetic Ink Character Recognition
DC & CV Lab.
CSIE NTU

60. 2.4 Signature Segmentation and Analysis
Diagonal projections:
: from upper left to lower right
: from upper right to lower left
object area: sum of all the projections values in the segment
DC & CV Lab.
CSIE NTU

61. 2.4 Signature Segmentation and Analysis
DC & CV Lab.
CSIE NTU

62. DC & CV Lab.
CSIE NTU

63. 2.5 Summary Create binary image by thresholding.
Find out connected component on binary image.
Generate signature segmentation by projection.
DC & CV Lab.
CSIE NTU

64. Project due Oct. 4 Write a program to generate:
a binary image (threshold at 128)
a histogram
connected components (regions with +
at centroid, bounding box)
DC & CV Lab.
CSIE NTU