1. What is Image Segmentation?
There are many definitions. Three common ones are
Image Segmentation is the process of isolating objects of interest from the rest of the scene. (Castleman )
Image segmentation is the process of partitioning (分割) an image into non-intersecting region such that each region is homogeneous (相似的) and the union of no two adjacent regions is homogeneous. (Pal )

2. What is Image Segmentation?
Image segmentation is to divide an image into parts that have a strong correlation with objects or areas of the real world contained in the image. (Watt )
In brief, segmentation is to subdivide an image into its constituent (组成的) regions or objects.
Segmentation should stop when the objects of interest in an application have been isolated.

3. Image Processing Flow based on Image Segmentation
Feature
Extraction
Classification
Input
Image
Object
Image
Feature
Vector
Object
Type

4. Why is it difficult?
In general autonomous (自主的) segmentation is one of the most difficult tasks in image processing. It is difficult because of many reasons. Here are some typical obstacles:
Non-uniform illumination
No control of the environment
Inadequate model of the object of interest
Noise
etc.

5. Principal approaches
Segmentation methods can be divided into three groups according to the dominant features they employ.
Segmentation based on global knowledge about an image;
The knowledge is usually represented by a histogram of image features.
Edge-based segmentations;
Utilizing edge detection processes to find a closed boundary so that an inside and an outside can be defined.
Region-based segmentations;
This techniques proceed by dividing the image into regions that exhibit similar properties.

6. 2 basis properties of intensity values
Segmentation algorithms generally are based on one of 2 basis properties of intensity values
Discontinuity
to partition an image based on abrupt (突然的) changes in intensity (such as edges)
Similarity
to partition an image into regions that are similar according to a set of predefined criteria.

7. Detection of Discontinuities
detect the three basic types of gray level discontinuities
points , lines , edges
the common way is to run a mask through the image

8. Contents Thresholding Point Detection Line Detection
Edge-based Segmentation
Region-based Segmentation

9. 7.1 Thresholding
Thresholding is a labeling operation on a gray scale image that distinguishes pixels of a higher intensity from pixels with a lower intensity value.
The output of thresholding usually is a binary image.
This technique is particularly useful for scenes which contain solid objects on a uniform contrasting background.

10. Classification of Thresholding
Thresholding can be viewed as an operation that involves tests against a function T of the form:
where p(x,y) denotes some local property of this point.

11. Classification of Thresholding
When T depends on
only f(x,y) : only on gray-level values
􀀾  Global thresholding
both f(x,y) and p(x,y) : on gray-level values and its neighbors
 Local thresholding
x and y (in addition)
 Dynamic thresholding

12. Basic Global Thresholding
See “basic_global_thre.m”
Original image
Histogram
Solution: use T midway between the max and min gray levels

13. Basic Global Thresholding
Let light objects in dark background
To extract the objects:
Select a “T” that separates the objects from the background
i.e. any (x,y) for which f(x,y)>T is an object point.
A thresholded image:

14. Heuristic Global Thresholding
Select an initial estimate for T.
Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values < T and G2 consisting of pixels with gray level values  T
Compute the average gray level values μ1 and μ2 for the pixels in regions G1 and G2
Compute a new threshold value: T=0.5(μ1+μ2)
Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter T see”h….m”

15. Basic Adaptive Thresholding
subdivide original image into small areas.
utilize a different threshold to segment each subimages.
since the threshold used for each pixel depends on the location of the pixel in terms of the subimages, this type of thresholding is adaptive.
See “Adaptive_threshold.m”

16. Multilevel Thresholding
a point (x,y) belongs to
an object class, if T1 < f(x,y) ≤ T2
another object class, if f(x,y) > T2
to background if f(x,y) ≤ T1

17. The histogram of an image containing two principal brightness regions can be considered an estimate of the brightness probability density function p(z).
p(z) is the sum (or mixture) of two unimodal (单峰的) densities (one for light, one for dark regions).

18. Optimal Thresholding
If the form of the densities is known or assumed, in terms of minimum error, determining an optimal threshold for segmenting the image is possible.

19. Optimal Thresholding
Probability of erroneously

20. Optimal Thresholding Minimum error
Differentiating ( 微分) E(T) with respect to T (using Leibniz’s rule) and equating the result to 0
find T which makes

21. Optimal Thresholding Minimum error
Specially, if P1 = P2 then the optimum threshold is where the curve p1(z) and p2(z) intersect

22. Optimal Thresholding For example,
Let PDF=Gaussian density : p1(z) and p2(z)
where μ1 and σ12 are the mean and variance of the Gaussian density of one object
μ2 and σ22 are the mean and variance of the Gaussian density of the other object

23. Optimal Thresholding
Quadratic equation (二次方程)

24. Problems of Thresholding
Original image
Thresholded image

25. Problems of Thresholding
Exact threshold segmentation
Threshold too low
Threshold too high

26. 7.2 Point Detection
a point has been detected at the location on which the mark is centered if
|R|≥T
where T is a nonnegative threshold
R is the sum of products of the coefficients with the gray levels contained in the region encompassed by the mark

27. 7.2 Point Detection
Note that the mark is the same as the mask of Laplacian Operation (in chapter 3)
The only differences that are considered interest are those large enough (as determined by T) to be considered isolated points.

28. Example

29. 7.3 Line Detection
Horizontal mask will result with max response when a line passed through the middle row of the mask with a constant background.
the similar idea is used with other masks.
Note: the preferred direction of each mask is weighted with a larger coefficient (i.e.,2) than other possible directions.

30. Idea 1 of Line Detection Apply every masks on the image
let R1, R2, R3, R4 denotes the response of the horizontal, +45 degree, vertical and -45 degree masks, respectively.
if, at a certain point in the image
|Ri|>|Rj|,
for all j≠i, that point is said to be more likely associated with a line in the direction of mask i.

31. Idea 2 of Line Detection
Alternatively, if we are interested in detecting all lines in an image in the direction defined by a given mask, we simply run the mask through the image and threshold the absolute value of the result.
After thresholding, the points that are left are the strongest responses, which, for lines one pixel thick, correspond closest to the direction defined by the mask.

32. Example

33. 7.4 Edge-based Segmentation
Edge-based segmentations rely on edges found in an image by edge detecting operators
these edges mark image locations of discontinuities in gray level
Edge detection is the most common approach for detecting meaningful discontinuities in gray level
There are a large group of methods based on information about edges in the image

34. What is edge? Edge is where change occurs
Change is measured by derivative in 1D
Biggest change, derivative has maximum magnitude
Or 2nd derivative is zero.
we discuss approaches for implementing
first-order derivative (Gradient operator)
second-order derivative (Laplacian operator)
we have introduced both derivatives in chapter 3
Here, we will talk only about their properties for edge detection.

35. What is edge? In other words, an edge is a set of connected pixels
that lie on the boundary between two regions with relatively distinct gray-level properties.
Note: edge vs. boundary
an edge is a “local” concept.
whereas a region boundary, owing to the way it is defined, is a more global idea.

36. Ideal and Ramp (斜坡) Edges
because of optics, sampling, image acquisition imperfection

37. Thick and Thin Edge
The slope of the ramp is inversely proportional to the degree of blurring in the edge.
Namely, we no longer have a thin (one pixel thick) path.
Instead, an edge point now is any point contained in the ramp, and an edge would then be a set of such points that are connected.
The thickness is determined by the length of the ramp.
The length is determined by the slope, which is in turn determined by the degree of blurring.
Blurred edges tend to be thick and sharp edges tend to be thin.

38. First and Second derivatives (导数)
Gray-level profile
First derivate
the signs of the derivatives would be reversed for an edge that transitions from light to dark
Second derivate

39. Second derivatives an undesirable feature zero-crossing property
produces 2 values for every edge in an image
zero-crossing property
an imaginary straight line joining the extreme positive and negative values of the second derivative would cross zero near the midpoint of the edge.
quite useful for locating the centers of thick edges.

40. Basic idea of edge detection
A profile is defined perpendicularly to the edge direction and the results are interpreted.
The magnitude of the first derivative is used to detect an edge (if a point is on a ramp)
The sign of the second derivative can determine whether an edge pixel is on the dark or light side of an edge.

41. Review of First Derivate
Gradient Operator:
simplest approximation, 22
Roberts cross-gradientoperators, 22
Sobel operators, 33

42. Edge direction and strength
Let α(x,y) represents the direction angle of the vector f at (x,y)
α(x,y)=tan-1(Gy/Gx)
The direction of an edge at (x,y) perpendicular (垂直) to the direction of the gradient vector at that point
The edge strength is given by the gradient magnitude

43. Gradient Masks

44. Diagonal edges with Prewitt and Sobel masks

45. Review of Second Derivate
Laplacian Operator
Laplacian
Mask

46. Example of edge detection
See Matlab Help--demo--toolbox--image processing--analysis…--Edge detection
Note: The Laplacian is seldom used in practice, because:
unacceptably sensitive to noise (as second-order derivative)
produces double edges
unable to detect edge direction

47. Canny edge detector The most powerful edge-detection method
It differs from the other edge-detection methods in that
it uses two different thresholds (to detect strong and weak edges),
and includes the weak edges in the output only if they are connected to strong edges.
This method is therefore less likely than the others to be "fooled" by noise, and more likely to detect true weak edges.

48. Laplacian of Gaussian
Laplacian combined with smoothing to find edges via zero-crossing.
 determines the degree of blurring that occurs.

49. Laplacian of Gaussian (Mexican hat)
The coefficient must sum to zero

50. Edge Detection and Segmentation
Image resulting from edge detection cannot be used as a segmentation result.
Supplementary processing steps must follow to combine edges into edge chains that correspond better with borders (boundary) in the image.

51. 7.5 Region-based Segmentation
Goal: find regions that are “homogeneous” by some criterion
Segmentation is a process that partitions R into n subregions R1, R2, …, Rn such that:
P(Ri) is a logical predicate property defined over the points in set Ri
For example, P(Ri) = TRUE if all pixel in Ri have the same gray level

52. Two methods of Region Segmentation
Region Growing
Region Splitting
Region growing is the opposite of the split and merge approach.

53. Region Growing
The objective of segmentation is to partition an image into regions
A region is a connected component with some uniformity (say gray-levels or texture)
In region growing, we start with a set of “seed” points, growing by appending to each seed’s neighbor pixels, if they have similar properties, such as specific ranges of gray level, and ‘8-connected neighbor’
Need: initialization, similarity criterion

54. Steps of Region Growing
Start by choosing an arbitrary seed pixel and compare it with neighbor pixels.
When seed and neighbor pixels are similar and 8-connected neighbor, region is grown from the seed pixel by adding neighbor pixels.
When the growth of one region stops, choose another seed pixel which does not yet belong to any region and start again.

55. Region growing
An initial set of small areas are iteratively merged according to similarity constraints.

56. Example: defective welds (焊缝) detecting
Seed points
X ray of weld (the horizontal dark region) containing several cracks and porosities (孔, the bright, white streaks running horizontally through the image)
We need initial seed points to grow into regions
On looking at the histogram and image, cracks are bright
Select all pixels having value of 255 as seeds

57. Result of region growing and boundaries of defects
Criterion:
There is a valley at around 190 in the histogram. A pixel should have a value>190 to be considered as a part of region to the seed point.
The pixel should be a 8-connected neighbor to at least one pixel in that region
Result of region growing and boundaries of defects

58. Region Splitting The opposite approach to region merging.
A top-down approach and starts with the assumption that the entire image is homogeneous.
If this is not true, the image is split into four sub images
This splitting procedure is repeated recursively(递归的) until the image is split into homogeneous regions.
Need: homogeneity criterion, split rule

59. Region Splitting Disadvantage
they create regions that may be adjacent and homogeneous, but not merged.

60. Region Splitting and Merging
Procedure:
Split image into 4 disjointed quadrants, for any region Ri, P(Ri)=FALSE
Merge any adjacent regions Rj and Rk, for which P(Rj∪Rk)=TRUE
Stop when no further splitting or merging is possible.

61. Region Splitting and Merging
Quadtree
(四叉树)

62. Result of Splitting and Merging
Example
Original image
Thresholded image
Result of Splitting and Merging
P(Ri) = TRUE if at least 80% of the pixels in Ri have the property |zj-mi|≤2σi
where zj is the gray level of the jth pixel in Ri
mi is the mean gray level of that region
σi is the standard deviation of the gray levels in Ri