1. Chap 10 Image Segmentation
Preview
Subdivide an image into its constituent regions or objects
Process stop when the objects of interest have been isolated
Determine the eventual success or failure of computerized analysis procedure
Based on one of the two properties of intensity values :discontinuity (edge-based) and similarity (region-based)
gray level discontinuities: point , line and edges
assemble edges into region boundary (edge linking)
region-oriented segmentation: watershed segmentation
Region-based, edge based and boundary based

2. 10.1 Detection of discontinuities
For 3x 3 mask, the response of the mask at any point in the image is given by
Point detection of isolated points if | R|T then a point has been detected
def.of isolated point: a point whose gray level is significantly different from its background
Line detection
move mask around an image
preferred direction of each mask is weighted with a large coefficient than other possible direction
use the mask associated with one direction and thresholds its output
Edge detection
the most common approach for detecting meaningful discontinuities in gray level
Basic formulation

3. Chapter 10
Image Segmentation

4. Chapter 10
Image Segmentation

5. Chapter 10
Image Segmentation

6. Chapter 10
Image Segmentation

7. The difference between an edge (local ) and boundary (global)
an ideal edge : a set of connected pixels
step edge, ramp edge, roof edge
edge blurring problem cause by: optics, sampling, and other image acquisition
the magnitude of the first derivative can be used to detect the presence of an edge at a point in an image
the sign of the second derivative can be used to determine whether an edge pixel lies on the dark or light side of an image
two additional properties: (1) double edges (2) zero-crossing properties
Gradient operators
First derivative operator based on approximation of 2-D gradient operator
The gradient of an image f(x,y) at location (x,y) is defined as
The magnitude of this vector denoted
The direction of this is perpendicular to the direction of an edge direction

8. Robert cross-gradient operator Prewitt operators Sobel operators
First order derivative
Prewitt operators
Sobel operators
Use a weight of 2 in the center coefficient to achieve smoothing by giving more important to the center point
Have superior noise suppression to Prewitt operator
Harder to implement than Prewitt operator
Approximate the gradient operator by absolute vales
The approximation will not be isotropic (invariant to rotation)
The Laplacian
Is a second-order derivative defined as
For a 3x3 region
A digital approximation including the diagonal neighbor is given by

9. Chapter 10
Image Segmentation

10. Chapter 10
Image Segmentation

11. Chapter 10
Image Segmentation

12. Chapter 10
Image Segmentation

13. Chapter 10
Image Segmentation

14. Chapter 10
Image Segmentation

15. Chapter 10
Image Segmentation

16. Chapter 10
Image Segmentation

17. Is not used in its original form for edge detection
Is unacceptably sensitive to noise
Produces double edges, complicate segmentation
Is unable to detect edge direction
Consists of : (a) use its zero crossing for edge location
(b) use it for the complementary purpose of establishing whether a pixel is on the dark or light region of an edge
The Laplacian is combined with smoothing as precursor to finding edge via zero crossing
Consider the function (Laplacian Gaussian)
This approximation is not unique
Capture the essential shape of
Smooth the image, and provide an image with zero crossing
Sometime is called the Mexican hat functions

18. 10.3.3 Basic global thresholding
access to illumination source is available—a sol to compensate for nonuniformity
project the illumination pattern onto a constant, white reflective surface (compensate for non-uniformity) , g(x,y)=ki (x,y)
H(x,y)=f(x,y)/g(x,y)
Basic global thresholding
the simplest of thresholding-partition the image histogram by using a global threshold
segmentation is accomplished by
scanning the image pixel
label each pixel depending on whether the gray level of that pixel is greater or less than T

19. Chapter 10
Image Segmentation

20. select an initial estimate for T segment the image using T
threshold by using a heuristic approach, based on the visual inspection of the histogram
select an initial estimate for T
segment the image using T
compute the average gray level values
compute a new threshold
repeat steps 2 and 4 until the difference in T is smaller than a predefined T (convergence)
In general, a good initial value for T is the average gray level of the image
When objects are small compared to the area occupied by the background

21. Chapter 10
Image Segmentation

22. 10.3.4 Basic adaptive thresholding
uneven illumination causes a histogram that cannot be partitioned by a global threshold
sol: (1) divide the original image into sub-images; (2) utilize a different threshold to segment each sub-image.
How to divide an image? and How to estimate the threshold
threshold depends on the location of the pixel in terms of sub-images.
Ex: 10.12
do not contain a boundary between object and background: variances < 75
contains a boundary : variances > 100 ;
using the threshold discussed in the previous section
the initial T was selected at the point midway between the minimum and the maximum

23. Chapter 10
Image Segmentation

24. Chapter 10
Image Segmentation

25. 10.3.5 Optimal global and adaptive thresholding
A method for estimating thresholds that produce the minimum average segmentation error
histogram may be considered as probability density function (PDF), p(z)
the overall density function is the sum or mixture of two densities
If the form of the densities is known, it is possible to determine an optimal threshold
the mixture probability density function is p(z)=P1p1(z)+ P2p2(z)
an image is segmented by classifying as background all pixels with gray level is greater than a threshold T
Objective: select the value of T that minimize the average error;

26. Continue
Determine a given pixel which belongs to an object or to the background
the extreme case: background points are known never to occur (P2=0)
differentiate T with respect to T (using Leibneiz rule) to find minimum error; and the optimal threshold

27. Estimating these densities is not feasible
sol: employ densities whose parameters are reasonably simple to obtain
One of the principal densities is Gaussian density
two threshold values may be required to obtain the optimal solution
if the variances are equal, a single threshold is sufficient ( )
If P 1 =P2, the optimal threshold is the average of the means
the optimal threshold may be accomplished for other densities if the Raleigh and log-normal densities
mean square error may be used to estimate a composite gray-level PDF of an image from the image histogram

28. The histograms have peaks of approximately the same height
Use of boundary characteristics for histogram improvement and local thresholding
“ good” threshold are enhanced considerably if the histogram peaks are tall, narrow, and separated by deep valleys
Improve the shape of histogram by considering only those pixels that lie on or near the edges
If histogram of the pixel on or near the edge between objects and background were used—improve the symmetry of the histogram peak
The histograms have peaks of approximately the same height
Equal probability: improve the symmetry of the histogram peak
Using measures based on gradient and Laplacian to deep the valley between histogram peak
Three level image based on gradient
For a black object on a light background

29. Chapter 10
Image Segmentation

30. Chapter 10
Image Segmentation

31. Chapter 10
Image Segmentation

32. Chapter 10
Image Segmentation

33. Chapter 10
Image Segmentation

34. Chapter 10
Image Segmentation

35. Chapter 10
Image Segmentation

36. 10.3.7 Threshold based on several variables
Multi-spectral thresholding
3-D histogram
Find clusters of points in 3-D space
Shortcoming
Cluster seeking becomes an increasingly complex

37. Chapter 10
Image Segmentation

38. For Ex: land-use satellite imagery depends on color
10.4 Region-based segmentation
Basic formulation
segmentation must be complete;every pixel must be in a region
points in a region must be connected
the region must be disjointed
the pixels in a segmented region (P(Ri)=TRUE) if all pixel in Ri have the same gray level
P(Ri∪ Rj)= False --Ri an Rj is different
Region growing
a procedure that groups pixel or sub-region into larger regions based on predefined start with a set of “seed” criteria-similar to the seed (specific gray-level or color)
points and from these grow regions by appending each seed
priori information is not available--compute the same set of properties
Criteria—gray-level, texture, color
the selection of similarity
For Ex: land-use satellite imagery depends on color

39. seed points and seed regions (Fig. 10.40)
when the images are monochrome, carried out with set of descriptor based on gray levels and spatial properties (such as moments or textures)
The stopping rule
when on more pixels satisfy the criteria
additional criteria
utilize size, likeness between a candidate pixel and the pixels grown so far, and the shape of the region being grown (history of the growth)
seed points and seed regions (Fig )
If connectivity or adjacency information is not used, descriptor alone can yield misleading result
Region splitting and merging
Splitting
subdivide the entire region successively into smaller and small quadrant regions if P(R )=FALSE
quad tree -- the root of the tree corresponds to the entire image and each node corresponds to a subdivision

40. can be solved by merging
only splitting causes the final partition would contain adjacent regions with identical properties
can be solved by merging
Merging
Is limited to groups of four blocks
the splitting and merging steps
split into four disjoint quadrants
merge adjacent adjacent regions
stop when no further merging or splitting
Variations of the scheme
Split the image into a set of blocks
The advantage: use the same quad-tree for splitting and merging
Texture segmentation

41. Chapter 10
Image Segmentation

42. Chapter 10
Image Segmentation

43. Chapter 10
Image Segmentation

44. Chapter 10
Image Segmentation

45. 10.5 Segmentation by morphological watersheds
No further merging
The procedure is terminated by one final merging of regions satisfying step2
The merged regions may be of different size
Texture segmentation is based on the measures of texture
10.5 Segmentation by morphological watersheds
Advantage (speed in the case of global thresholding) and disadvantages of traditional segmentation (the need to post-processing)
The approach often produce stable segmentation results (continuous segmentation boundaries)
Basic concepts
Based on visualizing an image in three dimension:two spatial coordinates versus gray levels
“Topographic” interpretation (three types of points):

46. (a). Points belonging to a regional minimum
(b). Points at which of a drop of water (catchment basin or watershed)
©. Points at which water would be equally likely to fall to more than one such minimum (crest line on the topographic surface; divide line or watershed lines)
The principal objective is to find the watershed lines
Suppose that a hole is punched in each regional minimum and the entire topography is flooded
Water rise through the holes
Rising water in distinct catchment basin is about to merge
A dam is built to prevent the merging
Reach when the tops of the dam are visible
Dam construction
Based on binary images
The simplest way
Construct dams separating sets of binary points based on morphological operations

47. Dilated by the structuring element, subject to two conditions:
Use dilation
Def of catchment basin and the sets of points in two regional minimum at stage n-1 : Cn-1(M2) and Cn-1(M2)
The two components extracted from q by performing the simple AND operation q  C[n-1]:becomes one connected component
Dilated by the structuring element, subject to two conditions:
The dilation has to be constrained to q
The dilation cannot be performed on points that would cause the sets being dilated to merge

48. Chapter 10
Image Segmentation

49. Chapter 10
Image Segmentation

50. Chapter 10
Image Segmentation

51. Chapter 10
Image Segmentation

52. 10.5.4 The use of Makers Watershed generally cause over segmentation
A large number of segmented regions
Sol: limit the number of allowable regions by incorporating a preprocessing stage
based on a marker: internal maker ( associated with object) and external marker(associated with background)
steps for selection of marker: (1) preprocessing; and (2) definition of a set of criteria that markers must satisfy
Def. of an internal marker: (1) a region that is surrounded by points of higher altitude (2) the points in the region form a connected component; (3) all points in the connected component have the same gray-level value

53. preprocessing scheme
smoothing
apply watershed algorithm
external markers correspond to watershed line
partition each of these regions into two: a single object and its background
10.6 The use of motion in segmentation
motion can extract objects of interest from a background of irrelevant detail
Arise form a relative displacement between the sensing system and the scene viewed
Spatial technique
Difference image and reference image (containing only stationary components)

54. Chapter 10
Image Segmentation

55. Chapter 10
Image Segmentation

56. 1-values caused by noise
A difference image between two images taken at time t iand tj
all pixels in dij with value 1 are considered as the result of object motion
Is applicable only if the two images are registered spatially and if the illumination is relatively constant within the bounds established by T
1-values caused by noise
Sol (removal of noises): form 4- or 8-connected regions of 1’s in dij(x,y) and ignore any region that has less than a predefined number of entries
Accumulative difference
Isolated entries problem
Sol(1) consider changes at a pixel location over frames (memory)
Sol(2) ignore changes that occur sporadically over a frame sequence (random noise)

57. Consider a sequence of image frames
ADI (accumulative difference image)
Consider a sequence of image frames
compare the difference image with every subsequent image in the sequence
a count for each location in the accumulative image is incremented
the entry in a given pixel of the accumulative image gives the number of times the gray level at that position was different from the corresponding pixel value
Assume the gray levels of the moving objects are larger than the background: three accumulative difference images: absolute, positive, and negative—are the same size as the image in the sequence
ADI values are counts, and start out with all zero values(10-6.2~4)
Ex. 19:
The nonzero area of the positive ADI is equal to the size of the moving object
The location of the positive ADI corresponds to the location of the moving object in the reference fame

58. The number of counts in the positive ADI stop increasing when the moving object is displaced completely with respect to the same object in the reference frame
The absolute ADI contains positive and negative ADI
The direction and speed of the moving object can be determined from the entries in the absolute and negative ADIs
Establishing a reference image
noise problem:
cancel all stationary components, leaving only image elements that correspond to noise and to the moving objects
can be handled by the filtering approach or by forming an accumulative difference image
With only stationary elements is not always possible
build a reference from a set of images containing one or more objects becomes necessary
Used to describe busy scenes

59. Consider the first image to be the reference image
How to generate a reference frame
Consider the first image to be the reference image
When a non-stationary component has been moved out in the reference frame, duplicate the corresponding background into the location occupied by the objects.
When all moving objects has moved completely

60. Chapter 10
Image Segmentation

61. Chapter 10
Image Segmentation

62. Chapter 10
Image Segmentation

63. Chapter 10
Image Segmentation

64. Chapter 10
Image Segmentation

65. Chapter 10
Image Segmentation