1. Digital Image Processing
Image Segmenattion
Dr. Ghassabi Tehran shomal University Spring 2015

2. Outline Introduction Digital Image Fundamentals
Intensity Transformations and Spatial Filtering
Filtering in the Frequency Domain
Image Restoration and Reconstruction
Color Image Processing
Wavelets and Multi resolution Processing
Image Compression
Morphological Operation
Image Segmentation
Object representation
Object recognition

3. Image Segmentation
The goal of image segmentation is to cluster pixels into salient image regions, i.e., regions corresponding to individual surfaces, objects, or natural parts of objects.

4. The goals of segmentation
Separate image into coherent “objects”
image
human segmentation
CS 376 Lecture 7 Segmentation

5. 3 2 1 Image segmentation pixel count input image intensity
white pixels
pixel count 3
black pixels
gray pixels 1 2
input image
intensity
These intensities define the three groups.
We could label every pixel in the image according to which of these primary intensities it is.
i.e., segment the image based on the intensity feature.
What if the image isn’t quite so simple?
CS 376 Lecture 7 Segmentation

6. pixel count input image intensity pixel count input image intensity
Now how to determine the three main intensities that define our groups?
Now how to determine the three main intensities that define our groups?
CS 376 Lecture 7 Segmentation

7. Low-level image segmentation
Edge-based
Region-based
Thresholding
Region Growing
Divide and Merge
Hybrid
Watershed
تفاوتهای بین ناحیه ای
شباهتهای درون یک ناحیه مثل بافت و رنگ و میانگین

8. Edge-based Segmentation
Use Filters that find edges.
Prewitt operator
Roberts Cross-Difference Operator
Sobel Edge Detector
Canny Edge Detector
Challenges:
Noise sensitive
None closed edges
Consider all edges

9. Thresholding a point (x,y) belongs to T depends on
Multi-level Thresholding
پیکسلهای شی و پس زمینه دارای مقادیر شدتی اند که به دو دسته غالب تقسیم می شوند. اونوقت یه آستانه کلی عمل جداسازی رو انجام میدهد.
زمانیکه در تصویر دو شی غالب داریم. آنوقت هیستوگرام سه حالت غالب را دارد. پس آستانه گذاری چندگانه داریم.
a point (x,y) belongs to
to an object class if T1 < f(x,y)  T2
to another object class if f(x,y) > T2
to background if f(x,y)  T1
T depends on
only f(x,y) : only on gray-level values  Global threshold
both f(x,y) and p(x,y) : on gray-level values and its neighbors  Local threshold
image with dark background and a light object
image with dark background and two light objects

10. Thresholding
Peaks and valleys of the image histogram can help in choosing the appropriate value for the threshold(s).
Some factors affects the suitability of the histogram for guiding the choice of the threshold:
the separation between peaks;
the noise content in the image;
the relative size of objects and background;
the uniformity of the illumination;
the uniformity of the reflectance.

13. Illumination and reflection role in thresholding
easily use global
thresholding
object and
Background
are separated
difficult to segment
f(x,y) = i(x,y) r(x,y)
a). computer generated reflectance function
b). histogram of reflectance function
c). computer generated illumination function (poor)
d). product of a). and c).
e). histogram of product image
تصویر ساده که فاقد تغییرات روشنایی هست. با ضرب آن در تصویری با اثرات روشنایی غیر یکنواخت تصویری با روشنایی غیر یکنواخت داریم. دره در هیستوگرام اول خیلی بهم ریخته است.
باید illumination correction انجام بشود.
میتوانید روش top-hat را امتحان کنید.
میتوانید روی آستانه گذاری متغیر کار کنید.

14. Basic Global Thresholding
Select an initial To
Segment image use:
Compute the average intensity
and for the pixels in and .
Compute a new threshold:
Until the difference between values of T is smaller than a predefined parameter.
آستانه اولیه را بین مینیمم و ماکزیمم مقادیر انتخاب کنید.
این الگوریتم موقعی که دره آشکاری در هیستوگرام هست ، خوب کار میکند.
based on visual inspection of histogram
Select an initial estimate for T.
Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values  T
Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2
Compute a new threshold value
T = 0.5 (1 + 2)
Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter To.

15. clear valley of histogram and effective segmentation between object and background
3 iterations
with result T = 125

16. Optimum Global Thresholding Using Otsu’s Method
Segmentation is based on “region homogeneity”.
Region homogeneity can be measured using variance (i.e., regions with high homogeneity will have low variance).
Otsu’s method selects the threshold by minimizing the within-class variance.
هدف کمینه کردن میانگین خطای ناشی از تخصیص پیکسلها به دو یا چند گروه هست.
Assumptions
It does not depend on modeling the probability density functions.
It does assume a bimodal histogram distribution
In computer vision and image processing, Otsu's method is used to automatically perform clustering-based image thresholding,[1] or, the reduction of a graylevel image to a binary image. The algorithm assumes that the image contains two classes of pixels following bi-modal histogram (foreground pixels and background pixels), it then calculates the optimum threshold separating the two classes so that their combined spread (intra-class variance) is minimal.[2] The extension of the original method to multi-level thresholding is referred to as the Multi Otsu method.[3] Otsu's method is named after Nobuyuki Otsu

17. Otsu’s Method
Consider an image with L gray levels and its normalized histogram
P(i) is the normalized frequency of i.
Assuming that we have set the threshold at T, the normalized fraction of pixels that will be classified as background and object will be: T
background
object
احتمال کلاس 1 (OBJECT)
احتمال کلاس دو (BACKGROUND)

18. Otsu’s Method
The mean gray-level value of the background and the object pixels will be:
The mean gray-level value over the whole image (“grand” mean) is:

19. Otsu’s Method
The variance of the background and the object pixels will be:
The variance of the whole image is:

20. Otsu’s Method (cont’d) Within-class and between-class variance
It can be shown that the variance of the whole image can be written as follows:
Within-class variance
between-class variance
should be minimized!
should be maximized!

21. Otsu’s Method (cont’d) Determining the threshold
Since the total variance does not depend on T, the T that Minimizes will also maximize
rewrite as
Find the T value that maximizes
Start from the beginning of the histogram and test each gray- level value for the possibility of being the threshold T that maximizes
Otsu’s method is aimed in finding the optimal value for the
global threshold.
I It is based on the interclass variance maximization.
I Well thresholded classes have well discriminated intensity
values.

22. Otsu’s Method (cont’d)
Drawbacks of the Otsu’s method
The method assumes that the histogram of the image is bimodal (i.e., two classes).
The method breaks down when the two classes are very unequal (i.e., the classes have very different sizes)
In this case, may have two maxima.
The correct maximum is not necessary the global one.
The method does not work well with variable illumination.

24. Using Image Smoothing to improve Global Thresholding
Otsu’s method may not work in presence of noise.
Smoothing can produce a histogram with separated peaks.

25. If the distribution is not balanced, no information can be
extracted from the histogram.
Smoothing cannot help.
ناحیه کوچک باشد روش اتسو به مشکل بر میخورد.

26. Using Edges to improve Global thresholding

27. اگر قله های هیستوگرام بلند باریک و دارای دره های عمیق باشه بهتر میشه آستانه بندی کرد.
Edge extraction techniques (e.g., Laplacian), can be used for
selecting the region that carry the valuable information:
Those pixels that belong to the objects and to the background
with an equal probability.

28. Use of edge for global thresholding
Changing the threshold of the Laplacian, several
segmentations are obtained.
It can be useful for nested classes.

29. Multiple Thresholds
Otsu’s method can be extended to a multiple thresholding method
Between-class variance can be reformulated as

30. Multiple Thresholds
The K classes are separated by K-1 thresholds and these optimal thresholds can be solved by maximizing
For example (two thresholds)

31. Multiple Thresholds The following relationships hold:
The optimum thresholds can be found by :
The image is then segmented by

33. Thresholding Exercise

34. Image partitioning based thresholding
In order to face non uniform illumination or reflectance, the
image is partitioned and the thresholding is operated on each
partition.
In each partition, the illuminance and reflectance is supposed
uniform.
In each partition, objects and background have to be equally
represented.

35. Local properties based thresholding
Local properties (e.g., statistics) based criteria can be used for adapting the threshold. For example:
The segmentation is operated using a suitable predicate, Qxy :
where Qxy can be, for instance:
This technique can be easily generalized to multiple thresholds segmentation.

37. Moving averages thresholding
Pixels are visited following a zigzag path and the statistics are
computed using only the last n visited pixels.

38. Using moving average
It discussed is based on computing a moving average along scan lines of an image.
denote the intensity of the point at step k+1.
n denote the number of point used in the average.
is the initial value.
,where b is constant and is the moving average at point (x,y)

39. Growing based segmentation
Region growing is a technique based on a controlled growing of some initial pixels (seeds).
The selection of the seeds can be operated manually or using automatic procedures based on appropriate criteria.
A priori knowledge can be included.
It is strictly application-dependent.
The growing is controlled by the connectivity.
The stop rule is another parameter of the algorithm.
It can depend on the a-priori knowledge on the problem.

40. Region-based segmentation Region Growing
Algorithm:
Choose a random pixels
Use 8-connected and threshold to determine
Repeat a and b until almost points are classified.
Talk about advantage and disavantage

43. Region splitting and Merging
Quadtree based partitioning
Iterative subdivision of the image in homogeneous regions
(splitting).
Joining of the adjacent homogeneous regions (merging).

44. Region splitting and Merging

46. Region merging

47. Region merging

48. Splitting and Merging

49. Watershed Segmentation
Considers the gradient magnitude of an image as a topographic surface.
Markers
Internal – associated with objects of interest
External – associated with background
Pixels having the highest gradient magnitude intensities (GMIs) correspond to watershed lines, which represent the region boundaries.
Water placed on any pixel enclosed by a common watershed line flows downhill to a common local intensity minimum (LMI).
Pixels draining to a common minimum form a catch basin, which represents a segment.
Watershed lines hold the key.
Basic idea:
Hole in regional minimums topography uniformly flooded
from below build dams to prevent merging dam boundaries correspond to divide lines.
Process continued till max level of flooding (corresponding to highest gray-level value) is reached.
Watershed lines form connected path – continuous boundaries.
Often applied to gradient of image rather than image.

51. Oversegmentation
Oversegmentation obtained applying watershed to the image gradient.

52. Watershed with marker
Noise as well as irrelevant details make difficult the application of the watershed technique in real images.
Oversegmentation can be produced.
These problems can be handled limiting the flooding through markers:
internal, associated to the object of interest;
external, associated to the background (border of the objects).
The watershed algorithm can then be applied considering the marker as the only minimum points from which starting the procedure.
The criteria used for defining the markers incorporate the a-priori knowledge on the problem.
External markers:
Points along the watershed
line along highest points.
Internal markers:
(1) That is surrounded higher points .
(2) Points in region form a connected component
(3) All points in connected component have the same
intensity.

53. (a) Smoothing of the original image
(a) Smoothing of the original image. Internal markers are defined as minimum points that forms connected components (in red). The application of the watershed starting from internal markers generates the dams (in green), that can be used as external markers.
(b) Segmentation obtained applying watershed in each region of (a). Other segmentation algorithms can be applied to the single regions, as well.

54. MALTAB files
graythresh, im2bw,
bwdist, watershed,regionprob

55. Watershed center1 = -10; center2 = -center1;
dist = sqrt(2*(2*center1)^2);
radius = dist/2 * 1.4;
lims = [floor(center1-1.2*radius) ceil(center2+1.2*radius)];
[x,y] = meshgrid(lims(1):lims(2));
bw1 = sqrt((x-center1).^2 + (y-center1).^2) <= radius;
bw2 = sqrt((x-center2).^2 + (y-center2).^2) <= radius;
bw = bw1 | bw2;
figure, imshow(bw,'InitialMagnification','fit'), title('bw')
D = bwdist(~bw);
figure, imshow(D,[],'InitialMagnification','fit')
title('Distance transform of ~bw')
D = -D;
D(~bw) = -Inf;
L = watershed(D);
rgb = label2rgb(L,'jet',[ ]);
figure, imshow(rgb,'InitialMagnification','fit')
title('Watershed transform of D')

56. http://www. eecs. berkeley