Digital Image Processing Image Segmenattion Dr. Ghassabi z.r.ghassabi@gmail.com Tehran shomal University Spring 2015

Outline Introduction Digital Image Fundamentals Intensity Transformations and Spatial Filtering Filtering in the Frequency Domain Image Restoration and Reconstruction Color Image Processing Wavelets and Multi resolution Processing Image Compression Morphological Operation Image Segmentation Object representation Object recognition

Image Segmentation The goal of image segmentation is to cluster pixels into salient image regions, i.e., regions corresponding to individual surfaces, objects, or natural parts of objects.

The goals of segmentation Separate image into coherent “objects” image human segmentation CS 376 Lecture 7 Segmentation

3 2 1 Image segmentation pixel count input image intensity white pixels pixel count 3 black pixels gray pixels 1 2 input image intensity These intensities define the three groups. We could label every pixel in the image according to which of these primary intensities it is. i.e., segment the image based on the intensity feature. What if the image isn’t quite so simple? CS 376 Lecture 7 Segmentation

pixel count input image intensity pixel count input image intensity Now how to determine the three main intensities that define our groups? Now how to determine the three main intensities that define our groups? CS 376 Lecture 7 Segmentation

Low-level image segmentation Edge-based Region-based Thresholding Region Growing Divide and Merge Hybrid Watershed تفاوتهای بین ناحیه ای شباهتهای درون یک ناحیه مثل بافت و رنگ و میانگین

Edge-based Segmentation Use Filters that find edges. Prewitt operator Roberts Cross-Difference Operator Sobel Edge Detector Canny Edge Detector Challenges: Noise sensitive None closed edges Consider all edges

Thresholding a point (x,y) belongs to T depends on Multi-level Thresholding پیکسلهای شی و پس زمینه دارای مقادیر شدتی اند که به دو دسته غالب تقسیم می شوند. اونوقت یه آستانه کلی عمل جداسازی رو انجام میدهد. زمانیکه در تصویر دو شی غالب داریم. آنوقت هیستوگرام سه حالت غالب را دارد. پس آستانه گذاری چندگانه داریم. a point (x,y) belongs to to an object class if T1 < f(x,y)  T2 to another object class if f(x,y) > T2 to background if f(x,y)  T1 T depends on only f(x,y) : only on gray-level values  Global threshold both f(x,y) and p(x,y) : on gray-level values and its neighbors  Local threshold image with dark background and a light object image with dark background and two light objects

Thresholding Peaks and valleys of the image histogram can help in choosing the appropriate value for the threshold(s). Some factors affects the suitability of the histogram for guiding the choice of the threshold: the separation between peaks; the noise content in the image; the relative size of objects and background; the uniformity of the illumination; the uniformity of the reflectance.

Illumination and reflection role in thresholding easily use global thresholding object and Background are separated difficult to segment f(x,y) = i(x,y) r(x,y) a). computer generated reflectance function b). histogram of reflectance function c). computer generated illumination function (poor) d). product of a). and c). e). histogram of product image تصویر ساده که فاقد تغییرات روشنایی هست. با ضرب آن در تصویری با اثرات روشنایی غیر یکنواخت تصویری با روشنایی غیر یکنواخت داریم. دره در هیستوگرام اول خیلی بهم ریخته است. باید illumination correction انجام بشود. میتوانید روش top-hat را امتحان کنید. میتوانید روی آستانه گذاری متغیر کار کنید.

Basic Global Thresholding Select an initial To Segment image use: Compute the average intensity and for the pixels in and . Compute a new threshold: Until the difference between values of T is smaller than a predefined parameter. آستانه اولیه را بین مینیمم و ماکزیمم مقادیر انتخاب کنید. این الگوریتم موقعی که دره آشکاری در هیستوگرام هست ، خوب کار میکند. based on visual inspection of histogram Select an initial estimate for T. Segment the image using T. This will produce two groups of pixels: G1 consisting of all pixels with gray level values > T and G2 consisting of pixels with gray level values  T Compute the average gray level values 1 and 2 for the pixels in regions G1 and G2 Compute a new threshold value T = 0.5 (1 + 2) Repeat steps 2 through 4 until the difference in T in successive iterations is smaller than a predefined parameter To.

clear valley of histogram and effective segmentation between object and background 3 iterations with result T = 125

Optimum Global Thresholding Using Otsu’s Method Segmentation is based on “region homogeneity”. Region homogeneity can be measured using variance (i.e., regions with high homogeneity will have low variance). Otsu’s method selects the threshold by minimizing the within-class variance. هدف کمینه کردن میانگین خطای ناشی از تخصیص پیکسلها به دو یا چند گروه هست. Assumptions It does not depend on modeling the probability density functions. It does assume a bimodal histogram distribution In computer vision and image processing, Otsu's method is used to automatically perform clustering-based image thresholding,[1] or, the reduction of a graylevel image to a binary image. The algorithm assumes that the image contains two classes of pixels following bi-modal histogram (foreground pixels and background pixels), it then calculates the optimum threshold separating the two classes so that their combined spread (intra-class variance) is minimal.[2] The extension of the original method to multi-level thresholding is referred to as the Multi Otsu method.[3] Otsu's method is named after Nobuyuki Otsu

Otsu’s Method Consider an image with L gray levels and its normalized histogram P(i) is the normalized frequency of i. Assuming that we have set the threshold at T, the normalized fraction of pixels that will be classified as background and object will be: T background object احتمال کلاس 1 (OBJECT) احتمال کلاس دو (BACKGROUND)

Otsu’s Method The mean gray-level value of the background and the object pixels will be: The mean gray-level value over the whole image (“grand” mean) is:

Otsu’s Method The variance of the background and the object pixels will be: The variance of the whole image is:

Otsu’s Method (cont’d) Within-class and between-class variance It can be shown that the variance of the whole image can be written as follows: Within-class variance between-class variance should be minimized! should be maximized!

Otsu’s Method (cont’d) Determining the threshold Since the total variance does not depend on T, the T that Minimizes will also maximize rewrite as Find the T value that maximizes Start from the beginning of the histogram and test each gray- level value for the possibility of being the threshold T that maximizes Otsu’s method is aimed in finding the optimal value for the global threshold. I It is based on the interclass variance maximization. I Well thresholded classes have well discriminated intensity values.

Otsu’s Method (cont’d) Drawbacks of the Otsu’s method The method assumes that the histogram of the image is bimodal (i.e., two classes). The method breaks down when the two classes are very unequal (i.e., the classes have very different sizes) In this case, may have two maxima. The correct maximum is not necessary the global one. The method does not work well with variable illumination.

Using Image Smoothing to improve Global Thresholding Otsu’s method may not work in presence of noise. Smoothing can produce a histogram with separated peaks.

If the distribution is not balanced, no information can be extracted from the histogram. Smoothing cannot help. ناحیه کوچک باشد روش اتسو به مشکل بر میخورد.

Using Edges to improve Global thresholding

اگر قله های هیستوگرام بلند باریک و دارای دره های عمیق باشه بهتر میشه آستانه بندی کرد. Edge extraction techniques (e.g., Laplacian), can be used for selecting the region that carry the valuable information: Those pixels that belong to the objects and to the background with an equal probability.

Use of edge for global thresholding Changing the threshold of the Laplacian, several segmentations are obtained. It can be useful for nested classes.

Multiple Thresholds Otsu’s method can be extended to a multiple thresholding method Between-class variance can be reformulated as

Multiple Thresholds The K classes are separated by K-1 thresholds and these optimal thresholds can be solved by maximizing For example (two thresholds)

Multiple Thresholds The following relationships hold: The optimum thresholds can be found by : The image is then segmented by

Thresholding Exercise

Image partitioning based thresholding In order to face non uniform illumination or reflectance, the image is partitioned and the thresholding is operated on each partition. In each partition, the illuminance and reflectance is supposed uniform. In each partition, objects and background have to be equally represented.

Local properties based thresholding Local properties (e.g., statistics) based criteria can be used for adapting the threshold. For example: The segmentation is operated using a suitable predicate, Qxy : where Qxy can be, for instance: This technique can be easily generalized to multiple thresholds segmentation.

Moving averages thresholding Pixels are visited following a zigzag path and the statistics are computed using only the last n visited pixels.

Using moving average It discussed is based on computing a moving average along scan lines of an image. denote the intensity of the point at step k+1. n denote the number of point used in the average. is the initial value. ,where b is constant and is the moving average at point (x,y)

Growing based segmentation Region growing is a technique based on a controlled growing of some initial pixels (seeds). The selection of the seeds can be operated manually or using automatic procedures based on appropriate criteria. A priori knowledge can be included. It is strictly application-dependent. The growing is controlled by the connectivity. The stop rule is another parameter of the algorithm. It can depend on the a-priori knowledge on the problem.

Region-based segmentation Region Growing Algorithm: Choose a random pixels Use 8-connected and threshold to determine Repeat a and b until almost points are classified. Talk about advantage and disavantage

Region splitting and Merging Quadtree based partitioning Iterative subdivision of the image in homogeneous regions (splitting). Joining of the adjacent homogeneous regions (merging).

Region splitting and Merging

Region merging

Region merging

Splitting and Merging

Watershed Segmentation Considers the gradient magnitude of an image as a topographic surface. Markers Internal – associated with objects of interest External – associated with background Pixels having the highest gradient magnitude intensities (GMIs) correspond to watershed lines, which represent the region boundaries. Water placed on any pixel enclosed by a common watershed line flows downhill to a common local intensity minimum (LMI). Pixels draining to a common minimum form a catch basin, which represents a segment. Watershed lines hold the key. Basic idea: Hole in regional minimums topography uniformly flooded from below build dams to prevent merging dam boundaries correspond to divide lines. Process continued till max level of flooding (corresponding to highest gray-level value) is reached. Watershed lines form connected path – continuous boundaries. Often applied to gradient of image rather than image.

Oversegmentation Oversegmentation obtained applying watershed to the image gradient.

Watershed with marker Noise as well as irrelevant details make difficult the application of the watershed technique in real images. Oversegmentation can be produced. These problems can be handled limiting the flooding through markers: internal, associated to the object of interest; external, associated to the background (border of the objects). The watershed algorithm can then be applied considering the marker as the only minimum points from which starting the procedure. The criteria used for defining the markers incorporate the a-priori knowledge on the problem. External markers: Points along the watershed line along highest points. Internal markers: (1) That is surrounded higher points . (2) Points in region form a connected component (3) All points in connected component have the same intensity.

(a) Smoothing of the original image (a) Smoothing of the original image. Internal markers are defined as minimum points that forms connected components (in red). The application of the watershed starting from internal markers generates the dams (in green), that can be used as external markers. (b) Segmentation obtained applying watershed in each region of (a). Other segmentation algorithms can be applied to the single regions, as well.

MALTAB files graythresh, im2bw, bwdist, watershed,regionprob

Watershed center1 = -10; center2 = -center1; dist = sqrt(2*(2*center1)^2); radius = dist/2 * 1.4; lims = [floor(center1-1.2*radius) ceil(center2+1.2*radius)]; [x,y] = meshgrid(lims(1):lims(2)); bw1 = sqrt((x-center1).^2 + (y-center1).^2) <= radius; bw2 = sqrt((x-center2).^2 + (y-center2).^2) <= radius; bw = bw1 | bw2; figure, imshow(bw,'InitialMagnification','fit'), title('bw') D = bwdist(~bw); figure, imshow(D,[],'InitialMagnification','fit') title('Distance transform of ~bw') D = -D; D(~bw) = -Inf; L = watershed(D); rgb = label2rgb(L,'jet',[.5 .5 .5]); figure, imshow(rgb,'InitialMagnification','fit') title('Watershed transform of D')

http://www. eecs. berkeley http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html