Digital Image Processing Morphological operation Dr. Ghassabi z.r.ghassabi@gmail.com Tehran shomal University Spring 2015

Outline Introduction Digital Image Fundamentals Intensity Transformations and Spatial Filtering Filtering in the Frequency Domain Image Restoration and Reconstruction Color Image Processing Wavelets and Multi resolution Processing Image Compression Morphological Operation Object representation Object recognition

Morphological Image processing Used to extract image components that are useful in the representation and description of region shape, such as boundaries extraction skeletons convex hull morphological filtering thinning pruning

Morphological Image processing Sets in mathematic morphology represent objects in an image: binary image (0 = white, 1 = black) : the element of the set is the coordinates (x,y) of pixel belong to the object Z2 gray-scaled image : the element of the set is the coordinates (x,y) of pixel belong to the object and the gray levels  Z3

Morphological Image processing Operators by examples:

Morphological Image processing Preliminary: A: A set in Z2 with elements of a=(a1,a2)

Morphological Image processing Reflection and Translation by examples: Need for a reference point. Reference point

Morphological operations Erosion Dilation Opening Closing Reconstruction

Fundamental Morphological Operators: Dilation: Set B: A structural elements. Result Relation to Convolution mask: Flipping Overlapping Other names: Grow, Expand

Dilation Set A

Dilation Output

Dilation Output

Dilation Output

Dilation Output

Dilation Result

Morphological Image processing

Morphological Image processing Dilation by example: B: a 3×3 mask.

Morphological Image processing Application: Gap filling

Dilation in MATLAB I=imread(‘ im.jpg‘); I1=im2bw(I); S=[0 1 0;1 1 1;0 1 0]; I2=imdilate(I1,S); Subplot(1,2,1);imshow(I); Subplot(1,2,2);imshow(I1);

Dilation in MATLAB Exercise What is the result by S=[1 1 1;1 1 1;1 1 1]; What is the result when I1=1-I? Do dilation for a retinal image. Use different structuring elements by “Strel” Se1=strel(‘line’,3,0); Se1=strel(‘line’,3,90); Se1=strel(‘square,3);

Morphological Image processing Erosion: Set B: A structural elements. Other banes: Shrink, Reduce Result در صورتی جواب داریم که تمام المانهای B با A تطابق داشته باشه.

Erosion Set A Structuring element

Erosion

Erosion

Erosion

Erosion

Erosion

Erosion Result

Erosion Set A Structuring element

Erosion

Erosion

Erosion

Erosion

Erosion

Erosion

Erosion Result

Erosion in MATLAB I=imread(‘ im.jpg‘); I1=im2bw(I); S=[0 1 0;1 1 1;0 1 0]; I2=imerode(I1,S); Subplot(1,2,1);imshow(I); Subplot(1,2,2);imshow(I1);

Erosion in MATLAB Exercise Se1= Stre1(‘square’,3); Se1= Stre1(‘square’,5); Se1= Stre1(‘square’,7);

Morphological Image processing

Morphological Image processing Erosion by example: B: a 3×3 mask.

Morphological Image processing Erosion by example: 11 pixels (Diameter)

Morphological Image processing Erosion (More examples):

Morphological Image processing Dilation-Erosion Duality:

Morphological Image processing Erosion Application: Remove details 1,3,5,7,9, and 15 Erode with 13 Dilate with 13

Morphological Image processing Opening and Closing: Dilation expands and Erosion shrinks. Opening: Smooth contour Break narrow isthmuses (Means:(تنگه Remove thin protrusion Closing: Fuse narrow breaks, and long thin gulfs. Remove small holes, and fill gaps.

Morphological Image processing Opening and Closing: Dilation expands and Erosion shrinks. Opening: A erosion followed by a dilation using the same structuring element for both operations. Closing: A Dilation followed by a erosion using the same structuring element for both operations. Opening برای صاف کردن لبه ها و از بین بردن خطوط باریک closing\ برای smoothing و از بین بردن حفره ها و فاصله های نزدیک

Opening Set A Structuring element B

Opening Erosion

Opening Dilation

Morphological Image processing Opening illustration

Morphological Image processing Opening illustration

Morphological Image processing opening : Separate out the circles from the lines, so that they can be counted. Opening with a disk shaped structuring element 11 pixels in diameter gives

Morphological Image processing opening : Extracting the horizontal and vertical lines. The results of an Opening with a 3×9 vertically and 9x3 horizontally oriented structuring element is shown

Closing Set A Structuring element B

Closing Dilation

Closing Erosion

Morphological Image processing Closing illustration:

Morphological Image processing Closing Illustration:

Morphological Image processing Closing (More examples): Removing the small holes while retaining the large holes Closing with a 22 pixel diameter disk Closing with a disk-shaped structuring element with a diameter larger than the smaller holes

Morphological Image processing Closing (More examples): Enhance binary images of objects obtained from thresholding

Opening and Closing:

Morphological Image processing Opening and Closing Duality: Opening Properties: A○B is a subset (subimage) of A If C is a subset of D, then C○B is a subset of D○B (A○B)○B = A○B ↔ Multiple apply has no effect. Closing Properties: A is a subset (subimage) of AB If C is a subset of D, then CB is a subset of DB (AB)B = AB ↔ Multiple apply has no effect.

Noise Reduction Opening Dilation of Opening Closing of Opening

Opening and Closing in MATLAB Exercise I=imread(‘ im.jpg‘); I1=im2bw(I); S=strel(‘square’,13); I2=imopen(I1,S); I2=imclose(I2,S); Subplot(1,2,1);imshow(I); Subplot(1,2,2);imshow(I1);

Morphological Image processing Hit-or-Miss Shape Detection X-Y-X shape X enclosed by W B1: Object related B2: Background related For shape recognition

Morphological Image processing Hit-or-Miss: Application: shape recognition

Hit-Miss in MATLAB B1=[0 1 0; 1 1 1;0 1 0]; B2=[1 0 1; 0 0 0; 1 0 1]; I2=imerode(I1,B1); I3=imerode((~I1), B2); I4=I2.^I3; I4 are objects that we want to detect

Morphological Image processing Hit-or-Miss: Another application: Corners

Morphological Image processing Morphological Operator Applications: Boundary Extraction: Eroded Difference

Boundary Extraction  (A) = A – (A B) – Set A B

Boundary Extraction A – B  (A)

Boundary Extraction in MATLAB I2=imerode(I,B2); I3=I-I2;

Morphological Image processing Region Filling: Start form p inside boundary. مختصات نقطه شروع را باید بدهیم که با structure دایلیت میشه و بعد اشتراک میگیریم با مکمل A آخرش دورش رو هم در نظر گرفته

Morphological Image processing Region Filling Example: Semi-automated to cancel reflection effect Fig. 9.16(c) shows the result of filling all the spheres. Because it must be known whether black points are background points or sphere inner points, fully automating this procedure requires that additional "intelligence" be built into the algorithm. We give a fully automatic approach in Section 9.5.9 based on morphological reconstruction. (See also Problem 9.23.)

Region Filling in MATLAB [M,N]=size(I1); M1=[0 1 0; 1 1 1; 0 1 0]; P0=zeros(M,N); P0(126,152)=1; I2=imdilate(P0,M1); I2=I2.*(~I1); done=1; While(done) if (I2==P0) done=0; else P0=I2 end I2=imdilate(I2,M); end I2-=I1+I2; imshow(I2)

Region Filling in MATLAB I2=imfill(I1, [126,152]);

Morphological Image processing Connected components Extraction: Start from p belong to desired region. Image labeling مکان اولین نقطه سفید در عکس نقطه شروع هست.

Connected Component - Labeling Starting point p q Structuring element B Image A

Connected Component - Labeling X0  B X1 = (X0  B)  A

Connected Component - Labeling X2 = (X1  B)  A X8 = (X7  B)  A

Connected Components in MATLAB I1=imread(); I1=im2bw(A); [M,N]=size(I1); I1=uint8(I1).*255; M1=[0 1 0; 1 1 1; 0 1 0]; I1=imopen(I1,M1); P0=zeros(M,N); I2=P0; %find P0----------- P=0; While(P==0&& i<M) if (Ij<N) j=j+1; else j=1; i=i+1 end if (I1(i,j)==255) P=1; end end P0(i,j)=255;

Connected Components in MATLAB I2=imdilate(P0,M1); I2=uint8(I2); I2=I2.*(I1); done=1; While(done) if (I2==P0) done=0; else P0=I2 end I2=imdilate(I2,M);I2=I2.*(I1); end [x,y]=find(I2==255); x1=mean(x); Y1=mean(y); S=sum(sum(I2./255)); imshow(I2) This code Finds one of objects

Connected Components in MATLAB I1=imread(); I1=im2bw(A); [M,N]=size(I1); I1=uint8(I1).*255; M1=[0 1 0; 1 1 1; 0 1 0]; I1=imopen(I1,M1); P0=zeros(M,N); I2=P0; P=1; n=1; %find P0----------- While (P==1) P=0; While(P==0&& i<M) if (Ij<N) j=j+1; else j=1; i=i+1 end if (I1(i,j)==255) P=1; end end P0(i,j)=255; This code finds other objects

Connected Components in MATLAB I2=imdilate(P0,M1); I2=uint8(I2); I2=I2.*(I1); done=1; While(done) if (I2==P0) done=0; else P0=I2 end I2=imdilate(I2,M);I2=I2.*(I1); end [x,y]=find(I2==255); X1(n)=mean(x); Y1(n)=mean(y); S(n)=sum(sum(I2./255)); I2=(I2./255).*(n); I1=I1-I2 n=n+1; End X1(n-1)=[]; y1(n-1)=[]; imshow(I2) This code finds other objects

Connected Components in MATLAB I=imread(I); I1=im2bw(I); [l,n]=bwlabel(I1) [r,c]=find(l==1) R=mean(r);c=mean(c) n % the number of objects

Morphological Image processing Original Connected Components with size Thr Erosion (5×5)

Morphological Image processing Convex Hull of S: Smallest Convex set H, containing S Define Four basic structural elements, Bi, i=1,2,3,4 Convex

Morphological Image processing Converged: C C C C

Morphological Image processing Shortcoming of previous algorithm: Grow more than minimum required convex size. Limit to vertical-horizontal expansion.

Morphological Image processing Thinning:

Morphological Image processing Convert to m-connection

Morphological Image processing Thickening: Structural elements are as before. where B is a structuring element suitable for thickening. As in thinning. thickening can be defined as a sequential operation: The structuring elements used for thickening have the same form as those shown in Fig. 9.2l(a). but with all 1s and 0s interchanged. However, a separate algorithm for thickening is seldom used in practice. Instead, the usual procedure is to thin the background of the set in question and then complement the result. In other words. to thicken a set A. we form C = AC, thin C, and then form C C. Figure 9.22 illustrates this procedure. Depending on the nature of A. this procedure can result in disconnected points, as Fig. 9.22(d) shows. Hence thickening by this method usually is followed by post processing to remove disconnected points Note from Fig. 9.22(c) that the thinned background forms a boundary for the thickening process This useful feature is not present in the direct implementation of thickening using Eq. (9.5-I0). and it is one of the principal reasons for using background thinning to accomplish thickening.

Morphological Image processing Skeletonization A with notation S(A): For z belong to S(A) and (D)z, the largest disk centered at z and contained in A, one can not find a larger disk containing (D)z and included in A. Disk (D)z touches the boundary of A at two or more different points.

Morphological Image processing Skeleton by example: As Fig. 9.23 shows. the notion of a skeleton. S(A). of a set A is intuitively simple. We deduce from this figure that (a) lf z is a point of S(A) and (D)z is the largest disk centered at z and contained in A. one cannot find a larger disk (not necessarily centered at z) containing (D)z and included in A. The disk (D)z is called a maximum disk. (b) The disk (D)Z touches the boundary of A at two or more different places.

MATLAB bwmorph

Morphological Image processing Formulation:

Morphological Image processing

Morphological Image processing Pruning

Morphological Image processing Morphological Reconstruction

Reconstruction in MATLAB B1=ones(51,1); I2=imerode(I1, B1); I3=imreconstruct(I2,I1);

Morphological Image processing

Morphological Image processing

Morphological Image processing

Morphological Image processing Morphological reconstruction has a broad spectrum of practical applications, each determined by the selection of the marker and mask images, by the structuring elements used, and by combinations of the primitive operations defined .in the preceding discussion. The following examples illustrate the usefulness of these concepts. Opening by reconstruction: In a morphological opening, erosion removes small objects and the subsequent dilation attempts to restore the shape of objects that remain. However, the accuracy of this restoration is highly dependent on the similarity of the shapes of the objects and the structuring element used.

Morphological Image processing Opening by reconstruction restores exactly the shapes of the objects that remain after erosion. The opening by reconstruction of size n of an image F is defined as the reconstruction by dilation of F from the erosion of size n of F; that is, where (F nB) indicates n erosions of F by B Note that F is used as the mask in this application.

Morphological Image processing Figure 9.29 shows an example of opening by reconstruction. In this illustration, we are interested in extracting from Fig. 9.29(a) the characters that contain long. vertical strokes. Opening by reconstruction requires at least one erosion. so we perform that step first. Figure 9.29(b) shows the erosion of Fig. 9.29(a) with a structuring element of length proportional to the average height of the tall characters (51 pixels) and width of one pixel.

Morphological Image processing

Morphological Image processing Figure 9.31 shows a more practical example. Figure 9.31(b) shows the complement of the text image in Fig. 9.31(a), and Fig. 9.31(c) is the marker image, F, generated using Eq. (9.5-28).This image has a border of 1s, except at locations corresponding to 1s in the border of the original image. Finally, Fig. 9.31 (d) shows the image with all the holes filled.

Morphological Image processing Border clearing: A border-clearing procedure based on morphological reconstruction: original image as the mask and the following marker image: first computes the morphological reconstruction (F) (which simply extracts the objects touching the border) and then computes the difference to obtain an image, X, with no objects touching the border The extraction of objects from an image for subsequent shape analysis is a fundamental task in automated image processing An algorithm for removing objects that touch (i.e., are connected to) the border is a useful tool because (1) it can be used to screen images so that only complete objects remain for further processing, or (2) it can be used as a signal that partial objects are present in the field of view.

Morphological Image processing

Morphological Image processing As an example, consider the text image again. Figure 9.32(a) in the previous page shows the reconstruction (F) obtained using a 3 3 structuring element of all 1s (note the objects touching the boundary on the right side), and Pig. 9.32(b) shows image X, computed using Eq. (9.5-31). If the task at hand were automated character recognition. having an image in which no characters touch the border is most useful because the problem of having to recognize partial characters (a difficult task at best) is avoided

Morphological Image processing Extension to Gray-Level images: f(x,y): the input image b(x,y): a structuring element (a subimage function) (x,y) are integers. f and b are functions that assign a gray-level value (real number or real integer) to each distinct pair of coordinate (x,y)

Morphological Image processing Extension to Gray-Level images: Dilation: Df and Db are the domains of f and b, respectively. condition (s-x) and (t-y) have to be in the domain of f and (x,y) have to be in the domain of b is similar to the condition in binary morphological dilation where the two sets have to overlap by at least one element

Gray-Scale Morphology 5 100 102 110 103 109 10 12 14 11 17 112 13 101 106 105 104 15 111 100 102 110 103 109 115 117 17 108 13 10 111 12 114 11 14 107 116 101 20 15 112 104 Structuring element b تصویر روشن تر می شود Image f Dilated image

Gray-Scale Morphology 5 100 102 110 103 109 10 12 14 11 17 112 13 101 106 105 104 15 111 100 102 110 103 109 5 7 6 17 13 10 95 12 11 14 101 15 112 104 Structuring element b تصویر تاریک می شود Image f Eroded image

Morphological Image processing Dilation Similarity with Convolution: f(s-x) : f(-x) is simply f(x) mirrored with respect to the original of the x axis. the function f(s-x) moves to the right for positive s, and to the left for negative s. Max operation replaces the sums of convolution Addition operation replaces with the products of convolution. General effect If all the values of the structuring element are positive, the output image tends to be brighter than the input Dark details either are reduced or eliminated, depending on how their values and shapes relate to the structuring element used for dilation.

Morphological Image processing 1D example:

Morphological Image processing Erosion: Condition (s+x) and (t+y) have to be in the domain of f and (x,y) have to be in the domain of b is similar to the condition in binary morphological erosion where the structuring element has to be completely contained by the set being eroded.

Morphological Image processing Similarity to 2D correlation f(s+x) moves to the left for positive s and to the right for negative s. General effect If all the elements of the structuring element are positive, the output image tends to be darker than the input The effect of bright details in the input image that are smaller in area than the structuring element is reduced, with the degree of reduction being determined by the gray-level values surrounding the bright detail and by the shape and amplitude values of the structuring element itself.

Morphological Image processing 1D case:

Morphological Image processing Erosion-Dilation Duality:

Morphological Image processing Structuring elements in gray-scale morphology: Non flat Flat

Morphological Image processing

Morphological Image processing Dilation-Erosion Original Dilation Erosion

Morphological Image processing Erosion and Dilation Morphological Image processing Application of dilation: Removing noise

Morphological Image processing Erosion and Dilation Morphological Image processing Dilation in edge detection:

Morphological Image processing Erosion and Dilation Morphological Image processing Erosion in removing salt noise:

Morphological Image processing Opening-Closing: Same (binary) relation with Dilation-Erosion: هر دو تصویر را smooth میکنند.

Morphological Image processing 1D Example: One scan line Rolling ball for opening Opening results Rolling ball for Closing Closing Results

Morphological Image processing Opening-Closing Properties: Opening: Closing: er indicates that the domain of e is a subset of the domain of r, and also that e(x,y) ≤ r(x,y) for any (x,y) in the domain of e

Morphological Image processing Opening The structuring element is rolled underside the surface of f All the peaks that are narrow with respect to the diameter of the structuring element will be reduced in amplitude and sharpness So, opening is used to remove small light details, while leaving the overall gray levels and larger bright features relatively undisturbed. The initial erosion removes the details, but it also darkens the image. The subsequent dilation again increases the overall intensity of the image without reintroducing the details totally removed by erosion.

Morphological Image processing Closing The structuring element is rolled on top of the surface of f Peaks essentially are left in their original form (assume that their separation at the narrowest points exceeds the diameter of the structuring element) So, closing is used to remove small dark details, while leaving bright features relatively undisturbed. The initial dilation removes the dark details and brightens the image The subsequent erosion darkens the image without reintroducing the details totally removed by dilation

Morphological Image processing Opening Closing Example Opening: Decreased size of small bright details. No changes to dark region Closing: Decreased size of small dark details. No changes to bright region

Morphological Image processing

Morphological Image processing Erosion Opening Opening:suppresses bright details smaller than the specified SE and closing suppresses dark details. They are used often in combination as morphological filters for image smoothing and noise removal

Morphological Image processing Opening in removing salt noise:

Morphological Image processing Opening in removing pepper noise:

Morphological Image processing Closing for pepper noise:

Morphological Image processing Closing for salt noise:

Morphological Image processing Gray Level Morphological Examples: Smoothing: Gradient: Laplacian: Dilation Erosion Smoothing Gradient Laplacian Dilation and erosion can be used in combination with image subtraction to obtain the morphological gradient of an image: The dilation thickens regions in an image and the erosion shrinks them. Their difference emphasizes the boundaries between regions.

Morphological Image processing Smoothing

Morphological Image processing Gradient:

Gradient and smoothing in MATLAB I=imread(); I=rgb2gray(I); M=strel([5 5 5; 5 5 5; 5 5 5]); I1=imdilate(I,M); I2=imerode(I,M) I3=imopen (I,M) I4=imclose(I,M); I5=imclose(I3,M) %smoothing I6=imclose(I3,M);

Morphological Image processing Combining image subtraction with openings and closings results in top-hat and bottom-hat transformations. Top-hat transformation: Bottom-hat transformation: Notice: The top-hat transform is used for light objects on a dark background, and the bottom-hat transform is used for the converse.

Morphological Image processing Top-hat: Enhancing details in presence of shades

Morphological Image processing Granulometry Determining the size distribution of particles in an image. Granulometry consists of applying openings with SEs of increasing size. For each opening, the sum of the pixel values in the opening is computed. To emphasize changes between successive openings, we compute the difference between adjacent elements of the 1-D array. The peaks in the plot are an indication of the size distributions of the particles in the image.

Morphological Image processing Granulometry

Morphological Image processing Texture Segmentation

Morphological Image processing Finding a boundary between two regions based on their textural content.

Morphological Image processing Gray-scale Morphological reconstruction: Let f and g denote the marker and mask images. Geodesic dilation of size 1: ^ denotes the point-wise minimum operator. Geodesic dilation of size n: Geodesic erosion of size 1: Geodesic erosion of size n:

Morphological Image processing Gray-scale Morphological reconstruction: Morphological reconstruction by dilation: Morphological reconstruction by erosion:

Morphological Image processing Gray-scale Morphological reconstruction: Opening by reconstruction of size n: Closing by reconstruction of size n: