Allan Hanbury Mathematical Morphology: from Erosion to Watersheds PRIP, Vienna University of Technology Favoritenstraße 9/1832 A-1040 Vienna, Austria PRIP, Vienna University of Technology Favoritenstraße 9/1832 A-1040 Vienna, Austria

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

Image Processing (I) => Image processing may be divided into three classes of questions, namely Codification, Extraction of characteristics and Segmentation. 1)Codification: Codification concerns the representation of the images. It comprises: Acquisition: Transformation of analogue images into numerical ones. Compression: Modification of image representation. Synthesis: Generation of an image from a more symbolic representation. Analog Image Numerical Image Acquisition Numerical Image n° 2 Compression Numerical Image n° 1 Synthesis Numerical Image Numbers

Image Processing (II) 2)Extraction of Characteristics: The aim here is to improve image quality or to exhibit some of its features. This includes in particular measurements, noise reduction, and filtering. 3)Segmentation: Segmentation consists in partitioning the images into zones which are homogeneous according to a given criterion. Image2 Image1 Image Tranformations Numbers PixelsRegions Segmentation

Definitions of Mathematical Morphology MathematicsPhysics Engineering Lattice theory for objects or operators in continuous or discrete spaces; Topological and stochastic models. Signal analysis techniques based on set theory aiming at the study of relations between physical and structural properties. Algorithms and software / hardware tools for developing signal processing applications. Signal Processing Nonlinear signal processing technique based on minimum and maximum operations.

Extensivity and Anti-extensivity: A transformation  is extensive if its output is always greater than its input. It is anti-extensive when the output is always smaller than the input. Extensivity : X  (X) Anti-extensivity : X  (X) X  Set (extensivity) Function (extensivity) Input Output Comparison (I)

Idempotence: A transformation  is idempotent if its output is invariant with respect to the transformation itself: Idempotence:  [  (X) ] =  (X) Increasingness: A transformation  is increasing if it preserves the ordering relation between images: Increasing :  f, g, f  g   ( f )   ( g ) Input Output Comparison (II)

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

A structuring element (SE) is a small set used to probe the image under study. For the Erosion of a set, we ask the question “Where does the structuring element fit the set?” The erosion of a set X by a structuring element B is denoted by  B (X) and is defined as the locus of points x such that B is included in X when its origin is placed at x:  B (X) = X  B = { x: B x  X } A structuring element (SE) is a small set used to probe the image under study. For the Erosion of a set, we ask the question “Where does the structuring element fit the set?” The erosion of a set X by a structuring element B is denoted by  B (X) and is defined as the locus of points x such that B is included in X when its origin is placed at x:  B (X) = X  B = { x: B x  X } Erosion Initial set X Erosion of X Image size 256  256 pixels SE size 15  15 pixels

Dilation is the dual operator of erosion. It is based on the question “Where does the structuring hit the set?” The dilation of a set X by a structuring element B is denoted by  B (X) and is defined as the locus of points x such that B hits X when its origin coincides with x:  B (X) = X  B = { x: B x  X   } Dilation is the dual operator of erosion. It is based on the question “Where does the structuring hit the set?” The dilation of a set X by a structuring element B is denoted by  B (X) and is defined as the locus of points x such that B hits X when its origin coincides with x:  B (X) = X  B = { x: B x  X   } Dilation Initial set X Dilation of X Image size 256  256 pixels SE size 15  15 pixels

Equivalence between Sets and Functions A function can be viewed as a stack of decreasing sets. Each set is the intersection between the umbra of the function and a horizontal plane. X (f) = { x  E, f(x)  }  f(x) = sup {  x  X (f) }

Dilation and Erosion by a flat structuring Element Definition: The dilation (erosion) of a function by a flat structuring element B is introduced as the dilation (erosion) of each set X f  by B. They are said to be planar. This definition leads to the following formulae : ( f  B) (x) = sup{ f(x  y), y  B } ( f  B) (x) = inf { f(x  y), y  B } Erosion shrinks positive peaks. Peaks thinner that the structuring element disappear. It also expands the valleys and the sinks. Dilation produces the dual effects Space grey levels Original Dilate Eroded Structuring element

Greyscale Dilation and Erosion examples Initial image Dilate Erode Image size 256  256 pixels SE size 9  9 pixels

Properties of Dilation and Erosion (I) Dilation and Erosion are dual transformations with respect to complementation. This means that any erosion of an image is equivalent to a complementation of the dilation of the complemented image with the same structuring element (and vice-versa). In terms of symbols:  B =   B  Dilation and Erosion are dual transformations with respect to complementation. This means that any erosion of an image is equivalent to a complementation of the dilation of the complemented image with the same structuring element (and vice-versa). In terms of symbols:  B =   B  BB  BB

Properties of Dilation and Erosion (II) Dilation and Erosion are increasing transformations.

Residues of Transformations Definition The residue between two transformations  et  is their difference Set case: Functions case:   (X) =  X)   (X) Residue      ,  (X) =  ( X) \  (X)

Morphological Gradients Gradient by erosion: It is the residue between the identity and an erosion, i.e.: for sets g  (X) = X / (X  B) for functions g  (f) = f  (f  B) The goal of gradient transformations is to highlight contours. In digital morphology, three Beucher’s gradients based on the unit disc are defined: Gradient by dilation: It is the residue between a dilation and the identity, i.e.: for sets g + (X) = (X  B) / X for functions g + (f ) = (f  B)  f

Morphological Gradients (II) and Laplacian Symmetrical gradient: It is the residue between a dilation and an erosion for sets g (X) = (X  B) / (X  B) for functions g(f) = (f  B)  (f  B) Laplacian: It is the residue between the gradients by dilation and erosion, for functions L(f) = g + (f )  g  (f) Note: These notions correspond the “classical” notions of gradient and Laplacian (if they exist), in the limit when the radius of the disc tends towards zero.

Morphological Gradient examples Initial image Gradient by erosion Image size 256  256 pixels SE size 3  3 pixels Symmetrical Gradient Gradient by dilation

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

The mapping is called opening, and is denoted by       XoB = [(X  B)  By commuting the factors   and    we obtain the closing       X  X   B] Opening and Closing The problem of an inverse operator Several different sets may admit a same erosion, or a same dilate. But among all possible inverses, there always exists a smallest one (a largest one). It is obtained by composing erosion with the corresponding dilation (or vice versa). Erosion structuring element ?

Effects of Opening Geometrical interpretation The opened set   (X) is the union of the structuring elements B x which are included in set X. When B is a disc, the opening reduces the capes, removes the small islands and opens isthmuses. Structuring element Opening

Effects of Closing Geometrical interpretation The closing   (X) is the complement of the domain swept by B as it misses set X. All background structures that cannot contain the structuring element are filled by the closing. When B is a disc, the closing closes the channels, fills completely the small lakes, and partly the gulfs. Structuring element Closing

Effects on Functions The opening and closing create a simpler function than the original. They smooth in a nonlinear way. The opening (closing) removes positive (negative) peaks that are thinner than the structuring element. The opening (closing) remains below (above) the original function.

Greyscale Opening and Closing examples Initial image Closing Opening Image size 256  256 pixels SE size 9  9 pixels

Properties of Opening and Closing Opening and Closing are dual transformations with respect to complementation. Openings are anti-extensive transformations (some pixels are removed) and closings are extensive transformations (some pixels are added). Opening and closing are both increasing transformations. Opening and closing are both idempotent transformations:  and  Opening and Closing are dual transformations with respect to complementation. Openings are anti-extensive transformations (some pixels are removed) and closings are extensive transformations (some pixels are added). Opening and closing are both increasing transformations. Opening and closing are both idempotent transformations:  and 

Suprema of Openings Theorem: Any supremum of openings is still an opening. Any infimum of closings is still a closing. Theorem: Any supremum of openings is still an opening. Any infimum of closings is still a closing. Application: For creating openings with specific selection properties, one can use structuring elements with various shapes and take their supremum. Application: For creating openings with specific selection properties, one can use structuring elements with various shapes and take their supremum. Original Opening by sup.

Goal The “top-hat” transformation, due to F. Meyer, aims to suppress slow trends, therefore to enhance the contrasts of some features in images, according to size or shape criteria. This operator is used mainly for numerical functions. Definition The “top-hat” transformation is the residue between the identity and an opening:  (f) = f   (f) ( functions) ;  (X) = X \  (X) (sets) A dual top-hat can be defined: the residue between a closing and the identity:  *(f) =  (f)  f ( functions) ;  (X) =  (X) \ X (sets) “Top-hat” Transformation

Use of the Top-hat Sets The top-hat extracts the objects that have not been eliminated by the opening. That is, it removes objects larger than the structuring element. Functions The top-hat is used to extract contrasted components with respect to the background. The basic top-hat extracts positive components and the dual top hat the negative ones. Typically, top-hats remove the slow trends, and thus performs a contrast enhancement.

Example of a Top-hat Negative image of the retina. Top hat by a hexagonal opening of size 10. Top hat by the sup of three openings by linear segments of size 10. Comment: The goal is the extraction of the aneurisms (the small white spots). Top hat c), better than b) is far from being perfect. Here opening by reconstruction yields a correct solution.

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

Geodesic Distance Definition The set geodesic distance d X :  n   n  , with respect to reference set X is defined by d X (x,y) =Infimum of the lengths of the paths going from x to y and included X; d X (x,y) = +   if no such path exists. Properties 1) d X is a generalized distance, since d X (x,y) = d X (y,x) d X (x,y) = 0  x = y d X (x,z)  d X (x,y) + d X (y,z) 2) The geodesic distance is always larger than the Euclidean one; 3) A geodesic segment may not be unique. X x z y y'  Examples of geodesics in  2 : N.B. the portions of geodesics included in the interior of X are line segments

Geodesic Discs The notion of geodesic path is seldom used. By contrast, the notion of geodesic discs appears very often: B X, (z) = {y, d X (z,y)  } When the radius r increases, the discs progress as a wave front emitted from z inside the medium X. For a given radius, the discs B X, can be viewed as a set of structuring elements which vary from place to place. z X B X, ( z )

The geodesic dilation of size  of Y inside X is written as follows:  X   Y    B X, (y),  y  Y  The geodesic dilation of size  of Y inside X is written as follows:  X   Y    B X, (y),  y  Y  Geodesic Dilation X Y geodesic dilation

Geodesic Erosion The geodesic erosion is the dual transformation of the geodesic dilation with respect to set complementation. But the complement is taken inside the mask X (i.e. Y  X \ Y = X  Y C ), which results in the erosion :  X (Y) = X \  X (X \ Y) i.e.  X (Y) =  ( Y  X c )  X where  stands for an isotropic erosion of size 1. Note the difference between and  X (Y) and  (Y)  X. X Y  (Y)  X Y  X Geodesic erosion

In the digital metrics on  n, and when  (x) stands for the unit ball centered at point x, then the unit geodesic dilation is defined by the relation:  X  Y   Y  X The dilation of size n is then obtained by iteration :  X,n (Y) =  X (n)  Y , with  X (n)  Y   Y  X  X...  X X is called the mask, and Y the marker. (Binary) Digital Geodesic Dilation X Y X(Y)X(Y)  X (2) ( Y )

Reconstruction by dilation The reconstruction by dilation of a mask image X from a marker image Y is defined as the geodesic dilation of Y with respect to X until stability. This can be seen as an infinite geodesic dilation:  X  (Y) =   X, (Y),  or as an iteration of unit geodesic dilations until there is no further change: R X (Y) =  X (i)  Y  where i is such that  X (i)  Y  X (i+1)  Y  This an opening as it is Increasing [ Y 1  Y 2  R X (Y 1 )  R X (Y 2 ) ] Anti-extensive [ R X (Y)  X ] Idempotent [ R R (Y) (Y) = R X (Y) ] The reconstruction by dilation of a mask image X from a marker image Y is defined as the geodesic dilation of Y with respect to X until stability. This can be seen as an infinite geodesic dilation:  X  (Y) =   X, (Y),  or as an iteration of unit geodesic dilations until there is no further change: R X (Y) =  X (i)  Y  where i is such that  X (i)  Y  X (i+1)  Y  This an opening as it is Increasing [ Y 1  Y 2  R X (Y 1 )  R X (Y 2 ) ] Anti-extensive [ R X (Y)  X ] Idempotent [ R R (Y) (Y) = R X (Y) ] X Y X

Application: Filtering by Erosion-Reconstruction Firstly, the erosion X  B  suppresses the connected components of X that cannot contain a disc of radius  This is then used as the marker Y. Then the opening R X (Y) of marker Y = X  B   “re-builds” all the others. Firstly, the erosion X  B  suppresses the connected components of X that cannot contain a disc of radius  This is then used as the marker Y. Then the opening R X (Y) of marker Y = X  B   “re-builds” all the others. a) Initial imageb) Eroded of a) by a disc c) Reconstruction of b) inside a)

Application: Removal of the edge grains Let Z be the set of the edges, and X be the grains under study; Set Y is the reconstruction of Z  X inside set X ; the set difference between X and Y extracts the internal particles. Let Z be the set of the edges, and X be the grains under study; Set Y is the reconstruction of Z  X inside set X ; the set difference between X and Y extracts the internal particles. X Z a) Initial imageb) Particles that hit the edges c) Residue a) - b)

Application: Hole Filling initial image X A = part of the edge that hits X C reconstruction of A inside X C Comment: efficient algorithm, except for the particles that hit the edges of the field.

Individual Analysis of Particles Algorithm (J.C. Klein) While set X is not empty do { ~ p := first point of the video scan; ~ Y := connected component of X reconstructed from p; ~ Processing of Y (and various measurements) ; ~ X := X \ Y } Algorithm (J.C. Klein) While set X is not empty do { ~ p := first point of the video scan; ~ Y := connected component of X reconstructed from p; ~ Processing of Y (and various measurements) ; ~ X := X \ Y } X Y new X Individual extraction of particles

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

Numerical Geodesic Dilations (I) Let f and g be two numerical functions from  d into , with g  f. The binary geodesic dilation of size of each cross section of g inside that of f at the same level induces on g a dilation  f, (g) (S. Beucher). Function g is the marker, and f is the mask. Let f and g be two numerical functions from  d into , with g  f. The binary geodesic dilation of size of each cross section of g inside that of f at the same level induces on g a dilation  f, (g) (S. Beucher). Function g is the marker, and f is the mask. Numerical geodesic dilation of g with respect to f f g  f, (g)

Numerical geodesic Dilations (II) The digital version starts from the unit geodesic dilation:  f (g) = inf (g  B, f ) which is iterated n times to give a geodesic dilation of size n  f,n (g) =  f (n) (g) =  f (  f.... (  f (g))). The Euclidean and digital erosions derive from the corresponding dilations by the following duality  f (g) = m    m  f ) (m  g), which is different to the binary duality. m is the maximum possible value in the image (i.e. 255 for 8-bit images). The digital version starts from the unit geodesic dilation:  f (g) = inf (g  B, f ) which is iterated n times to give a geodesic dilation of size n  f,n (g) =  f (n) (g) =  f (  f.... (  f (g))). The Euclidean and digital erosions derive from the corresponding dilations by the following duality  f (g) = m    m  f ) (m  g), which is different to the binary duality. m is the maximum possible value in the image (i.e. 255 for 8-bit images). Numerical geodesic erosion of f with respect to g f g  f, (g)

Numerical Reconstruction The reconstruction opening of f from g is the supremum of the geodesic dilations of g inside f, this sup being considered as a function of f: R f (g) =   f, (g),  or, as before: R f (g) =  f (i)  g  where i is such that  f (i)  g  f (i+1)  g  The corresponding reconstruction by erosion is: R f *(g) =  f (i)  g  where i is such that  f (i)  g  f (i+1)  g  The reconstruction opening of f from g is the supremum of the geodesic dilations of g inside f, this sup being considered as a function of f: R f (g) =   f, (g),  or, as before: R f (g) =  f (i)  g  where i is such that  f (i)  g  f (i+1)  g  The corresponding reconstruction by erosion is: R f *(g) =  f (i)  g  where i is such that  f (i)  g  f (i+1)  g  Numerical Reconstruction of g inside f : g  rec (f ; g) f

Applications of Reconstruction The three major applications are : swamping, or reconstruction of a function by imposing markers for the maxima; reconstruction from an erosion contrast opening, which extracts and filters the maxima. The three major applications are : swamping, or reconstruction of a function by imposing markers for the maxima; reconstruction from an erosion contrast opening, which extracts and filters the maxima.

Reconstruction Opening by Erosion Goal: contour preservation Whereas the standard opening modifies contours, this transform is aimed at efficiently and precisely reconstructing the contours of the objects which have not been totally removed by the filtering process. Algorithm - the mask is the original signal, - the marker is an eroded of the mask. Structuring element Erosion Dilation Reconstruction Original opening by a disc Opening by reconstruction :B R f [  B (f) ] = {  f (n) [  B (f )], n > 0 }

Application to Retina Examination a) Initial image c) difference a) minus b) followed by a threshold b) closing by dilation-reconstruction followed by opening by erosion-reconstruction Comment: The aim is to extract and to localise aneurisms. Reconstruction operators ensure that one can remove exclusively the small and isolated peaks (case study due to F. Zana and J. C. Klein).

Reconstruction of a Function from Markers Goal To remove the useless maxima (or minima) of a function. Algorithm The “marker” is a bi-valued (0,m) function identifying the peaks of interest. The reconstruction process result is the largest function  f and admitting maxima at the marked points only. It is called the swamping of f by opening (S. Beucher, F. Meyer), Swamping of f by markers m ( by opening ) marker

An Example of Swamping: Contrast Opening Algorithm -Shift down the initial function f by constant c; -Rebuild f from function f - c, i.e. R f (f  c) = {  f (n) (f  c), n > 0 } Goal Both morphological and reconstruction openings reduce the functions according to size criteria which work on their cross sections. In opening by dynamics, the criterion holds on grey tone contrast (M.Grimaud).

Application to Maxima Detection The maxima of a numerical function on a space E are the connected components of E where f is constant and surrounded by lower values. Therefore they are given by the residues of contrast opening of shift c = 1 More generally, the residuals associated with a shift c extract the maxima surrounded by a descending zone deeper than c. They are called Extended Maxima (S. Beucher). The maxima of a numerical function on a space E are the connected components of E where f is constant and surrounded by lower values. Therefore they are given by the residues of contrast opening of shift c = 1 More generally, the residuals associated with a shift c extract the maxima surrounded by a descending zone deeper than c. They are called Extended Maxima (S. Beucher).

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

SKIZ, or Skeleton by Influence Zones Definition (C. Lantuejoul) Consider a compact set X in  2. The zone of influence of a component X i of X is the set of points of the plane that are closer to X i than to any other component. The SKIZ is then defined as the boundary of all zones of influence. Algorithm In the digital case, the SKIZ is constructed in two steps: 1)Thinning of the background (with L in the hexagonal case) 2)Pruning of the thinned transform (with E in the hexagonal case)

Let Y =  { Y i, i  be a set of the Euclidean plane made of I compact connected components, and included in a compact set X. The geodesic zone of influence of a component Y i in X, is formed by all points of X whose geodesic distance to Y i is smaller than to any other component of Y zi (Y i \ X) = {a  X,  k  i, d X (a, Y j )  d X (a,Y k )} where the geodesic distance from point a to set Y is the inf of the geodesic distances from a to all points of Y. The geodesic SKIZ is then the boundaries of all geodesic zones of influence. Geodesic SKIZ An example of geodesic SKIZ SKIZ (Y) Y 3 Y 2 Y 1 X zi X (Y 1 )

Distance function (I) Definition : The distance function is an intermediate step between sets and functions. When a distance has been defined on E, it is possible to associate, with each set X, the subset X composed of all those points of X whose distance to the boundary is larger than. As increases, the subsets X are included within each other (and parallel in the Euclidean case). They can be considered as the horizontal thresholds of a function whose grey level is at point x if x is at distance  to the boundary. This function is called Distance Function.

Distance Function (II): an Example Set X Corresponding Distance Functions Comment : Figure b shows the supremum of both distance functions of X and X c

x y x y Topographic Image Representation The height in the topographic representation corresponds to the greylevel in the initial image.

Watershed Lines for Numerical Functions surface of the function Minima Catchment Basins Watershed Lines topographical patterns of a numerical image N.B. Beucher-Lantuejoul’s algorithm is presented below for the sake of pedagogy. But it is not the unique one, it has been improved by P. Soille and L. Vincent. Another implementation, based on hierarchical queues, due to F. Meyer is more efficient. If a drop of water falls on such a topographic surface, it will obey the law of gravitation and flow along the steepest slope path until it reaches a minimum. The whole set of points of the surface whose steepest slope paths reach a given minimum constitutes the catchment basin associated with this minimum. The watersheds are the zones dividing adjacent catchment basins. If a drop of water falls on such a topographic surface, it will obey the law of gravitation and flow along the steepest slope path until it reaches a minimum. The whole set of points of the surface whose steepest slope paths reach a given minimum constitutes the catchment basin associated with this minimum. The watersheds are the zones dividing adjacent catchment basins.

Suppose that holes are made in each local minimum and that the surface is flooded from these holes. Progressively, the water level will increase. In order to prevent the merging of water coming from two different holes, a dam is progressively built at each contact point. At the end, the union of all complete dams constitute the watersheds. Suppose that holes are made in each local minimum and that the surface is flooded from these holes. Progressively, the water level will increase. In order to prevent the merging of water coming from two different holes, a dam is progressively built at each contact point. At the end, the union of all complete dams constitute the watersheds. Construction of the Watersheds by Flooding (I) Minima Dams in construction Water level

Construction of the Watersheds by Flooding – Example

Construction of the Watersheds by Flooding (II) Flooding algorithm (S. Beucher, Ch. Lantuejoul) Let m be the minimum of function f. Put: X 0 = { x: f(x) = m}, X k = { x: f(x)  m+k } with 1  k  max f Denote by Y 1 the geodesic zones of influence of X 0 inside X 1. Distinguish three types of connected components of X 1 –those, X 1,1 that do not contain points of X 0 : then they do not belong to Y 1 –those, X 1,2 that contain a unique c.c. of X 0 : then they fully belong to Y 1 –those, X 1,3 that contain several c.c. of X 0 : Y 1 recovers then X 1,3 minus the branches of its geodesic SKIZ. Flooding algorithm (S. Beucher, Ch. Lantuejoul) Let m be the minimum of function f. Put: X 0 = { x: f(x) = m}, X k = { x: f(x)  m+k } with 1  k  max f Denote by Y 1 the geodesic zones of influence of X 0 inside X 1. Distinguish three types of connected components of X 1 –those, X 1,1 that do not contain points of X 0 : then they do not belong to Y 1 –those, X 1,2 that contain a unique c.c. of X 0 : then they fully belong to Y 1 –those, X 1,3 that contain several c.c. of X 0 : Y 1 recovers then X 1,3 minus the branches of its geodesic SKIZ. X 1.1 c.c. of X 0 X 1.3 X 1.2 c.c. of X 0 skiz ( X 0 \ X 1 ) Evolution of the flooding

Construction of the Watersheds by Flooding (III) Since the X 1,1 's are minima which appear at level 1, we have to incorporate them into the flooding process. Thus we replace X 1 by Y 1  X 1,1...and we iterate. The geodesic zones of influence –Y 2 of Y 1  X 1,1 inside X 2 are calculated; – They provide markers Y 2  X 2,1 ; etc... The process ends when level k = max (f) is reached. Then one has: Y max f = union of the basins; [ Y max f ] c = Watershed lines. Since the X 1,1 's are minima which appear at level 1, we have to incorporate them into the flooding process. Thus we replace X 1 by Y 1  X 1,1...and we iterate. The geodesic zones of influence –Y 2 of Y 1  X 1,1 inside X 2 are calculated; – They provide markers Y 2  X 2,1 ; etc... The process ends when level k = max (f) is reached. Then one has: Y max f = union of the basins; [ Y max f ] c = Watershed lines.

An example of Watershed by Flooding (I) Initial image. Minima (1), and next level (2). 2 1 Geodesic skiz of (1) into (2) (in white lines).

An example of Watershed by Flooding (II) Level 2, minus the first skiz, and level 3. Second skiz (note that it extends the first one). Final watershed (The result is significant in spite of the small number of gray levels).

Contents n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications n Introduction n Erosion and Dilation n Opening and Closing n Binary Geodesic Operators n Numerical Geodesic Operators n SKIZ and Watershed n Applications

Two Examples Electrophoresis: ( S. Beucher) 1- Delineate the spots, 2- Determine the neighbors. Highway: ( S. Beucher) Delineate the lanes.

Morphological Segmentation Paradigm ( S. Beucher, F. Meyer ) One, or more, images Pre-processing Markers M Function f synthetic function f' ( f, M ) Watershed of f' Possible Hierarchy Intelligent Phase Automatic Phase

Input Variables The watersheds extend the minima of an image as far as its topography allows it. One can act on these minima - either by means of filtering which removes some minima, - or by swamping, which impose their markers as new minima. One can also force the segmentation to pass along some portions of contours which are known a priori (the dotted lines on the road in example 2). Then the watersheds are conditioned by means of an input image where these portions are maxima. Finally, a first watershed can generate a new image, if we fill its divides with convenient values, and create a mosaic image. This new image is no longer based on pixels, but on the components of a planar graph. One can apply new watersheds or swampings, leading to a second mosaic image, etc. The watersheds extend the minima of an image as far as its topography allows it. One can act on these minima - either by means of filtering which removes some minima, - or by swamping, which impose their markers as new minima. One can also force the segmentation to pass along some portions of contours which are known a priori (the dotted lines on the road in example 2). Then the watersheds are conditioned by means of an input image where these portions are maxima. Finally, a first watershed can generate a new image, if we fill its divides with convenient values, and create a mosaic image. This new image is no longer based on pixels, but on the components of a planar graph. One can apply new watersheds or swampings, leading to a second mosaic image, etc.

Minima selection by Filtering As a general rule, images have too many minima, and a careless computation of their watersheds often leads to a disastrous over- segmentation. In order to obtain significant minima, one can begin with filtering the images: –either “horizontally” by plane alternating filters, with or without reconstruction ; –or “vertically” by closings R f *(f  h) of dynamic h. In particular, for h=1 all the minima are extracted. If dealing with maxima, one takes R f (f  h). As a general rule, images have too many minima, and a careless computation of their watersheds often leads to a disastrous over- segmentation. In order to obtain significant minima, one can begin with filtering the images: –either “horizontally” by plane alternating filters, with or without reconstruction ; –or “vertically” by closings R f *(f  h) of dynamic h. In particular, for h=1 all the minima are extracted. If dealing with maxima, one takes R f (f  h). “horizontal” filtering “vertical” filtering

Changing the Minima by Swamping The markers may not coincide with the minima of f. In that case, they act on f via the swamping transformation. The marker image is calculated as Two steps: –The point-wise minimum between the input image and the marker image is computed: (f + 1)  g Remark: f +1 is used to avoid the case where two minima to impose belong to the same minimum in f. –A reconstruction by erosion of (f + 1)  g from g is performed. R* [(f + 1)  g] (g) The markers may not coincide with the minima of f. In that case, they act on f via the swamping transformation. The marker image is calculated as Two steps: –The point-wise minimum between the input image and the marker image is computed: (f + 1)  g Remark: f +1 is used to avoid the case where two minima to impose belong to the same minimum in f. –A reconstruction by erosion of (f + 1)  g from g is performed. R* [(f + 1)  g] (g)

Changing the Minima by Swamping – Example

1st Example: Electrophoresis Gels (a): Initial image of electrophoresis (b): Watershed of the initial image ( Problem: to segment the spots, and to characterize the adjacency relations. The solution for this famous case study is due to S. Beucher and F. Meyer.

Electrophoresis: Minima ( c): Inital image minima ( d): Minima of the filtered image (e) Criticism: the over-segmentation comes from the excess of minima; so we will filter initial image by a  (opening followed by closing) of size 1 before computing the watershed.

Electrophoresis: zones of influence (e): Hexagonal alternating filter of (a) (f): Watershed of the filtered image (e) Criticism: the segmentation, now correct, provides the zones of influence of the spots, but not their contours. The latter should be derived from the watershed of the  gradient 

Electrophoresis: Watershed of the Gradient (I) (g): Modulus of the gradient of the filtered image ( h): Watershed of the gradient (g) Criticism: a few contours are revealed, blurred in a maze of over- segmentation. Since we know the significant minima of (e), we can introduce them, by swamping, as markers in the gradient image (g)

Electrophoresis: Watershed of the Gradient (II) (j): Watershed of (i) (i): Swamping of gradient (g) by the minima (d) Criticism: We forgot to mark that the gradient is zero not only at the centres of the spots, but also in the background. We must swamp (g) with the watershed (f).

Electrophoresis: Contours (l): Watershed of (k), superimposed on the initial image (k): (g) Swamped by the union of the minima (d) and of the watershed(f) Criticism: BRAVO !

Electrophoresis: Edge Effects Comment: We must make an assumption about the outside of the field. Up to now, we implicitly assumed it was white, i.e. lighter than all the pixels in the image. If, on the contrary, we take a black outside, then the graph no longer crosses the boundary of the field.

Lessons Drawn from the Example In a first analysis, over-segmentation may be corrected by filtering, but this approach can only suppress minima. If we want to add some new ones, or to move some of them, we have to apply swamping. An object is individualized when it has a unique minimum. The watershed lines of individualized objects delineate their zones of influence. Therefore they are exclusively located in the background, for which they can provide an ideal marker. For obtaining the contours of individualized objects, we must deal with the watershed of their gradients. The outside of the image has always to be chosen either as being a minimum or a maximum.....Finally, a number of situations are far from pertaining to that of the previous example, as we will see now. In a first analysis, over-segmentation may be corrected by filtering, but this approach can only suppress minima. If we want to add some new ones, or to move some of them, we have to apply swamping. An object is individualized when it has a unique minimum. The watershed lines of individualized objects delineate their zones of influence. Therefore they are exclusively located in the background, for which they can provide an ideal marker. For obtaining the contours of individualized objects, we must deal with the watershed of their gradients. The outside of the image has always to be chosen either as being a minimum or a maximum.....Finally, a number of situations are far from pertaining to that of the previous example, as we will see now.

Segmentation of Road Lanes (I) Problem: Delineate the traffic lanes of a road, from a sequence of 450 views taken from a fixed camera with a high angle shot. Differences: Unlike the previous case: - we now start from a sequence of images. How to synthesize information? - The segmentation holds on the road only, without any distinction between foreground and background. Then, how do we build up markers? - The watershed must pass through the white dotted lines. How can we introduce this constraint? Problem: Delineate the traffic lanes of a road, from a sequence of 450 views taken from a fixed camera with a high angle shot. Differences: Unlike the previous case: - we now start from a sequence of images. How to synthesize information? - The segmentation holds on the road only, without any distinction between foreground and background. Then, how do we build up markers? - The watershed must pass through the white dotted lines. How can we introduce this constraint? (a): Image extracted from the initial sequence.

Road (II) : Synthetic Images (a): Sum  ( f i, 1  i  450 )  450 (b): Sum  (  f i+1  f i , 1  i  449 )  449 Comment: We summarized the whole useful information in two images (instead of 450 ). One of them emphasises the static portions, and the other the moving ones.

Road (III) : Markers (c): thresholded version of (b) (d): dilation of (c) by a horizontal segment whose length increases as it moves down the image. (e): difference between (c) and (d). In white, the four zones to thin. Comment: image (b), under threshold, provides a first set of markers (c); it is completed by the complement (d) of the road, which marks the outside (e).

Road (IV) : Conditioning Comment: the condition of passing through the dotted lines demands we build up a function in which these dotted lines are maxima. It results in variants (g) and (h). (f): top hat of image (a). (g): 3 level function, derived from (e) and from the filtered restriction of (f) to the central stripes of (c) ( h): (inversed) geodesic distance function of the higher level of (g) inside its median level. The two stripes without dotted white lines have been added.

Road (V) : Results (i): Watershed of the three level function (g) Comment: OK for (j). The three level function is also not such a bad starting point. In both cases, the digitalization of the watershed creates slight variations from the dotted lines. (j): Watershed of the geodesic distance function

Lessons Drawn from the Example For imposing some points or lines on a watershed, one must introduce them as maxima in the source image. This goal can be achieved by taking their geodesic distance function with respect to a significant zone of the image. Such an image synthesis may viewed as complementary to the swamping procedure, which holds on the minima. In the limit, it is possible to concentrate information in a three level synthetic image based upon the two following conditioning sets - low level: markers of the minima, - high level: arcs which are imposed to the watershed. The median level is a constant value given to the rest of the input function. Finally, if the upper level is missing, the watershed comes back to a SKIZ of the minima. In sequences taken by a fixed camera without zoom, the first two abstracts of the information are the means of the images and of their time gradients. For imposing some points or lines on a watershed, one must introduce them as maxima in the source image. This goal can be achieved by taking their geodesic distance function with respect to a significant zone of the image. Such an image synthesis may viewed as complementary to the swamping procedure, which holds on the minima. In the limit, it is possible to concentrate information in a three level synthetic image based upon the two following conditioning sets - low level: markers of the minima, - high level: arcs which are imposed to the watershed. The median level is a constant value given to the rest of the input function. Finally, if the upper level is missing, the watershed comes back to a SKIZ of the minima. In sequences taken by a fixed camera without zoom, the first two abstracts of the information are the means of the images and of their time gradients.

Summary You have seen –Some of the basic operations of mathematical morphology. –Some examples of where they are useful. Many more exist, which haven’t been covered: Leveling, Granulometry, Morphology for graph representations, etc. Applying these operations to colour images is more complicated, as will be discussed in my next talk. You have seen –Some of the basic operations of mathematical morphology. –Some examples of where they are useful. Many more exist, which haven’t been covered: Leveling, Granulometry, Morphology for graph representations, etc. Applying these operations to colour images is more complicated, as will be discussed in my next talk.

Acknowledgements This lecture is based on: –Jean Serra’s one week course on Mathematical Morphology held at the Paris School of Mines (it’s part of the GEI-Athens European courses, so it’s most probably possible to get money from the EU to attend it). –Pierre Soille’s book. This lecture is based on: –Jean Serra’s one week course on Mathematical Morphology held at the Paris School of Mines (it’s part of the GEI-Athens European courses, so it’s most probably possible to get money from the EU to attend it). –Pierre Soille’s book.