1. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Vito Di Gesù, Giosuè Lo Bosco DMA – University of Palermo, ITALY digesu@math.unipa.it THE COST-TIST 283 Image Segmentation based on Genetic Algorithms Combination

2. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Introduction - Supervised Global Segmentation (SGS) - Unsupervised Tree Segmentation (UTS) The image segmentation problem as a GOP (Global Optimization Problem) Combined strategies

3. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Related works Shi, Malik, Normalized Cuts and Image Segmentation, 2000. V.Di Gesù A Clustering Approach to Texture Classification, 1988. Jain and Flynn, Image Segmentation Using Clustering, 1996, Ridder, Kittler, Lemmers, an Duin. The adaptive subspace map for texture segmentation, 2000..

4. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Combined Genetic Segmentation (CGS) Unsupervised Tree Segmentation Supervised Global Segmentation Maximal Connected Components Relaxation procedure

5. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Graphs and perception G   is a distance (similarity) function:  (x,y) x y

6. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Image Segmentation and Graph Partitioning Problem Input: A (weighted) graph G=(V,E  Integers j, k, and m. Problem: Partition the vertices into m subsets such that each subset has size at most j, while the cost of the edges spanning subsets is bounded by k. a b c d e

7. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Bipartition partition The optimal bi-partition is the one that minimize (  similarity function) or maximize (  distance function) Problem: disjoin A and B removing edges connecting the two parts. The cut of A and B is defined: AB

8. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 A weighted graph G is associated to the image X A pixel x  X is represented with (i x, j x, g x )

9. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Supervised Global Segmentation (SGS) P={p 1, p 2,...,p k } partition

10. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 SGS Algorithm Procedure SGS (X,K max ) choose at random p k, k=1,2 …., K max classes; repeat for x  X if then update (  k,  k ) compute until (F reaches the minimum) assign (x,p k ) end Genetic computation Fitness function Optimization

11. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Unsupervised Tree Segmentation (UTS)

12. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 UTS Algorithm Procedure UTS (A) if not(uniform (A)) then (A l, A r )  SGS(A,2); UTS(A l ); UTS(A r ); else return (A); end The function uniform(A) returns the growing condition and it is based un a uniformity test.

13. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 G.A. terminology Population: set of individuals named chromosome Chromosome: sequence of genes. ABCDABCD Code symbols Coded information

14. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Crossover operator with probability qCut point random

15. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Mutation operator Binary alphabet: with probability p  q Mutation point random

16. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Clonation To strength the survival of parents features in the chromosome population

17. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Evolutionary computing (EC) EC are optimization procedures in the space o events Fitness function The fitness function depends on the problem to be solved The goal of EC is to maximize the fitness function

18. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002

19. Genetic Solution for the segmentation problem GA Data coding : The generic pixel x is coded by a 32 bit binary string that codes the pixel-label, x in the 8 less significant bits and the pixel position (i x,j x ) in the 24 most significant bits. Here, x identifies the cluster to which the pixel belongs. k x =i x *m+j x and K is the maximum number of clusters.

20. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Genetic Solution Fitness Function : the inverse function of L and S, L -1 (  and S -1 (  return the label  L -1 (  of a pixel in position (i,j)= S -1 (  The fitness function f is defined on the basis of the similarity function computed between a given chromosome a and the corresponding segment P

21. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Genetic Solution Genetic operator  : the application of the classical single point crossover and the bit mutation. Selection process :

22. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Genetic Solution Halting Condition : total variance

23. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Genetic Algorithm 1. (Input) - Read image X of size nxm; 2. (Initial condition) - Set up a population of chromosomes and assign at random a label to each  i (0); 3. (Genetic process) - Apply the genetic operators (sinlge point crossover and bit mutation) to current population P(t); 4. ( Selection process) - Build population P(t+1) choosing by selecting the best chromosome from P(t) and  ( P(t)); 5. ( Set iteration) - t  t + 1; 6. (Halting condition) – if |Var t-1 - Var t |  goto 3; else stop.

24. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Convergence of CGS

25. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Maximal connected component (MCC)

26. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Experimental result on syntetic images

27. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Experimental result and comparison The results of the application of the CGS on real data is compared with three methods : C-means (Bezdek, 1981) Single-Link (EPRI, 1999) Graph partition segmentation (Malik, 2000)

28. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Normalized cut criterion Shi, Malik 1999 Min-cut procedure

29. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Normalized cut criterion A partition of the image into regions such that there is high similarity within a region and low similarity across regions is obtained by solving a generalized eigenvalue problem. Minimizing normalized CUT is NP-Complete even for graph on grid (Papadimitriou 1999) The resulting eigenvectors provide a hierarchical partitioning of the image into regions ordered according to salience. Brightness, color, texture, motion similarity, proximity and good continuation can all be encoded into this framework.

30. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 1.From an image X built G=(V,E  x 2.Solve (D-W)x= Dx for eigenvectors with the smallest eigenvalues. 3.Use the eigenvector with the second smallest eigenvalue to bipartition the graph. 4.Decide if the current partition should be subdivided and recursively repartition the segmented parts if necessary. The grouping algorithm 1.G is only locally connected  the eigensystem is sparse 2.Only the top few eigenvectors are needed. 3.The precision requirement is low  Lanczos method Time complexityn=number of nodes Time complexitywhere: m=maximum number of matrix-vector computations M(n)= the cost of a matrix-vector computations

31. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Experimental result and comparison Corel Image Database http://elib.cs.berkeley.edu/photos/corel Range Images http://marathon.csee.usf.edu/range/DataBase.html Astronomical images Miscellanea

32. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 CGS C-meansSingle-link GPS Human Corel Image Database

33. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 CGS C-means Single-link GPS Human

34. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Range images CGS YAR http://marathon.csee.usf.edu/

35. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 CGSYAR

36. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Images from astronomy

37. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Images from astronomy

38. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Evaluation of CGS The comparison has been performed between the automatic segmentation and the segmentation deriving from the evaluation of an odd number (5) of persons. Seg k denotes the k-th segment retrieved by humans S denotes the k-th segment retrieved by the machine | Seg k | and |S| denote the corresponding size #agr k is the largest pixel intersection between Seg k and S.

39. iAstro/IDHA Workshop Strasbourg Observatory 28-29 November 2002 Comparision The CPU times are referred to an INTEL PENTIUM III 1GHz.