1. Segmentation via Genetic Programming

2. Segmentation segments  Splitting an image into segments §Hard problem ill-defined  Conceptually ill-defined Segmentation

3. Segmentation Which segmentation is the correct one?

4. Segmentation

5. Segmentation

6. Segmentation §Tagging function: §Deciding function: Defining the problem

7. Segmentation §Berkley’s segmentation dataset and benchmark: Images + Human made segmentation maps Benchmark for segmentation algorithms The Dataset

8. Genetic Programming  Bio-Inspired  Bio-Inspired Artificial Intelligence method §Inspired by Darwin’s evolutionary principles What is Genetic Programming?

9. Genetic Programming  Variety  Variety of species individuals within the population  Competition  Competition for limited resources  Overproduction  Overproduction of offspring generation §Survival of the fittest Origin of Species, 1859 Darwin’s principles

10. Genetic Programming The computational model GnGnGnGn 55 44 12 31 95 32 87 12 0 65 53 2 91 73 + G n+1 = crossover mutation fitness

11. Segmentation via GP §Individuals are represented as LISP-like functions X2  (* x x) 2x-1  (- (* (+ 1 1) x) 1) Individual Representation

12. Segmentation via GP §Equivalent to tree representation Individual Representation

13. Segmentation via GP §Crossover Genetic Operators

14. Segmentation via GP §Mutation Genetic Operators

15. Segmentation via GP §Function Set: {+, -, *, %, neg, conv, opp, sqrt} §Terminal Set: {image, 0, 1, const, gradx, grady, ckernel} §Strongly-Typed GP: conv(matrix,kernel) +(matrix,matrix), +(kernel,kernel), +(kernel,number), … Building the trees

16. Segmentation via GP §An example: Gradient Magnitude (sqrt (+ (* (conv image gradx) (conv image gradx)) (* (conv image grady) (conv image grady)))) Building the trees

17. Segmentation via GP §Accuracy: §Modified Accuracy: Fitness measure

18. Segmentation via GP §Individuals produce ‘soft boundary maps’, need threshold value §Value too low – more points are reported (false positives) §Value too high – less points are reported (true negatives) Threshold

19. Segmentation via GP equal  Berkley’s benchmark: split the threshold range into N equal parts, threshold and keep best. unequal  My method: split the threshold range into N unequal parts, by equal increase in reported points, proportional to number of points in the human made segmentation map. Threshold

20. Segmentation via GP §Population size: 60-150 §Generation count: unspecified §Crossover rate: 90% §Mutation rate: 10% §Selection: tournament of 3 §Tree depth: limited to 6-9 Sean Luke ’ s ECJ13 was used for running GP sessions Miscellaneous Evolutionary Parameters

21. Results A typical GP session

22. Results §Generation 0: 0.176 best, 0.08 average §Generation 93: 0.262 best, 0.242 average §Fitness does not always increase! §Fitness leaps in best individual A typical GP session

23. Results Segmentation function (- (- (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 5.381114 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 -9.389916 6.183886 - 7.817788) grady) (- (kernel -2.334486 -4.6182337 - 9.115009 8.010966 -3.0507333 3.22619 2.068446 - 2.932576 -6.243905) 0.0))) (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 2.4412537 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 - 9.389916 6.183886 -7.817788) grady) (- (kernel 9.936699 -4.6182337 -9.115009 8.010966 -3.0507333 3.22619 2.068446 -2.932576 -6.243905) 0.0)))) (- (- (- (conv image grady) (* 1.0 9.336973)) (% (* 1.0 9.336973) 9.336973)) (% (% (* -3.9138038 0.0) (* 0.0 0.0)) (% (* 1.0 9.336973) (* 1.0 9.336973))))) Best Individual

24. Results Segmentation function gradx gradx grady gradxgradx grady grady (- (- (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 5.381114 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 -9.389916 6.183886 - 7.817788) grady) (- (kernel -2.334486 -4.6182337 - 9.115009 8.010966 -3.0507333 3.22619 2.068446 - 2.932576 -6.243905) 0.0))) (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 2.4412537 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 - 9.389916 6.183886 -7.817788) grady) (- (kernel 9.936699 -4.6182337 -9.115009 8.010966 -3.0507333 3.22619 2.068446 -2.932576 -6.243905) 0.0)))) (- (- (- (conv image grady) (* 1.0 9.336973)) (% (* 1.0 9.336973) 9.336973)) (% (% (* -3.9138038 0.0) (* 0.0 0.0)) (% (* 1.0 9.336973) (* 1.0 9.336973))))) Best Individual

25. Results Segmentation function (- (- (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 5.381114 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 -9.389916 6.183886 - 7.817788) grady) (- (kernel -2.334486 -4.6182337 - 9.115009 8.010966 -3.0507333 3.22619 2.068446 - 2.932576 -6.243905) 0.0))) (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 2.4412537 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 - 9.389916 6.183886 -7.817788) grady) (- (kernel 9.936699 -4.6182337 -9.115009 8.010966 -3.0507333 3.22619 2.068446 -2.932576 -6.243905) 0.0)))) (- (- (- (conv image grady) (* 1.0 9.336973)) (% (* 1.0 9.336973) 9.336973)) (% (% (* -3.9138038 0.0) (* 0.0 0.0)) (% (* 1.0 9.336973) (* 1.0 9.336973))))) Best Individual

26. Results Segmentation function (* (conv image gradx) (conv image gradx)) (* (conv image gradx) (conv image gradx)) (- (- (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 5.381114 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 -9.389916 6.183886 - 7.817788) grady) (- (kernel -2.334486 -4.6182337 - 9.115009 8.010966 -3.0507333 3.22619 2.068446 - 2.932576 -6.243905) 0.0))) (conv (* (conv image gradx) (conv image gradx)) (- (- (kernel 2.4412537 -8.362269 8.888325 1.1866289 -6.4069843 -8.251046 - 9.389916 6.183886 -7.817788) grady) (- (kernel 9.936699 -4.6182337 -9.115009 8.010966 -3.0507333 3.22619 2.068446 -2.932576 -6.243905) 0.0)))) (- (- (- (conv image grady) (* 1.0 9.336973)) (% (* 1.0 9.336973) 9.336973)) (% (% (* -3.9138038 0.0) (* 0.0 0.0)) (% (* 1.0 9.336973) (* 1.0 9.336973))))) Best Individual

27. Results Accuracy = 0.307 (GM accuracy = 0.280) Best Individual

28. Results Accuracy = 0.172 (GM accuracy = 0.193) Best Individual

29. Results Accuracy = 0.262 (GM accuracy = 0.245) Best Individual

30. Results Accuracy = 0.126 (GM accuracy = 0.119) Best Individual

31. Summary §It is possible to evolve segmentation functions using GP §Results are good, can be better §Evolved function ‘beats’ the gradient magnitude function §Improve results by more power Discussion

32. Summary §More CPU and RAM §More functions and terminals §Use ADFs §Evolve kernels separately §Evolve threshold function separately §Include more inputs: color images, textures, output of other edge detectors Future Work

33. References 1.The Berkley Segmentation Dataset and Benchmark http://www.cs.berkeley.edu/projects/vision/grouping/segbench/ 2.Koza, J. R.: Genetic Programming: On the programming of computers by natural selection. MIT press, Cambridge, Mass. (1992) 3.Tomassini M.: Evolutionary Algorithms. Swiss Scientific Computing Center, Manno. 4.Darwin, Charles: On the origin of species by means of natural selection. London, John Murray, 1859 5.Montana, D.J.: Strongly typed genetic programming. Evolutionary Computation 3 (1995) 199–230 6.Langdon, W.B.: Size fair and homologous tree genetic programming crossovers. Genetic Programming and Evolvable Machines 1 (2000) 95–119. 7.Luke S.: ECJ 13 - a Java based Evolutionary Computation and Genetic Programming research system http://cs.gmu.edu/~eclab/projects/ecj 8.Koza, J. R.: Genetic Programming II: Automatic Discovery of Reusable Programs. MIT press, Cambridge, Mass. (1994)