1. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs1 Vic Ciesielski and Xiang Li {vc, xiali}@cs.rmit.edu.au School of Computer Science and Information Technology RMIT University, Australia

2. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs2 Overview Introduction -Brief explain why we analyze runs -Our research questions Problems and their backgrounds Methodology -What information we looked into in detail. -How we measure them. Results –Duplicate evaluations, Tree shapes, Program sizes and depth changes along with generations. Conclusion –What we have learned from the runs, what their implications are and what we can do to improve the genetic programming process.

3. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs3 Introduction Analyzing GP runs is important –Help to understand the problem in depth; Find important patterns; Improve the search techniques Eg. MAX by Gathorcole and Ross, 1996 ; Langdon and Poli, 1997 Eg. Fitness Landscape by Kinnear, Jr. 1994; Ant by Langdon and Poli, 1998 Previous works have limitations –Only one or two problems –Only limited aspects of the problems, eg. convergence, bloat, fitness landscape, tree shapes (limited) Research questions –How many duplicate individuals are evaluated in a run? –What percentage of possible tree shapes are evaluated? –Are there any patterns in the size and depth of the trees examined? –Are there any differences between the toy and the real world problems? –Are any improvements to the GP process suggested by the discovered patterns?

4. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs4 Problems & Backgrounds & Genetic Variables Settings ‘ Toy’ Problems ( 9) –Santa Fe Ant ( Langdon and Poli, 1998 ), Pop. Size : 500, Max. Depth 5 –Modified Santa Fe Ant ( Ciesielski and Li, 2004 ), Pop. Size : 100, Max. Depth 5 –Lawnmower ( Koza, 1992 ), Pop Size : 100, Max. Depth 7 –MAX ( Gathercole and Ross, 1996; Langdon and Poli, 1997 ), Pop. Size 100, Max. Depth 5 –Symbolic Regression ( Koza, 1992 ), Pop. Size : 100, Max. Depth 7 –5 Even Parity ( Koza, 1992 ), Pop. Size : 500, Max. Depth 8 –Binary Image Classification 1 ( Li and Ciesielski, 2004 ) –Binary Image Classification 2, Pop. Size : 100, Max. Depth 7 –Soccer Goal Scoring ( Bajurnow and Ciesielski, 2004 ), Pop. Size : 100, Max. Depth 5 Real world problems (3) –Cephalometric Landmarks ( Ciesielski et al., 2003 ), Pop. Size : 100, Max Depth 9 –Texture Classification ( Song and Ciesielski, 2004 ), Pop. Size : 500, Max. Depth 14, Elitism 10%, Crossover 85%, Mutation 5%, Max. Gen. 50 –Evolution of Texture Features ( Lam Ciesielski, 2004 ), Pop. Size : 100, Max. Depth 7 Common Genetic Variables Settings unless specified –Elitism Rate : 2% Crossover Rate : 70% Mutation Rate : 28% Max. Gen. 100

5. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs5 Methodology GP Environments –RMIT-GP (http://www.cs.rmit.edu.au/~vc)http://www.cs.rmit.edu.au/~vc –Strong typed –Ramped half and half method for initialization Duplications are found, but hard to determine, because –Exactly identical –Logically equivalent, eg (+ A 1) vs. (+ A (/ B B)) –Commutative equivalent, eg (+ A B) vs. (+ B A)

6. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs6 Measures Number of fitness evaluations Number of tree shapes –Converting programs into shapes (* (+ (/ 24 X) (+ Y 81)) (+ (* X 34) (+ 86 44)) Translated into  (((##)(##))((##)(##)) –Determine Total number of possible trees Most of GP problem examined use binary trees Depth= { 1, 2, 3, 4, 5, 6, 7, …} N o.of.Possible.Shapes ={ 1, 3, 21, 651, 457653, 2.10E+11, 4.4E22, …} More shapes for ternary and grows exponentially with tree depth Number of commutative-distinct individuals evaluated Number of string-distinct individuals evaluated Possible Tree ShapesInvalid Tree Shapes (a)(b)(c)(d)

7. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs7 Results - 1 Run

8. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs8 Results - 5 Runs Comments : We understand 5 runs are still not enough, but at least they repeatedly demonstrate the same pattern. Duplications exist and they are not in a small amount.

9. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs9 Tree Shapes - 5 Runs Comments : Even 5 runs, the number of distinct of shapes is still trivial compared with possible shapes besides 1 runs.

10. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs10 Visualizations A “Toy”Problem - MAX

11. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs11 A Real World Problem Texture Features

12. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs12 Conclusion Number of duplicate individuals –There are a lot than expected. Some are understandable like MAX problem, some need more analysis Percentage of possible tree shapes –Only a very tiny percentage of the possible shapes were examined Patterns in size and depth –Fitness vs. generation vs. size follows a roughly triangular pattern Toy vs. Real World Problems –There are no clear differences. There are many duplications in real world and many single node trees were evaluated. Suggested Improvements Caching the string representation of the programs and reuse the fitness values when evaluations are expensive.

13. RMIT UNIVERSITY ASPGPAnalysis of Genetic Programming Runs13 Questions & Suggestions Acknowledgement: This work was partially supported by grant EPPNRM054 from the Victorian Partnership for Advanced Computing. Thanks for people in ECML group, Andy, Andrew, Brian, Teja for their providing some runs.