1. Genetic Programming and LIL-GP
Problem Set 4
Genetic Programming and LIL-GP

2. Where we’ve been
Problem Sets 1 and 2: Theory

3. Where we’ve been
Problem Sets 1 and 2: Theory
Combinatorics

4. Where we’ve been Problem Sets 1 and 2: Theory Combinatorics
Simple Genetic Algorithms (a.k.a. How to run an effective hamburger chain)

5. Where we’ve been Problem Sets 1 and 2: Theory Combinatorics
Simple Genetic Algorithms (a.k.a. How to run an effective hamburger chain)
Fundamental Theorem of Genetic Algorithms

6. Where we’ve been Problem Sets 1 and 2: Theory Combinatorics
Simple Genetic Algorithms (a.k.a. How to run an effective hamburger chain)
Fundamental Theorem of Genetic Algorithms
Hyperplane madness

7. Where we’ve been
Problem Set 3: Genetic Algorithms and the GENESIS system

8. Where we’ve been
Problem Set 3: Genetic Algorithms and the GENESIS system
Optimization of the function x10

9. Where we’ve been
Problem Set 3: Genetic Algorithms and the GENESIS system
Optimization of the function x10
Minimization of the De Jong function

10. Where we’re going
Problem Set 4:
Genetic Programming and LIL-GP

11. Where we’re going Problem Set 4: Genetic Programming and LIL-GP
Evolve LISP-like programs without writing one line of LISP! (sorry, John)

12. Where we’re going Problem Set 4: Genetic Programming and LIL-GP
Evolve LISP-like programs without writing one line of LISP! (sorry, John)
Solve two simple problems: symbolic regression of a quadratic function, and symbolic regression of a boolean even-3-parity function.

13. Getting the software
Change to your cs426 directory. cd cs426

14. Getting the software Change to your cs426 directory. cd cs426
Copy the software to your AFS space. cp –r /afs/ir/class/cs426/lilgp . This will create a directory called lilgp under your cs426 directory.

15. Uncompressing the code
Change to your new lilgp directory. cd lilgp You should have one file: lilgp.tar

16. Uncompressing the code
Change to your new lilgp directory. cd lilgp You should have one file: lilgp.tar
Uncompress with the following command: tar –xvf lilgp.tar

17. Uncompressing the code
You should now have a directory called lilgp1.1. Under this directory you only need to deal with the following subdirectories:
1.1: All the code.
htmlMan: All the documentation.
Don’t worry about the others.

18. Where to start
Browse through the documentation. It is indexed and easy to follow. If you want to download it yourself, you can get it at:
Start with: lil-gp.contents.htm

19. Where to start
Try running one of the samples. They’re located in the app directory. After compiling and building (just type make), they’re ready to run. Try the regression sample. To run, type: gp –f input.file If the program does not start running, there’s something wrong with your installation.

20. Warnings:
Don’t change anything in the kernel directory.

21. Warnings:
Don’t change anything in the kernel directory.

22. Warnings: Don’t change anything in the kernel directory.
Don’t modify the makefiles. Even though they’re called GNUmakefile, typing make still works.

23. Implementing a problem
Fill in a tableau, deciding on terminal sets, function sets, fitness determination, etc.

24. Implementing a problem
Fill in a tableau, deciding on terminal sets, function sets, fitness determination, etc.
Write the code.

25. Implementing a problem
Fill in a tableau, deciding on terminal sets, function sets, fitness determination, etc.
Write the code.
Create a parameter file for the run.

26. Implementing a problem
Fill in a tableau, deciding on terminal sets, function sets, fitness determination, etc.
Write the code.
Create a parameter file for the run.
Run the code and examine the output files.

27. Tableau
Terminal set: The leaves of the expression tree.

28. Tableau Terminal set: The leaves of the expression tree.
Function set: The internal nodes.

29. Tableau Terminal set: The leaves of the expression tree.
Function set: The internal nodes.
ex: symbolic regression terminal set = {x} function set = {+, -, *, %}

30. Tableau
Fitness
Raw fitness – sum of error over fitness cases, number of hits, etc.

31. Tableau
Fitness
Raw fitness – sum of error over fitness cases, number of hits, etc.
Standardized fitness – all positive; 0 is best

32. Tableau
Fitness
Raw fitness – sum of error over fitness cases, number of hits, etc.
Standardized fitness – all positive; 0 is best
Adjusted fitness – in between 0 and 1, with 1 being the most fit individual

33. Tableau
In general, LIL-GP was designed after what is described in Genetic Programming (Koza, 1992), so all terminology is described in the book. See chapters 6 and 7 for full explanations of all terms used in the program.

34. Writing the code Files you need to provide: app.h app.c appdef.h
function.h
function.c

35. Writing the code Files you need to provide:
app.h – Defines the global data structure used to pass information back and forth between files.
app.c – Defines all the user callbacks – initialization, fitness evaluation, etc.

36. Writing the code Files you need to provide:
appdef.h – Two #defines: MAX_ARGS, and DATATYPE.
MAX_ARGS: Maximum number of arguments your functions will take
DATATYPE: Type that all of your functions will return (ex: double, int, etc.)

37. Writing the code Files you need to provide:
function.h – Prototypes for all of the functions in your function set.
function.c – Implementations of the functions listed in function.h.

38. Writing the code
In general, all of the tricky code has been written for you. In fact, two of the sample applications are:
Symbolic regression of a function
Boolean-11 multiplexer
Both are explained in GP. You do not need to rewrite the functions in app.c, just modify them.

39. Creating a parameter file
The parameter files included with the samples are called input.file. Just modify these for the homework problems. If you want to learn more about the extensive parameters that can be set, look in the documentation.

40. Creating a parameter file
The only confusion about parameter files will come in specifying crossover rates for the first problem. To avoid this confusion, here is what you need to put in your parameter file to meet the problem criteria:

41. Creating a parameter file
Parameters for proper breeding:
breed_phases = 4
breed[1].operator = crossover, select=fitness, internal=0.0
breed[1].rate = 0.1
breed[2].operator = crossover, select=fitness, internal=1.0
breed[2].rate = 0.8

42. Creating a parameter file
Parameters for proper breeding:
breed[3].operator = reproduction, select=fitness
breed[3].rate = 0.1
breed[4].operator = mutation, select=fitness
breed[4].rate = 0.00

43. Creating a parameter file
Aside from this, all you need to change is pop_size, max_generations, random_seed, and perhaps output.basename. The parameter output.basename sets the file prefix for all of the output files.

44. Creating a parameter file
For problems 2 and 3, the default crossover parameters work fine. No need to change them unless you’re curious.

45. Running the code Very simple to run the code:
Compile and build with the command make. If there are compile errors, it will tell you now – fix them before continuing.

46. Running the code Very simple to run the code:
Compile and build with the command make. If there are compile errors, it will tell you now – fix them before continuing.
Run the code with the command: gp –f input.file

47. Output files LIL-GP generates lots of good stuff:
.sys – general info about the run
.gen – stats on tree size and depth
.prg – stats on fitness and hits
.bst – info about current best individual
.his – history of the .bst file
.stt – unreadable version of all stats
You will want the .bst and .his files.

48. Problem 1
This problem deals with symbolic regression of the function x2/2+2x+2. There is a sample app called “regression” provided. All you need to do is modify the input.file as described, delete a bunch of lines in app.c and function.h/c, change a few numbers in app.c, and it will run. Guaranteed.

49. Problem 1
Hint: With the provided random seed, it will not find a close solution even after 151 generations. Fiddle with the random seed and you will quickly find a perfect solution in much fewer than 151 generations.

50. Problem 2
This problem deals with symbolic regression of a boolean function that performs even-3-parity. Even-3-parity returns true given three inputs if an even number of inputs are true (i.e., 0 true, or 2 true).

51. Problem 2
There is a sample app called “multiplexer” that performs symbolic regression for a boolean-11 multiplexer. Just modify their files to do this problem. You will need to delete a bunch of functions and add code for nand and nor. You might want to add an even-3-parity test function in app.c for fitness determination….

52. Problem 2
You will also need to change the specification in app.h. Your global should either have three ints (1 for each input line) or 1 int (if you like bit logic).
Hint: With a specified population size of 1000, you will very likely have a perfect individual at generation 0, since the solution is very simple. This is fine. You should be happy about it, not stressed.

53. Problem 3
This is the same as problem 2, except now with an automatically defined function. Do not attempt this problem before you have problem 2 working completely. You will be able to quickly modify your problem 2 code (just app.c) to make problem 3 work.

54. Problem 3
In specifying the function sets for this problem, follow the model given in the “lawnmower” sample app. This shows precisely how to build the function sets for a tree with automatically defined functions. There will be two different function sets: one of the result producing branch, and one for the ADF branch.

55. General Hints
Once again, there is not much code to be written. All you are doing is modifying existing code. Don’t feel bad, this is done all the time.

56. General Hints
Once again, there is not much code to be written. All you are doing is modifying existing code. Don’t feel bad, this is done all the time.
For the function sets, make sure to read the docs to find out what each of the arguments should be for the different types of arguments, terminals, and functions.

57. General Hints
Don’t modify the sample apps directly. Copy the files to a new directory, so you’ll always have the original to look back at.

58. General Hints
Don’t modify the sample apps directly. Copy the files to a new directory, so you’ll always have the original to look back at.
If you get stuck, read the documentation. It’s actually very clear and well-indexed.

59. What to turn in Answers to the written problems.
app.c and function.c for each problem.
The .bst file for each problem. The default of one individual in this file is fine.

60. Final thoughts
This is not a test of your programming ability. It is just an introduction to one piece of easy-to-use software for genetic programming. Don’t make this harder than it needs to be by rewriting everything from scratch. Use what works.

61. Final thoughts Other GP software freely available:
GPCPP – Has basic functionality, relatively easy to add new problem classes. (C++) Download at:

62. Final thoughts Other GP software freely available:
GPQUICK – Simpler than GPCPP but less friendly interface. (C++) Download at:

63. Final thoughts Other GP software freely available:
ECJ8 – Most versatile, but very large and somewhat complicated. (Java) Download at:

64. Final thoughts
Good luck!