G ENETIC P ROGRAMMING Ranga Rodrigo March 17,

I NTRODUCTION Viewed by many researchers as a specialization of GA. Where GAs use string (or vector) representations, GP uses a tree representation. Originally, GP was developed by Koza to evolve computer programs. For each generation, each evolved program (individual) is executed to measure its performance within the problem domain. The result obtained from the evolved computer program quantifies the fitness of that program. 2

I MPLICATIONS OF T REE -B ASED R EPRESENTATION Adaptive individuals: – In Gas the size of individuals are fixed, – A GP population will usually have individuals of different size, shape and complexity. – Size: tree depth – Shape: branching factor of nodes in the tree. – The size and shape of a specific individual too may change due to application of the reproduction operators. Domain-specific grammar – A grammar needs to be defined that accurately reflects the problem to be solved. – It should be possible to represent any possible solution using the defined grammar. 3

G RAMMAR FOR C HROMOSOME R EPRESENTATION A grammar forms an important part of chromosome representation. As part of the grammar, a terminal set, function set, and semantic rules need to be defined. – The terminal set specifies all the variables and constants – The function set contains all the functions that can be applied to the elements of the terminal set. – These functions may include mathematical, arithmetic and/or Boolean functions. Decision structures such as if- then-else and loops can also be included in the function set. 4

G RAMMAR IN T REE T ERMINOLOGY Using tree terminology, elements of the terminal set form the leaf nodes of the evolved tree, and elements of the function set form the non-leaf nodes. For a specific problem, the search space consists of the set of all possible trees that can be constructed using the defined grammar. 5

E XAMPLE : XOR x1x1 x2x2 Target OP (x 1 AND NOT x 2 ) OR (NOT x 1 AND x 2 ) OR AND x2x2 NOT x1x1 x2x2 x1x1 Function set: {AND, OR, NOT} Terminal set: {x 1, x 2 }

E XAMPLE : M ATHEMATICAL E XPRESSION 7 Y = x*ln(a)+sin(z)/exp(-x) – *- a ln / x3.5 z sin x exp - Function set: {-, +, *, /, sin, exp, ln} Terminal set: {a, x, z, 3.5}

I NITIAL P OPULATION The initial population is generated randomly within the restrictions of – a maximum depth and – semantics as expressed by the given grammar. The branching factor of the root, and each non- terminal node are determined by the arity of the selected function. For each non-root node, the initialization algorithm randomly selects an element either from the terminal set or the function set. When an element from the terminal set is selected, the particular node becomes a leaf node. 8

F ITNESS F UNCTION Because individuals usually represent a program, calculation of fitness requires the program to be evaluated against a number of test cases. E.g.: – For the Boolean expression, fitness is calculated as the number of correctly predicted target outputs. – For the mathematical expression a data set of sample input patterns and associated target output is needed. 9

C ROSSOVER O PERATORS Generating one offspring: – A random node is selected within each of the parents. Crossover then proceeds by replacing the corresponding subtree in the one parent by that of the other parent. Generating two offspring: – A random node is selected in each of the two parents. – The corresponding subtrees are swapped to create the two offspring 10

O NE O FFSPRING 11

T WO O FFSPRING 12

M UTATION O PERATORS Function node mutation: – A non-terminal node, or function node, is randomly selected and replaced with a node of the same arity, randomly selected from the function set. Terminal node mutation: – A leaf node, or terminal node, is randomly selected and replaced with a new terminal node, also randomly selected from the terminal set. Swap mutation: – A function node is randomly selected and the arguments of that node are swapped 13

M UTATION O PERATORS Grow mutation: – With grow mutation a node is randomly selected and replaced by a randomly generated subtree. The new subtree is restricted by a predetermined depth. Gaussian mutation: – A terminal node that represents a constant is randomly selected and mutated by adding a Gaussian random value to that constant. Trunc mutation: – A function node is randomly selected and replaced by a random terminal node. This mutation operator performs a pruning of the tree. 14

F UNCTIONAL N ODE M UTATION 15

T ERMINAL N ODE M UTATION 16

S WAP M UTATION 17

G ROW M UTATION 18

G AUSSIAN M UTATION 19

T RUNC M UTATION 20

A DVANTAGES OF GP GP does not impose any fixed length of the solution. In principle, the maximal length can be extended up to the hardware limits. GP does not require as much knowledge about the problem and the possible solutions as do GAs. GPs can theoretically evolve any series of actions a computer can possibly do, provided that we give the GP algorithm a set of commands to choose from that can describe all possible actions. Slide from Erdem KOÇ & Ömer UZEL

A DVANTAGES OF GA They efficiently search the model space, so they are more likely to converge toward a global minima. There is no need of linearization of the problem. There is no need to compute partial derivatives. More probable models are sampled more frequently than less probable ones. Slide from Erdem KOÇ & Ömer UZEL

A PPLICATION F IELDS OF GP Symbolic Regression and Function Synthesis (Performing symbolic regression) Grammar Induction Data Mining and Data Analysis (Data mining and classification) Electrical Engineering and Circuit Design (Designing analog circuits) Medicine Economics and Finance Geometry and Physics Cellular Automata and Finite State Machines Automated Programming Robotics Networking and Communication Evolving Behaviors for Agents or Game Players Pattern Recognition Biochemistry Machine Learning Discovery of quantum algorithms Image processing Creating security protocols Slide from Erdem KOÇ & Ömer UZEL