1. Multi-variable non linear function deduction using Genetic Programming
Menarguez, Michael A.
April

2. Index
Introduction
Background
Methods
Discussion
Open problems
Results

3. Obtain original function
Introduction x1 X2 … xN
F(x1,x2,…, xN)
1.23
-2.54
9.3
8.32
8.123
2.43
18.32
4.23
f(x1,x2, … , xN)
Obtain original function
Collect
data
Genetic Programming
Eg: f(x, y)=(x-1)+xy2-xy+x3

4. Background (I) Genetic Programming (GP) Individual Population
Genetic operators
Mutation
Crossover
Selection

5. Replace random node with new tree
Background (II)
Genetic operators
Tournament
Random
Elitism
Crossover
Mutation
Selection
Replace random node with new tree
Simplification

6. Methods (I) Dataset: Performance metrics: (10 runs) Success / Failure
Evenly distributed points with normal random displacement.
100 data points.
No noise.
Performance metrics: (10 runs)
Success / Failure
MSE vs. Generation (Best and Worst)
Execution time

7. Methods (II) New node: Initialize node 50% ramp 50% half.
Elites: Best different individuals of previous population.
Tournament (size 4): get the two best individuals out of random 4 and crossover to produce 2 children.
Random crossover: Select random individuals from population and produce two children from them.
MUTATION_RATE = 0.15
MAX_NODE_INI_DEPTH = 3
POPULATION_SIZE: 100
Parameters:
+ - * / sin cos tan
Operators:
primes from 1 to 23.
Variable with ratio var/const of 2 to 1.
Terminals:

8. Results(I)
1.- Is it possible to find solution for a multi-variable non linear function?
f(x, y)=(x-1)+xy2-xy+x3
(((((x*(y^2.0))+(-1*(x*y)))+(x^3.0))+x)+(-1*1.0))
(((((x*(y^2.0))+(-1*(x*y)))+(x^3.0))+x)+(-1* ))
((((y*((x*y)+(-1*x)))+(x^3.0))+x)+(-1* ))
Some functions could not be found:
f(x, y)=sin(4x2+y)+sin(x+y)
f(x, y)= 7yx^(y+1)+13x2y+11
Probably because of complex and trigonometric functions with excessive depth

9. Results(II)
2- Can we use heuristics to get closer to the final result?
Add constant terms equal to +/-MSE to the fittest individual

10. Results(III) 3.- Does code bloating stall the search?
Solution: limit node depth.
We need to make sure there are enough nodes to represent the original function
Optimal node depth ~ 35
May vary depending upon function to be discovered

11. (Node depth = optimal = 35)
Results(IV)
4- What is the effect of random node simplification ratio?
(Node depth = optimal = 35)
Optimal simpl. rate~ 0.35

12. Open problems When function becomes complex solution may not be found.
Depth at some trigonometric and exponential functions should be limited (sin(sin(sin(sin((cos(x)+y)))))+sin((((sin(sin(sin(sin(x))))+sin(sin(4.0)))+y)+x)))
Results in equations like y + 4*ex*x+5*y*y are hard to find (Power function)

13. Future work
Design further heuristics to tune constant parameter at each term (eg: 4.32*x *y*x).
Test on noisy datasets
Implement a better simplificacion method.

14. GP demonstrates to be powerful for function deduction.
Conclusions
GP demonstrates to be powerful for function deduction.
Limiting node depth helps finding the solution faster and prevents it from getting stalled.
0.35 random simplification ratio helps finding the solution faster with its appropriate
Using heuristics for constant term approximation helps finding faster solutions.

15. Questions
Menarguez, Michael A.