Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2) Numerical solution: y’ = (x*y)/(x+y) + y^2 at y(0)=1 Can’t solve explicitly… So we’ll use numerical methods: (a) Dormand-Prince  (b) ode45: returns [x,y] values

 Given data points, find best fitting curve Normal regression finds coefficients of some predefined function (ie linear regression) Symbolic Regression finds function and coefficients  Since normal requires human intuition, we’ll use symbolic regression using genetic programming

 Tree Structure  Function Nodes from a Function Set  Terminal Nodes (numbers, variables) from a Terminal Set  Create Random Function Trees Making the function: y = x*sin(3.63/(2+x)) 

 Use Polish Notation Operator comes first. Ex: 5 + x – (x * 8))  + 5 – x * x 8 Quiz: + 3 * 6 – 2 1 = ?  Syntax-preserving: binary or unary functions? Binary: takes two nodes Unary: takes one node

 Evaluate error at each point  Fitness= -Total Sum of Error  If we want greater accuracy, we will want to evaluate more points in domain We may also want to limit the maximum error between any sample point and the function

 Random tournament to select who mates Randomly select two individuals in population The one with the best fit is parent 1 Randomly select crossover or mutation If mutate, mutate parent 1 Else find parent 2 and crossover Repeat reproduction until Population size is filled

 Crossover (GP) Crossover (DNA analogy) Parent 1: Parent 2: (a+b*sqrt(b)) / sqrt(b+b) a*(((b/b)/a) * sqrt(b+b)) Child: a+(b*(b/b)/a) / sqrt(b+b) ( Throw away second child)

 Randomly select node in tree to mutate  Randomly select a node of the same type  Mutate node  Mutation gets us out of pitfalls

 Determine Number of Data Points in Domain (Matlab) Step size:  Determine Function Set F = {sin, cos, e^, /, *, +, -}  Determine Terminal Set T = {x, pi/2, 100 random numbers between -5 and 5}  Create a Random Population of Function Trees 10,000 indiv. w/ initial depth of 5  Set Minimum Tolerance (error) Tolerance: 0.01  Set Maximum Num of Generations Max num of generations: 600  Set Crossover and Mutation rate Crossover: 80% Mutation: 1-Crossover = 20%

 y’=-y*cos(x)  Solvable  Ans: y=1 / e^(sin(x))  Using Genetic Programming Pop size: 1000 Domain [0, 5]  Problem Protected division

 y’=cos(2*x*y^2) No closed form solution Polyfit fails in matlab

…the explicit solution is…

 y = e^( ((((x / cos( sin( ((cos( ) / (x * ((x - (sin( e^( )) / (( (((cos( ) / (((x - e^( (sin( (x + ( cos( )))) * (sin( (x )) / (x + x ))))) / ) * cos( cos( cos( ((x ) / ((( / ) - ( ( / (cos( x ) / ( / cos( sin( ((( / cos( (( / ) / cos( cos( sin( ( ( (cos( ) / cos( )))))))))) ) + ( (cos( ) / cos( ))))))))))) ))))))) / ) + (cos( (( / ( * )) * (sin( (x + e^( ))) / (x - cos( e^( )))))) / cos( )))) / ( / cos( sin( ( ( (cos( ) / cos( )))))))))) * ))) / (sin( ) / ))))) / cos( sin( ((cos( ) / (x * ((x - (sin( e^( )) / (( (((cos( ) / ((( e^( (sin( (x )) * (sin( (x )) / (x + x ))))) / ) * cos( cos( cos( ))))) / ) + (cos( (e^( ) * (sin( (x + e^( ))) / (x - cos( e^( )))))) / cos( )))) / ( / cos( sin( ( ( (x / cos( )))))))))) * ))) / (sin( ) / ))))) - e^( (sin( (cos( (cos( ((( / ( * )) * (sin( (x + sin( ((( / ) / ) )))) / (x - cos( e^( ))))) )) / (((e^( x ) + sin( e^( (x / cos( cos( (x / e^( )))))))) - ((x / cos( sin( ((( cos( )) / (x * ( * ))) / (x / ))))) / )) + (( / ((x / cos( cos( x ))) - ( * cos( cos( ( / ((( * sin( x )) - cos( ( ( * cos( (x / )))))) ))))))) - cos( ))))) + (x + sin( ((( / ) / sin( )) ))))) * sin( (cos( (x / (cos( (x - (((cos( ) / (((x - e^( (sin( (x + (x - ((x - (sin( e^( )) / (( (((cos( ) / ((( e^( (sin( (x - (sin( (x )) ))) * (sin( (x )) / (x / ))))) ) + cos( cos( cos( ((x ) / ((( / ) / ( ( (cos( ) / ( / (x - cos( e^( )))))))) ))))))) * ) * (cos( (( / ( * )) * (sin( ( e^( ))) / (x - cos( e^( x )))))) + e^( )))) / ( / cos( sin( ( ( (cos( ) / cos( )))))))))) * )))) * sin( (cos( cos( )) + (sin( ) / (x - (((( / ) / sin( ((sin( (cos( (x - (x + e^( )))) + sin( (x + cos( sin( e^( ( * (sin( (x )) / (x + x )))))))))) / cos( x )) / x ))) ) )))))))) / ) * cos( ))) / ) / ((x - (cos( ) + (sin( e^( )) / (((x - (sin( e^( ((( / ((x / cos( cos( x ))) - ( * cos( (cos( ) / (x + cos( ( / x )))))))) - cos( (sin( (x - cos( (( / ( * (sin( ) * cos( )))) * ( / ( * )))))) * ))) / ))) / ( / ( / cos( ))))) * ) - cos( e^( )))))) / cos( cos( (x - cos( e^( ))))))))) ))) + (sin( e^( )) / (x - cos( e^( ))))))))) / ))  Not simplified (Maxima nor Matlab could simplify)  Fitness penalty for length

 Adapted Tiny_GP Java code Riccardo Poli. Found at  Burgess, Glenn. “Finding Approximate Analytic Solutions to Differential Equations” commissioned by Department of Defense (Australia) in 1999  Koza, John. Genetic Programming Riccardo Poli 