Automated Reverse Engineering of Nonlinear Dynamical Systems July 9, 2007 Automated Reverse Engineering of Nonlinear Dynamical Systems Josh Bongard† & Hod Lipson Computational Synthesis Laboratory Cornell University hod.lipson@cornell.edu †Current Address: Department of Computer Science University of Vermont josh.bongard@uvm.edu

Model: dθ/dt = ? dω/dt = ?

+f(θ,ω) [f = friction term] Best evolved model: dθ/dt = 1.008ω + 0.0028 dω/dt = -19.43sin(1.0009θ+0/-1.575/-2.673) +f(θ,ω) [f = friction term] Best human-created model: dθ/dt = ω dω/dt = -9.8Lsin(θ) +f(θ,ω) [L = length of arm] First reported algebraic model describing pendula published by H. Kamerlingh Onnes in 1879. Form of these equations still presented to this day in engineering textbooks as the mathematical description of a damped pendulum. Meets criteria (B): Our model matches the best-known model form for a pendulum. Our models contain explicit terms for the friction term f; friction terms are unknown for physical, damped pendula— dependent on mechanical coupling, shape of arm, wind resistance, etc. Our model automatically captures in symbolic form, changes to the system (rotation of the main box). Box is flat Rotated -1.57rad Rotated -2.67rad

Best evolved model: dG/dt = ? dA/dt = ? dL/dt = ?

dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1) Best evolved model: dG/dt = 0.96A2/(0.96A2+1) dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1) dG/dt = A2/(A2+1) – 0.01G + 0.001 dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1)

dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1) Best evolved model: dG/dt = 0.96A2/(0.96A2+1) dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1) Lac operon discovered in 1961: Jacob F, Monod J (1961). "Genetic regulatory mechanisms in the synthesis of proteins". J Mol Biol. 3: 318-56. As of 1998, the best human-created model was: from Keener J, Sneyd J (1998) Mathematical Physiology (Springer). Criteria (B): The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. Took humans 37 years to create this model from observations; Took our algorithm 5 minutes running on a desktop PC to create a very similar model from time series data. dG/dt = A2/(A2+1) – 0.01G + 0.001 dA/dt = G( L/(L+1) – A/(A+1) ) dL/dt = -GL/(L+1)

Best evolved model: dH/dt = ? dL/dt = ?

Best evolved model: dH/dt = 3.42x106 - 67.82H - 10.97L dL/dt = 3.10x105 + 32.66H - 63.16L

Best human-created model: dH/dt = αH – βHL dL/dt = -γL + δHL Best evolved model: dH/dt = 3.42x106 - 67.82H - 10.97L dL/dt = 3.10x105 + 32.66H - 63.16L Best human-created model: dH/dt = αH – βHL dL/dt = -γL + δHL Proposed independently by American biophysicist Alfred Lotka and Italian mathematician Vito Volterra in 1925. (Volterra, V. (1926) Jour. du conseil intl. pour l’exp. de la mer) Still forms the basis of most population dynamics models today. Meets criteria (B): Our model is similar, but simpler: No nonlinear terms, yet still conveys: which species in the prey, and which the predator; prey decreases proportionally to amount of predator; predator increases proportionally in response to prey; when integrated, produces coupled oscillations; predator peaks follow prey peaks.

Algorithm correctly inferred the underlying form in 5 minutes. Criteria (G): The result solves a problem of indisputable difficulty in its field. Algorithm presented with time series data from nonlinear, coupled systems of increasing size. Algorithm correctly inferred the underlying form in 5 minutes. System 1 System 2 System 3 2 variables dx1/dt= -3x1x1 -3x1x2 +2x2x2 dx2/dt= -x1x1 -3x1x2 -2x2x2 dx1/dt=-3x1x1 +3x1x2 +3x2x2 dx2/dt=-3x1x1-2x1x2 +2x2x2 dx1/dt=3x1x1 -x1x2 -x2x2 dx2/dt=x1x1 +3x1y2 -x2x2 Success rate 20% (no partitioning) 100% (partitioning) 0% (no partitioning) 96.7% (partitioning) 3 variables dx1/dt=-3x1x3 -2x2x3 -3x3x3 dx2/dt=-3x1x2 +x1x3 -3x2x3 dx3/dt=3x1x2 +3x1x3 –x2x3 dx1/dt=-x1x2 +x1x3 -x2x3 dx2/dt=x1x1 +2x1x2 +2x2x3 dx3/dt=-2x1x1 +x1x2 -3x2x3 dx1/dt=-3x1x2 +x1x3 -x3x3 dx2/dt=-2x1x3 +3x2x3 +3x3x3 dx3/dt=2x1x2 -2x1x3 -2x2x3 63.3% (partitioning) 4 variables dx1/dt=-x1x1+2x2x3+2x3x3 dx2/dt=x1x2 -3x1x3 -3x2x3 dx3/dt=-x1x1 -x2x4 +3x4x4 dx4/dt=-3x1x2 -3x1x4 –3x3x4 dx1/dt=x1x4+x2x4+x4x4 dx2/dt=-3x1x2-2x2x3-3x3x4 dx3/dt=2x1x2-x1x3+2x2x2 dx4/dt=x1x3+3x2x3-x3x4 dx1/dt=-3x1x1+3x1x2+3x2x4 dx2/dt=-x1x1-2x1x3-3x4x4 dx3/dt=-2x1x4+x2x2-3x3x4 dx4/dt=-x1x2+2x1x4-3x3x4 90% (partitioning) 83.3% (partitioning) 5 variables dx1/dt=-3x1x5 +3x2x3 -3x2x5 dx2/dt=-3x1x3 -2x3x4 -x4x5 dx3/dt=x1x1 -3x1x4 +x2x4 dx4/dt=3x1x3 -3x1x4 +2x2x2 dx5/dt=3x1x4 +3x3x3 +3x3x4 dx1/dt=-2x2x2+3x3x5+2x4x5 dx2/dt=3x1x2 +x1x5 -2x2x5 dx3/dt=x1x2 +2x2x5 +2x4x5 dx4/dt=2x1x2 +3x1x5 -x4x5 dx5/dt=2x1x5 -x2x5 -2x5x5 dx1/dt=2x1x4 +2x2x3 -x2x4 dx2/dt=x1x3 +3x1x4 +x2x4 dx3/dt=-2x1x1 +2x1x2 -3x1x3 dx4/dt=-3x2x5 +3x3x4 -x3x5 dx5/dt=x1x1 +x1x5 +x2x3 76.7% (partitioning) 6 variables dx1/dt=-2x1x6+x2x4-2x2x6 dx2/dt=x1x4-x1x5-2x4x4 dx3/dt=2x2x5-x3x4+x5x5 dx4/dt=-3x4x5-2x4x6+2x5x5 dx5/dt=x3x6-2x4x4-3x4x5 dx6/dt=x3x4-x3x6+2x4x6 dx1/dt=-2x1x3-3x2x4+2x3x6 dx2/dt=-3x2x4+x3x4-x3x6 dx3/dt=-x1x2-x1x3+x4x6 dx4/dt=-x1x4+x3x5-2x4x6 dx5/dt=3x1x2-3x1x6-x5x5 dx6/dt=-3x1x3-2x1x6-3x4x6 dx1/dt=x1x5+x1x6+x4x5 dx2/dt=-2x2x5-2x2x6+2x3x6 dx3/dt=-x1x5-2x3x4+x4x4 dx4/dt=3x1x2+3x2x3-2x4x5 dx5/dt=-3x1x5+x2x2+3x2x6 dx6/dt=-x2x5-2x3x5-3x5x6 80% (partitioning) 70% (partitioning) 93% (partitioning) 2 variables Best-known predator-prey model; single pendulum model 3 variables Best-known lac operon model of E. coli as of 1998 5 variables Best-known lac operon model of E. coli as of 2003 (Yildirim & Mackey, 2003)

Why should this entry be considered “best”? Work appeared in one of the world’s most prestigious scientific journals, and not in an evolutionary computation journal: Was favorably reviewed by four renowned scientists outside of the EC community. One reviewer remarked that this approach “…has a chance of changing the way some disciplines conduct science”. Citation—Bongard J. and Lipson H.(2007). Automated reverse engineering of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 104(24): 9943-9948.