1. Cellular Dynamics From A Computational Chemistry Perspective Hong Qian Department of Applied Mathematics University of Washington

2. The most important lesson learned from protein science is …

3. The current state of affair of cell biology: (1) Genomics: A,T,G,C symbols (2) Biochemistry: molecules

4. Experimental molecular genetics defines the state(s) of a cell by their “transcription pattern” via expression level (i.e., RNA microarray).

5. Biochemistry defines the state(s) of a cell via concentrations of metabolites and copy numbers of proteins.

6. Ghaemmaghami, S. et. al. (2003) “Global analysis of protein expression in yeast”, Nature, 425, 737-741. Protein Copy Numbers in Yeast

7. Roessner-Tunali et. al. (2003) “Metabolic profiling of transgenic tomato plants …”, Plant Physiology, 133, 84-99. Metabolites Levels in Tomato

8. But biologists define the state(s) of a cell by its phenotype(s)!

9. How does computational biology define the biological phenotype(s) of a cell in terms of the biochemical copy numbers of proteins?

10. Theoretical Basis: The Chemical Master Equations: A New Mathematical Foundation of Chemical and Biochemical Reaction Systems

11. The Stochastic Nature of Chemical Reactions Single-molecule measurements Relevance to cellular biology: small copy # Kramers’ theory for unimolecular reaction rate in terms of diffusion process (1940) Delbrück’s theory of stochastic chemical reaction in terms of birth-death process (1940)

12. Single Channel Conductance

13. First Concentration Fluctuation Measurements (1972) (FCS)

14. Fast Forward to 1998

15. Stochastic Enzyme Kinetics 0.2mM 2mM Lu, P.H., Xun, L.-Y. & Xie, X.S. (1998) Science, 282, 1877-1882.

16. Stochastic Chemistry (1940)

17. The Kramers’ theory and the CME clearly marked the domains of two areas of chemical research: (1) The computation of the rate constant of a chemical reaction based on the molecular structures, energy landscapes, and the solvent environment; and (2) the prediction of the dynamic behavior of a chemical reaction network, assuming that the rate constants are known for each and every reaction in the system.

18. Kramers’ Theory, Markov Process & Chemical Reaction Rate               P xF x x P D t P  )( 2 2 x xE xF    )( )( AB k2k2 k1k1 ),(),( ),( 21 tBPktAPk dt tAdP  AB

19. But cellular biology has more to do with reaction systems and networks …

20. Traditional theory for chemical reaction systems is based on the law of mass-action

21. Nonlinear Biochemical Reaction Systems and Kinetic Models A X k1k1 k -1 BY k2k2 2X+Y 3X k3k3

22. The Law of Mass Action and Differential Equations dt d c x (t) =k 1 c A - k -1 c x +k 3 c x 2 c y k 2 c B - k 3 c x 2 c y = dt d c y (t)

23. uu a = 0.1, b = 0.1 a = 0.08, b = 0.1 Nonlinear Chemical Oscillations

24. A New Mathematical Theory of Chemical and Biochemical Reaction Systems based on Birth- Death Processes that Include Concentration Fluctuations and Applicable to small systems.

26. The Basic Markovian Assumption: X+YZ k1k1 The chemical reaction contain n X molecules of type X and n Y molecules of type Y. X and Y bond to form Z. In a small time interval of  t, any one particular unbonded X will react with any one particular unbonded Y with probability k 1  t + o(  t), where k 1 is the reaction rate.

27. A Markovian Chemical Birth- Death Process nZnZ k1nxnyk1nxny k 1 (n x +1)(n y +1) k -1 n Z k -1 (n Z +1) k1k1 X+YZ k -1

28. Chemical Master Equation Formalism for Chemical Reaction Systems M. Delbrück (1940) J. Chem. Phys. 8, 120. D.A. McQuarrie (1963) J. Chem. Phys. 38, 433. D.A. McQuarrie, Jachimowski, C.J. & M.E. Russell (1964) Biochem. 3, 1732. I.G. Darvey & P.J. Staff (1966) J. Chem. Phys. 44, 990; 45, 2145; 46, 2209. D.A. McQuarrie (1967) J. Appl. Prob. 4, 413. R. Hawkins & S.A. Rice (1971) J. Theoret. Biol. 30, 579. D. Gillespie (1976) J. Comp. Phys. 22, 403; (1977) J. Phys. Chem. 81, 2340.

30. Nonlinear Biochemical Reaction Systems: Stochastic Version A X k1k1 k -1 BY k2k2 2X+Y 3X k3k3

31. (0,0) (0,1) (0,2) (1,0) (1,1) (2,0) (1,2) (3,0) (2,1) k1nAk1nA k1nAk1nA k1nAk1nA k1nAk1nA k1nAk1nA k 2 n B 2k 3 k -1 2k -1 3k -1 4k -1 k -1 (n+1) (n,m)(n-1,m) (n+1,m) (n,m+1)(n+1,m+1) k1nAk1nA k1nAk1nA (n,m-1) k 2 n B (n-1,m+1) k 3 n (n-1)m k 3 (n-1)n(m+1) k 3 (n-2)(n-1)(m+1) k -1 m k -1 (m+1) k 2 n B (n+1,m-1) k1nAk1nA k 3 (n-2)(n-1)n

32. Stochastic Markovian Stepping Algorithm (Monte Carlo)  =q 1 +q 2 +q 3 +q 4 = k 1 n A + k -1 n+ k 2 n B + k 3 n(n-1)m Next time T and state j? (T > 0, 1< j < 4) q3q3 q1q1 q4q4 q2q2 (n,m) (n-1,m)(n+1,m) k1nAk1nA (n,m-1) k 2 n B k 3 n (n-1)m k -1 n (n+1,m-1)

33. Picking Two Random Variables T & n derived from uniform r 1 & r 2 : f T (t) = e - t, T = - (1/ ) ln (r 1 ) P n (m) = k m / , (m=1,2,…,4) r2r2 0 p1p1 p 1 +p 2 p 1 +p 2 +p 3 p 1 +p 2 +p 3 +p 4 =1

34. Concentration Fluctuations

35. Stochastic Oscillations: Rotational Random Walks a = 0.1, b = 0.1a = 0.08, b = 0.1

36. Defining Biochemical Noise

37. An analogy to an electronic circuit in a radio If one uses a voltage meter to measure a node in the circuit, one would obtain a time varying voltage. Should this time-varying behavior be considered noise, or signal? If one is lucky and finds the signal being correlated with the audio broadcasting, one would conclude that the time varying voltage is in fact the signal, not noise. But what if there is no apparent correlation with the audio sound?

38. Continuous Diffusion Approximation of Discrete Random Walk Model

39. Stochastic Dynamics: Thermal Fluctuations vs. Temporal Complexity  FPPD t tvuP    ),,(            vubvu vuvuua D 22 22 2             vub vuua F 2 2 StochasticDeterministic, Temporal Complexity

40. Time Number of molecules (A) (C) (D) (B)(E) (F) Temporal dynamics should not be treated as noise!

41. A Second Example: Simple Nonlinear Biochemical Reaction System From Cell Signaling

42. We consider a simple phosphorylation-dephosphorylation cycle, or a GTPase cycle:

43. AA* S ATPADP I Pi k1k1 k -1 k2k2 k -2 Ferrell & Xiong, Chaos, 11, pp. 227-236 (2001) with a positive feedback

44. Two Examples From Cooper and Qian (2008) Biochem., 47, 5681. From Zhu, Qian and Li (2009) PLoS ONE. Submitted

45. Simple Kinetic Model based on the Law of Mass Action NTPNDP Pi E P RR*

46. activating signal:  activation level: f 14 1 Bifurcations in PdPC with Linear and Nonlinear Feedback  = 0  = 1  = 2 hyperbolicdelayed onset bistability

47. R R* K P 2R* 0R*1R* 3R* … (N-1)R* NR* Markov Chain Representation v1v1 w1w1 v2v2 w2w2 v0v0 w0w0

48. Steady State Distribution for Number Fluctuations

49. Large V Asymptotics

50. Beautiful, or Ugly Formulae

52. Bistability and Emergent Sates PkPk number of R* molecules: k defining cellular attractors

53. A Theorem of T. Kurtz (1971) In the limit of V →∞, the stochastic solution to CME in volume V with initial condition X V (0), X V (t), approaches to x(t), the deterministic solution of the differential equations, based on the law of mass action, with initial condition x 0.

54. We Prove a Theorem on the CME for Closed Chemical Reaction Systems We define closed chemical reaction systems via the “chemical detailed balance”. In its steady state, all fluxes are zero. For ODE with the law of mass action, it has a unique, globally attractive steady-state; the equilibrium state. For the CME, it has a multi-Poisson distribution subject to all the conservation relations.

55. Therefore, the stochastic CME model has superseded the deterministic law of mass action model. It is not an alternative; It is a more general theory.

56. The Theoretical Foundations of Chemical Dynamics and Mechanical Motion The Semiclassical Theory. Newton’s Law of MotionThe Schrödinger’s Eqn. ħ → 0 The Law of Mass ActionThe Chemical Master Eqn. V → x 1 (t), x 2 (t), …, x n (t) c 1 (t), c 2 (t), …, c n (t)  (x 1,x 2, …, x n,t) p(N 1,N 2, …, N n,t)

57. Chemical basis of epi-genetics: Exactly same environment setting and gene, different internal biochemical states (i.e., concentrations and fluxes). Could this be a chemical definition for epi-genetics inheritance?

58. Chemistry is inheritable!

60. Emergent Mesoscopic Complexity It is generally believed that when systems become large, stochasticity disappears and a deterministic dynamics rules. However, this simple example clearly shows that beyond the “infinite-time” in the deterministic dynamics, there is another, emerging stochastic, multi-state dynamics! This stochastic dynamics is completely non- obvious from the level of pair-wise, static, molecule interactions. It can only be understood from a mesoscopic, open driven chemical dynamic system perspective.

61. AB discrete stochastic model among attractors nyny nxnx chemical master equation cycy cxcx A B fast nonlinear differential equations appropriate reaction coordinate A B probability emergent slow stochastic dynamics and landscape (a) (b) (c) (d) In a cartoon fashion

62. The mathematical analysis suggests three distinct time scales, and related mathematical descriptions, of (i) molecular signaling, (ii) biochemical network dynamics, and (iii) cellular evolution. The (i) and (iii) are stochastic while (ii) is deterministic.

63. The emergent cellular, stochastic “evolutionary” dynamics follows not gradual changes, but rather punctuated transitions between cellular attractors.

64. If one perturbs such a multi- attractor stochastic system: Rapid relaxation back to local minimum following deterministic dynamics (level ii); Stays at the “equilibrium” for a quite long tme; With sufficiently long waiting, exit to a next cellular state.

65. alternative attractor local attractor Relaxation process abrupt transition Relaxation, Wating, Barrier Crossing: R-W-BC of Stochastic Dynamics

66. Elimination Equilibrium Escape

67. In Summary There are two purposes of this talk: On the technical side, a suggestion on computational cell biology, and proposing the idea of three time scales On the philosophical side, some implications to epi-genetics, cancer biology and evolutionary biology.

68. Into the Future: Toward a Computational Elucidation of Cellular attractor(s) and inheritable epi- genetic phenotype(s)

69. What do We Need? It requires a theory for chemical reaction networks with small numbers of molecules The CME theory is an appropriate starting point It requires all the rate constants under the appropriate conditions One should treat the rate constants as the “force field parameters” in the computational macromolecular structures.

70. Analogue with Computational Protein Structures – 40 yr ago While the equation is known in principle (Newton’s equation), the large amount of unknown parameters (force field) makes a realistic computation essentially impossible. It has taken 40 years of continuous development to gradually converge to an acceptable “set of parameters” But the issues are remarkably similar: defining biological (conformational) states, extracting the kinetics between them, and ultimately, functions.

71. Thank You!