1. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Model Reduction for Parameter Estimation Eric Mjolsness Scientific Inference Systems Laboratory (SISL) University of California, Irvine www.ics.uci/edu/~emj and Caltech Biological Network Modeling Center (BNMC) in collaboration with Rebecca Castaño, Dasha Chudova, Michael Duff, Victoria Gor, Henrik Jönsson, Tobias Mann, George Marnellos, Elliot Meyerowitz, John Reinitz, Bruce Shapiro, David Sharp, Padhraic Smyth, Yuanfeng Wang, Barbara Wold, Guy Yosiphon, Li Zhang Parameter Estimation In Systems Biology (PESB) Pascal Workshop, Manchester, UK March 28, 2007

2. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Topics A long-running thread in parameter estimation Biological applications: –transcriptional regulation –development Perspectives: –a near-universal bio. modeling language and semantics and its implications for … –parameter estimation and model reduction

3. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Transcriptional Gene Regulation Networks Gene Regulation Network [MSR’91] model Drosophila gap gene expression patterns. Reinitz, Mjolsness, Sharp, Journal Experimental Zoology 271(47-56) 1995. Fitting method demonstrated in Mittenthal and Baskin, The Principles of Organization of Organisms, Addison Wesley 1992. [Mjolsness et al. J. Theor. Biol. 152: 429-453, 1991] E.g. Drosophila A-P axis:

4. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 GRN Parameter Optimization Simulated Annealing [1990, ’92] –Lam/Delosme SA for real-valued params –Gap genes [JEZ 271(47-56) 1995]: 33 real-valued parameters Genetic Algorithm –Distributed over islands with migration, for diversity SA, GA compared in G. Marnellos thesis [1997] –GA won on evolution (life history) problems –SA won on development problems Other apps to GRN’s and signaling [Gor, Zhang] Then many others. Recently: –Kozlov BGRS 2006: differential evolution –Tomlin 2006: Adjoint method ~BP/cont. time

5. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 GRN ANN Equations ’91 Model statement and its derivation from stat mech: [Mjolsness Sharp and Reinitz, J. Theor. Biol. 152: 429-453, 1991] Key properties: (1) additivity, (2) saturation above and below, (3) monotonicity.

6. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07  [MSR91] equations are no longer just “phenomenological”. [J. Theor. Biol. 152: 429-453, 1991] Model Reduction Example: Gene Regulation Network Derived from Stat Mech [J.Bioinformatics & Comp. Biology, in press 2007]

7. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Dynamical Model Reduction via Clustering –Core/Halo Models: “From Coexpression to Coregulation …” [NIPS 1999 p.928-34] Identifiability by Gibbs sampling [Duff et al., ICSB 2005] –Functional Mixture Models [Chudova et al. NIPS 2003] Cluster M T v

8. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Core/Leaf Model Inference 3-node oscillator + leaves Modeled by S  E topologies Identifiability: x 25 time points: identifiable x 10 points: not identifiable x 10 points x 2 genotypes: ~identifiable (ranked #3) [Duff et al. ICSB2005]

9. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 SDE Advantages Intermediate cost for stochastic simulation Relationship to stochastic optimization Derivation from Fokker-Planck equation Eg. for GRN, HCA: [JBCB in press 2007]:

10. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Hierarchical Cooperative Activation: Alternative diagram notations Bio-like: Machine learning:

11. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Hierarchical Cooperative Activation Model (HCA) In: Computational Methods in Molecular Biology, eds. J. M. Bower and H. Bolouri, MIT Press 2001

12. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 How to model transcriptional regulation? [Robert P. Zinzen, Kate Senger, Mike Levine, and Dmitri Papatsenko. Current Biology 16, 1–8, July 11, 2006] E.g. Drosophila D-V axis:

13. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Hard vs. Soft Logic Hierarchical Cooperative Activation (HCA) Zinzen et al. modification Experiment: Yuanfeng Wang, UCI Physics

14. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 HCA - Z and ANN-like Equations Assume many binding sites per module Assume extreme (usually low) occupancy per site where A model reduction:

15. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 GRSN: Gene Regulation + Signal Transduction Network transcriptional regulation targets receptors ligands cell nucleus  T [Marnellos, Mjolsness, Shapiro] + … Drosophila neurogenesis [Marnellos, Mjolsness PSB ’98] Xenopus ciliated cells [PSB ’00] Arabidopsis SAM [Gor, Mjolsness,Meyerowitz, NASA Evolvable Hardware ’99]

16. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Arabidopsis Shoot Apical Meristem (SAM)

17. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Fletcher et al., Science v. 283, 1999 Brand et. al., Science 289, 617-619, (2000) WUS

18. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 SAM growth imagery H2B cell nuclei V. Reddy, Caltech

19. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 CLV3/WUS networks V. Agrawal, B. Shapiro, Caltech Z wus clv1 clv3 X diffusive YL1 diffusive

20. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 CLV/WUS model behavior Activation domains in Cellerator model: WUS (yellow), CLV3I1 (green), CLV3 (blue and purple), CLV1 (red and purple). B. Shapiro, JPL/Caltech

21. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 CLV/WUS Parameter Optimization by SA Courtesy H. Jönsson 2007; cf. ICSB 2006 14 parameters

22. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Biological scale hierarchies Biology, networks, & models: Objects(L) Processes(L) Objects(L+1) Processes(L+1) Objects(L-1) Processes(L-1) Objects(L+2) … … Noun and verb hierarchies: mutant wild type Perspective …

23. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Dynamical Grammar Aims Biology: Model complex systems –developmental biology (fly embryo, plant shoot/root) –molecular complexes –multiple-scale, heterogeneous, variable-structure systems Mathematics: Capture, unify, extend techniques –Generalized reactions cover all processes –Operator algebra, perturbation theory, … [Annals of Math. and A. I., 47(3-4), January 2007]

24. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Elementary Reactions A  B+ C with rate k f B+ C  A with rate k r Effective conservation laws E.g. N A + N B,N A + N C

25. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Elementary Processes A(x)  B(y) + C(z) with  f (x, y, z) B(y) + C(z)  A(x) with  r (y, z, x) Examples –Chemical reaction networks w/o params –. –XXX from paper Effective conservation laws –E.g. ∫ N A (x) dx + ∫ N B (y) dy, ∫ N A (x) dx + ∫ N C (z) dz

26. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Elementary process models Composition is by independent parallelism Create elementary processes from yet more elementary “Basis operators” –Term creation/annihilation operators: for each parm value, –Obeying Heisenberg algebra –Yet classical, not quantum, probabilities

27. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 SPG Modeling Language: Semantics Semantic map  from Grammar to Stochastic Process Commutative diagrams for composition operations Translation of a Rule  ’’ H, dp/dt H’, dp’/dt    

28. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Time Ordered Product Expansion (TOPE) formula: –H 0 = the easy part (if only recursively) –Feynman diagrams result (QFT: Perturbation theory, Wick’s theorem) Gillespie stochastic simulation algorithm –H 0 = diag( 1· H´) ; H 1 = H´ –Mixed (heterogeneous) ODE/SSA algorithm (novel) Implemented in “Plenum” (Yosiphon) [Annals of Math. and A. I., 47(3-4), January 2007]

29. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Model Reduction for Dynamical Systems Diagram: Objectives: Thus, parameter estimation can aid model reduction Uses of diagram: [UCI ICS TR #05-09]

30. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Composition vs. Specialization in a Lattice of Models Orthogonal kinds of model reduction/expansion (PartOf~InA, IsA) Commutative diagram for model lattice: Specialization: eg. discretized (DBN) vs. continuous (ODE) vs. quantized (stochastic) vbls, time, space - heterogeneous dynamics Initialize param search in specialized model high-level vision app [NIPS 1990]: Thus, model reduction can aid parameter estimation INA ISA

31. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 A Parameter Estimation Future Parameter estimation  model reduction –Multiscale, heterogeneous, variable-structure, … models all incorporated in a lattice –Common (operator algebra) semantics Perpetual data assimilation –Continual influx of data –Perpetual fitting to an expanding lattice of models Specialize to the limit of identifiability Model analyses to explain the “hits”

32. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Conclusions Model reductions for transcriptional regulation: –GRN’91, HCA Model reduction for large-scale data: –Core/Halo, Functional Mixture, … models Common framework: generalized reactions –Dynamical grammars  operator algebra Parameter estimation  model reduction –Mutually enhancing interaction

33. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 For further information: –www.ics.uci.edu/~emjwww.ics.uci.edu/~emj –www.computableplant.orgwww.computableplant.org Funding: US National Science Foundation FIBR program, NIH BISTI program Invitation…

34. UCI ICS IGB SISL Manchester PESB Workshop 28/3/07