1. Analysis and Modeling of AfCS Data What do we need and how do we get it?

2. What do we need? Statistical Data Analysis Mechanistic Modeling Thinkin’ about it. Paul Rama Madhu Michal Gil Lily Madhu The fabled sweet spot.

3. Some basic vocabulary Input: an experimental condition and/or treatment. –Ligands –siRNA –Toxins Observables –Calcium concentration –RNA expression –Chemotactic index Features –Peak calcium concentration –Basal calcium concentration

4. Statistical Data Analysis Did an observable change significantly during a treatment? What features of an observable change significantly? What groups of observables change in a correlated fashion across a number of treatments/measurements? Which input and observables are statistically prerequisites for other observables.

5. Mechanistic Models “Mechanistic” Models are a series of assertions about the causal structure and dynamics of a system. Examples –A is necessary for B to occur –P(A,B) != P(A)*P(B) –dB/dt= k1*A – k2*B –dP(X,t)/dt= W x’  x (X’,t)P(X’)-W x  x’ (X,t)P(X) Logical Statistical Deterministic Stochastic (I won’t talk about spatial today.)

6. What do we need? Most Alliance data is geared to statistical data analysis For modeling we get: –What is there? (e.g. Ryanodine receptors and IP3 receptors?) (Transcription) –Input/Output relations between ligands/siRNA and “outputs” (protein phosphorylation, Ca 2 + traces as a function of ligand and knock down) –Measures of uncertainty in these relations. We have an unprecedented set of quality-controlled data to get started.

7. Capturing Mechanistic Knowledge The FXM is using PathwayBuilder to capture “Mechanistic” knowledge –But there are still necessary and useful abstractions that are used We NEED highly-curated, biochemically reasonable pathways –Molecule pages are making great strides We MUST annotate model uncertainties –Confidences in the existence of an interaction –Confidences in the type of mechanism – We MUST annotate (relative) parameter value ranges

8. Pathway Builder Representation Initially very abstract But every “process” may be assigned a model Right now– different pathways must be made for each mechanistic hypothesis. But shortly we will be able to encode parameter uncertainty.

9. Abstract concepts can be modeled

10. Pathway Builder Representation Initially very abstract But every “process” may be assigned a model Right now– different pathways must be made for each mechanistic hypothesis. But shortly we will be able to encode parameter uncertainty.

11. Levels of Abstraction In the current AFCS release of PathwayBuilder the “Futile Cycle” is just graphical. But it is possible to assign an abstract model to that box to encode a phenomenological model of all the interactions within.

12. FXM Map Cytosolic calcium C5a UDP

13. Paring down models Current calcium models do NOT capture the path between receptors and calcium. But they do set the fundamental “response circuit” for calcium dynamics– given and initial change in IP 3 and DAG what calcium dynamics do you expect given expression of different channels and calcium and IP 3 receptors. So what ARE all the paths from receptors to effects on the calcium transients? How do they regulate each other and interact?

14. Current calcium measurements Current calcium models don’t explain different ligand response or variabilities. Peak height Peak width Final calcium upslope downslope downslope variability? C5A ResponseUDP Response

15. Single Cell Data Q: oscillation: C5a (high dose); UDP (low dose) Stolen from Lily Jiang New Feature Fraction of cells with each CLASS of response Particularities of each response

16. Modeling Issues How do we explain data variability with models? Mathematical representation of models

17. Model explanations The time dependent behavior of X depends on: Initial conditions of X Exact values of p The nature of the uncertainty, w E.g.

18. Bistability: Parameter dependence A simple model of the positive feedback Monostable Weakly bistable Irreversibly Bistable k C =1.6 kckc k c – catalytic constant for the trans-autophosphorylation. Stationary state [FAK-I]

19. B-p A A-p  Exogenous noise k c =1.6

20. Endogenous Noise

21. Dynamical Noise Effects With tiny noise on E+Without noise on E+

22. We are NOT talking about space Though we could…

23. Model Sensitivity and Features Can be used to bounds on parameter values

24. Model Building and (In)Validation with Data Collaboration Matt Onsum, Ryan Feeley, Michael Frenklach & Andrew Packard

25. Given a set of mechanistic models, we can determine which model is the most consistent with data. Set of Models Parameter Uncertainty Data Check for Consistency 1.Consistent Models 2.Invalidated models 3.Information on constrained data/parameters Model Invalidation

26. An experiment consists of: –Measured observable, D– features of the data –Experimental tolerance in measuring observable, e –Mathematical Model, M(), showing dependency on active variables  n –A set of acceptable values for . Since each parameter of the model has uncertainty, there exists a hypercube, H, of possible values for  The experiment actually asserts an inequality constraint among the active variables: |M() - D| < e. H Therefore we set up the following constrained optimization: Subject to Model/Data Consistency

27. Much can be accomplished in this optimization framework Check the consistency of the assertions. Does there exist a  satisfying all of the assertions? -- Invalidate proposed mechanisms -- Quick tests to indicate likely sources of inconsistency -- Subsets of the assertions may be readily considered The (deterministic) experimental uncertainties are directly transferred into prediction uncertainties. Generate a “best fit” parameter (more on this in the next slide)

28. Typical Data Processing Given: –A priori knowledge: -1  k 1 k n. –An experiment: (M(), D, e) with  n From this, all that can be concluded is |M()-D|<e. But, typically the procedure is: –Freeze all parameters except one, at the nominal:  k =0 for k  k 0 –Find range of the investigated (unfrozen) parameter: max/min  k 0 subject to:  k =0 for k  k 0 -1  k 0 1 |M()-D|<e The reported range is a subset of what can actually be inferred from (M(), D, e), but the implied higher dimensional cube (the new, in-literature feasible set) neither contains, nor is a subset of the feasible parameter set.    

29. Mistakes in Isolation 45

30. Related work: Consistency of Methane Combustion Database GRI-Mech has 300+ elementary reactions, 53 Species, and 102 “active” parameters The community needed a database containing all relevant experimental info for methane combustion to determine the “right” kinetic parameters It was realized that the best-fit parameter values did not give the combustion model good predictive power Needed a way to better incorporate uncertainty about the parameter values Pathway diagram for methane combustion [Turns] Michael Frenklach, Andrew Packard, Pete Seiler and Ryan Feeley, “Collaborative data processing in developing predictive models of complex reaction systems,” International Journal of Chemical Kinetics, vol. 36, issue 1, pp. 57-66, 2004. Michael Frenklach, Andy Packard and Pete Seiler, “Prediction uncertainty from models and data,” 2002 American Control Conference, pp. 4135-4140, Anchorage, Alaska, May 8-10, 2002.

31. Sensitivity of Data Set Consistency to Assertions Feature number Upper Bound Lower Bound

32. Modeling goal for the AfCS data Distinguish and rank competing mechanistic models Propose experiments that will further distinguish competing models Identify structural problems with current models

33. In the examples that follow pathway 1 will be the “true model” Data was generated by simulating the true model with random initial conditions.Each initial condition assumed to be Gaussian with nominal mean, and 0.01 variance. This was run 1000 times and the resulting pathways were averaged to give trace data We then try to find the maximum parameter uncertainty that still allows us to identify the true pathway. Distinguishing similar pathways

34. Example 1: Test for degenerate solutions Features: 4- Peak value

35. Example 2: Test for complex formation Feature:

36. Example 3: Test for reversibility Features: 1- Rise Time (i.c. 1,1) 2- Peak value 3- Rise Time (i.c. 2,2) 5- Rise Time after pretreatment

37. Example 4: Test for missing intermediate

38. Summary of Toy Examples 1.For large parameter and experimental uncertainty, multiple models can fit the same data. 2. Some experiments provide tighter constraints then others. 3.Repeats of these experiments (reduction of uncertainty) improves our ability to distinguish similar pathways.

39. Initial data is dose response Can we use legacy models to explain AFCS data?

40. We began with the model by Goldbeter Steady state Ca 2+ Peak Output Upper Bound Lower Bound Formation of active G- protein Inactivation of G-protein

41. Weisner model fits a single response

42. Simulations of base model show two sensitive parameters.

43. However, we could not fit the model to the dose response data. The model was not able to reproduce the change in steady state values Steady-state calcium level. Passive ER Ca 2+ leak K m ion pump V max ion exchanger V max Ca 2+ ATPase pump Lower BoundUpper Bound

44. Conclusions Method for model validation Showed that it can distinguish between canonical pathways even with high uncertainty We have begun invalidating literature models

45. So what do we need? (Experiment) A number of well-chosen knock-downs upstream of calcium and in “independent” parts of the different receptor pathways. (Accurate assessment of loss of function, induction) Ways of separating exogenous variability from endogenous variability from measurement noise. Measurement of the dose-response of intermediates (not just calcium) for the single FXM ligands and Determination of a set of physiologically relevant and significantly affected features. Similarly for double-ligand responses. Single cell assays should be expanded!

46. So what do we need? (Analysis) Identification of important features in the data that we wish to explain. Determination of value and variance of significantly changing features. Figure out a consistent way to classify single cell responses.

47. So what do we need? (Modeling) Biochemists and geneticists editing the maps and making hypotheses –Perhaps we should have a model hypothesis page as an addendum to Henry’s? An initial “frozen” data set to be the test bed for all initial modeling discussions. An initial “frozen” analysis thereof A series of “minimal” pathways derived from the FXM maps that are believed to be the significant determinants of our output signals. Choice of mathematical picture and inference about the significance of the single cell responses. A direct way of driving experiments from models.

48. Acknowledgements Matt Onsum Ryan Feeley Andrew Packard Michael Frenklach Michael Samoilov Alex Gilman Matt Andy Mike The Alliance and especially the FXM Lily Jiang Madhu Natarajan Gil Sambrano