1. Discovering Dynamic Models Lecture 21

2. Dynamic Models: Introduction Dynamic models can describe how variables change over time or explain variation by appealing to mechanisms. The informatics systems in this lecture address the discovery of three types of models. Each system combines domain knowledge provided by an expert with heuristic search enacted by a computer. Gretl:supports the discovery of descriptive, statistical models of time series. MECHEM:generates qualitative models that explain the results of chemical reactions. Prometheus:creates quantitative models that explain the dynamics of complex systems.

3. ARIMA Models ARIMA models are commonly used in statistical time series analysis, particularly within the field of econometrics. ARIMA stands for Autoregressive:X t is a linear combination of the prior p values and a random process Z t. X t = a 1 X t-1 + … + a p X t-p + Z t Integrated:a differencing method to remove a trend. e.g., X t is replaced with X t – X t-1 for estimation Moving Average:X t is a linear combination of the random processes in the previous q values. X t = μ + Z t + b 1 Z t-1 + … + b q Z t-q An ARIMA model describes the time series for one variable and may be used for forecasting. VARIMA models extend these to support multiple variables.

4. The Gretl Environment Gretl is an informatics tool used by econometricians that supports a wide variety of statistical analyses. The main window shows basic information about the data Gretle supports several, customizable plots. This data in this plot show seasonality and a general upward trend.

5. Input for Gretl Gretl can discover the parameters of a model, but not its structure. Scientists tell Gretl the variables to be described; the type of model to use; and the structure of the model. Gretl supports ARIMA and other types of time-series models. The structure of an ARIMA model reflects the number of terms.

6. Search in Gretl Gretl uses maximum likelihood estimation to fit ARIMA parameters. X t =a 1 X t-1 + … + a p X t-p + Z t + μ + b 1 Z t-1 + … + b q Z t-q Where Z 0 = 0, and Z t = X t – μ – (b 1 Z t-1 + … + b q Z t-q ) This window shows the resulting parameters and fitness scores. This graph shows the predictions (in blue) from an ARIMA model. The forecast is in blue and its 95% confidence interval in green. The observed values are in red.

7. The MECHEM Environment MECHEM is an interactive tool that generates plausible reaction pathways for chemical interactions. Scientists interact with MECHEM through a graphical interface as presented in the figure. Clockwise from the upper left, the interface shows the main menu; the current reaction; an example mechanism; a set of constraints; and the output log.

8. Knowledge in MECHEM 1.Every conjectured intermediate has at most two O atoms 2.O atoms cannot be bonded to O atoms 3.Reject Eley-Rideal mechanisms 4.The site M must be present in all steps 5.No multiple occurrences of any reactant pair on the left-hand side of steps 6.CH4 cannot appear on the left-hand side of a step Constraints on the structure of candidate mechanisms. starting materials: CO, H 2 catalyst site: M dual catalyst sites: MM observed C 1 products: CO 2, CH 3 OH, CH 4, H 2 O Reaction-specific information including chemical products. MECHEM also includes a general set of parameters, such as, “Consider at most 7 conjectured species.” The purpose of all this knowledge is to rule out a large number of implausible reaction-pathways.

9. Search in MECHEM MECHEM uses heuristic search through a space of symbolic structures to identify reaction pathways. The search has several features: it favors pathways with few species and steps; it ensures the unique generation of candidate pathways; it requires balanced chemical equations; and it limits steps to at most two reactants and two products. These aspects involve highly general constraints that limit the search space before a scientist adds prior knowledge. 1. H 2 + MM ➞ 2MH 2. CO + MM ➞ M 2 CO 3. MH + M 2 CO ➞ M 2 CHOM Partial reaction pathway discovered by MECHEM.

10. The Prometheus Environment Recall that Prometheus supports quantitative process models that relate variables through numeric processes. Scientists can build models from generic processes that appear in domain-specific libraries.

11. Knowledge in Prometheus Generic processes encode the knowledge used by Prometheus to discover quantitative process models. generic process exponential_loss variables: S{species}, D{detritus} parameters:  [0, 1] equations:d[S,t,1] =  1    S d[D,t,1] =   S generic process grazing variables: S1{species}, S2{species}, D{detritus} parameters:  [0, 1],  [0, 1] equations:d[S1,t,1] =     S1 d[D,t,1] = (1   )    S1 d[S2,t,1] =  1    S1 generic process nutrient_uptake variables: S{species}, N{nutrient} parameters:  [0,  ],  [0, 1],  [0, 1] conditions:N >  equations:d[S,t,1] =   S d[N,t,1] =  1      S Generic processes specify which variables may interact, the equations that govern the dynamics of their interaction, ranges on the parameters associated with the processes. Scientists also give Prometheus a list of variables that should appear in the model.

12. Search in Prometheus Unlike Gertl and MECHEM, Prometheus carries out multi- stage search for a model’s structure and its parameters: 1. Find all ways to instantiate known generic processes with specific variables, subject to type constraints; 2. Combine instantiated processes into candidate generic models subject to additional constraints; 3. For each generic model, carry out search through parameter space to find good coefficients; 4. Return the parameterized models with the best overall scores (e.g., sum of squared error). Like MECHEM, Prometheus returns multiple models for the scientist to inspect and possibly refine. To this end, the environment supports incremental revision.

13. Discovering Dynamic Models: Summary The informatics systems discussed in this lecture covered three types of dynamic models. Nevertheless there were commonalities as each system Moreover, researchers have applied these systems to generate new knowledge in scientific domains. used domain knowledge such as a model structure, constraints on candidates, or model components; used search techniques to discover model parameters, structures, or both; worked interactively with scientists to explain their data.