1. Recovering Temporally Rewiring Networks: A Model-based Approach
Fan Guo, Steve Hanneke, Wenjie Fu, Eric P. Xing
School of Computer Science, Carnegie Mellon University
11/10/2018
ICML 2007 Presentation

2. Social Networks Physicist Collaborations High School Dating
The Internet
11/10/2018
All the images are from That page includes original citations.

3. Biological Networks
Model for the Yeast cell cycle transcriptional regulatory network Fig. 4 from (T.I. Lee et al., Science 298, , 25 Oct 2002)
Protein-Protein Interaction Network in S. cerevisiae Fig. 1 from (H. Jeong et al., Nature 411, 41-42, 3 May 2001)
11/10/2018
The small image is from

4. When interactions are hidden…
Infer the hidden network topology from node attribute observations.
Methods:
Optimizing a score function; Information-theoretic approaches; Model-based approach …
Most of them pool the data together to infer a static network topology.
11/10/2018
ICML 2007 Presentation

5. And changing over time
Network topologies and functions are not static:
Social networks can grow as we know more friends
Biological networks rewire under different conditions
Fig. 1b from Genomic analysis of regulatory network dynamics reveals large topological changes
N. M. Luscombe, et al. Nature 431, , 16 September 2004
11/10/2018
ICML 2007 Presentation

6. Overview Network topologies and functions are not always static.
We propose probabilistic models and algorithms for recovering latent network topologies that are changing over time from node attribute observations.
11/10/2018
ICML 2007 Presentation

7. Rewiring Networks of Genes
Networks rewire over discrete timesteps
11/10/2018
Part of the image is modified from Fig. 3b (E. Segal et al., Nature Genetics 34, , June 2003).

8. The Graphical Model Transition Model Emission Model 11/10/2018
ICML 2007 Presentation

9. Technical Challenges
Latent network structures are of higher dimensions than observed node attributes
How to place constraints on the latent space?
Limited evidence per timestep
How to share the information across time?
11/10/2018
ICML 2007 Presentation

10. Energy Based Conditional Probablities
Energy-based conditional probability model (recall Markov random fields…)
Energy-based model is easier to analysis, but even the design of approximate inference algorithm can be hard.
11/10/2018
11/10/2018
ICML 2007 Presentation
ICML 2007 Presentation 10

11. Transition Model
Based on our previous work on discrete temporal network models in the ICML’06 SNA-Workshop.
Model network rewiring as a Markov process.
An expressive framework using energy-based local probabilities (based on ERGM):
Features of choice:
(Density)
(Edge Stability)
(Transitivity)
11/10/2018
11/10/2018
ICML 2007 Presentation
ICML 2007 Presentation 11

12. Emission Model in General
Given the network topology, how to generate the binary node attributes?
Another energy-based conditional model:
All features are pairwise which induces an undirected graph corresponding to the time-specific network topology;
Additional information shared over time is represented by a matrix of parameters Λ;
The design of feature function Φ is application-specific.
11/10/2018
11/10/2018
ICML 2007 Presentation
ICML 2007 Presentation 12

13. Design of Features for Gene Expression
The feature function
If no edge between i and j, Φ equals 0;
Otherwise the sign of Φ depends on Λij and the empirical correlation of xi, xj at time t.
11/10/2018
ICML 2007 Presentation

14. Graphical Structure Revisit
Hidden rewiring networks
Initial network to define the prior on A1
Time-invariant parameters dictating the direction of pairwise correlation in the example
11/10/2018
ICML 2007 Presentation

15. Inference
A natural approach to infer the hidden networks A1:T is Gibbs sampling:
To evaluate the log-odds
Conditional probabilities in a Markov blanket
Tractable transition model; the partition function is the product of per edge terms
Computation is straightforward
Given the graphical structure, run variable elimination algorithms, works well for small graphs
11/10/2018
ICML 2007 Presentation

16. Parameter Estimation
Grid search is very helpful, although Monte Carlo EM can be implemented.
Trade-off between the transition model and emission model:
Larger θ : better fit of the rewiring processes;
Larger η : better fit of the observations.
11/10/2018
ICML 2007 Presentation

17. Results from Simulation
Data generated from the proposed model.
Starting from a network (A0) of 10 nodes and 14 edges.
The length of the time series T = 50.
Compare three approaches using F1 score:
avg: averaged network from “ground truth” (approx. upper bounds the performance of any static network inference algorithm)
htERG: infer timestep-specific networks
sERG: the static counterpart of the proposed algorithm
Study the “edge-switching events”
11/10/2018
ICML 2007 Presentation

18. Varying Parameter Values
F1 scores on different parameter settings (varying )
11/10/2018
ICML 2007 Presentation

19. Varying the Amount of Data
F1 scores on different number of examples
11/10/2018
ICML 2007 Presentation

20. Capturing Edge Switching
Summary on capturing edge switching in networks
Three cases studied: offset, false positive, missing (false negative)
mean and rms of offset timesteps
11/10/2018
ICML 2007 Presentation

21. Results on Drosophila Data
The proposed model was applied to infer the muscle development sub-network (Zhao et al., 2006) on Drosophila lifecycle gene expression data (Arbeitman et al., 2002).
11 genes, 66 timesteps over 4 development stages
Further biological experiments are necessary for verification.
Network in (Zhao et al. 2006)
Embryonic
Larval
Pupal & Adult
11/10/2018
ICML 2007 Presentation

22. Summary
A new class of probabilistic models to address the problem of recoving hidden, time-dependent network topologies and an example in a biological context.
An example of employing energy-based model to define meaningful features and simplify parameterization.
Future work
Larger-scale network analysis (100+?)
Developing emission models for richer context
11/10/2018
ICML 2007 Presentation

23. Acknowledgement Yanxin Shi CMU Wentao Zhao Texas A&M University
Hetunandan Kamisetty CMU
11/10/2018
ICML 2007 Presentation

24. Thank You!
11/10/2018
ICML 2007 Presentation