1. ONR MURI: NexGeNetSci From Local Network Motifs to Global Invariants Third Year Review, October 29, 2010 Victor M. Preciado and Ali Jadbabaie Department of Electrical and Systems Engineering University of Pennsylvania

2. Theory Data Analysis Numerical Experiments Lab Experiments Field Exercises Real-World Operations First principles Rigorous math Algorithms Proofs Correct statistics Only as good as underlying data Simulation Synthetic, clean data Stylized Controlled Clean, real-world data Semi- Controlled Messy, real-world data Unpredictable After action reports in lieu of data Preciado Local Motifs and Global Invariants

3. ONR MURI: NexGeNetSci Motivation and context The role of local structural information Spectral analysis from local structural information Bounds on spectral properties via optimization Implications in dynamical processes Outline

4. ONR MURI: NexGeNetSci Complex Network: Properties Generic features: –Large number of nodes –Sparse connectivity –Lack of regularity Examples: –Comm networks (e.g. Internet) –Social networks (e.g. Facebook) –Biological networks We assume limited structural information: –Privacy and/or security concerns –Storage/computing limitations ??

5. Challenges when only local structural information is available: –Estimation: How could we aggregate local measurements to infer global properties of the network? –Inference: What could we say about the behavior of a dynamical process in the network from local measurements? –Actuation: How could we modify the structure of a network to induce a desired global behavior? Complex Networks: Some Challenges Estimation Inference Actuation

6. ONR MURI: NexGeNetSci Overly focused on random graph models and degree distributions, but we can have very different networks with the same degree distribution [Li et al., 2005]: Main drawbacks: 1.Degree distributions are a zero-th order approximation of the network structure, by far not enough 2.Random models are difficult, if not impossible, to justify from an engineering perspective Usual Approach in “Network Science”

7. ONR MURI: NexGeNetSci We also find random graph models capturing increasingly richer structural properties [Mahadevan et al., 2006] Main drawbacks: –Visual inspection is clearly not enough to measure similarity –What structural measurements are relevant in the behavior of dynamical processes in networks? More Structured Random Models Average degree Degree distributionJoint Degree Distribution Distribution of Triangles Original HOT model

8. ONR MURI: NexGeNetSci Our framework: We consider the dynamical behavior of networks Since the eigenvalues and eigenvectors are closely related with the network dynamical behavior, spectral graph theory is a convenient framework to study network dynamics Some relationships between spectra and dynamics are: –Spreading processes  adjacency spectral radius –Synchronization  (combinatorial) Laplacian eigenratio –Diffusion/Consensus  (normalized) Laplacian eigenvalues Networks = Graphs + Dynamics ))

9. Aggregation of local structural measurements Dynamical Implications ONR MURI: NexGeNetSci Inference from Local Measurements Our problems: 1. - What measurements are most relevant in the behavior of dynamical processes? 2. - How can we aggregate local measurements to say something about the global dynamical behavior? We study those problems in the framework of spectral graph theory and convex optimization, without making any assumption on the global network structure (i.e., no random models)

10. ONR MURI: NexGeNetSci I. Use algebraic graph theory to relate the frequency of certain small subgraphs, or motifs, with the so-called spectral moments of the network II. Propose a distributed technique to compute the frequency of subgraphs from the distribution of local network measurements III. Use convex optimization to extract relevant spectral information from a sequence of spectral moments IV. Study implications on dynamical processes Structure of our Approach

11. ONR MURI: NexGeNetSci Algebraic graph theory allows us to compute spectral moments from local structural information. We use the following result: Low-order moments: For k≤3 we have the following expressions I. From Subgraphs to Spectral Moments k=2 k=3 i i

12. ONR MURI: NexGeNetSci Higher-order moments: As we increase the order of the moments, a variety of more and more complicated subgraphs come into the picture. For k=4, we have the following types of closed walks: In the first expression, we observe that local measurements can be aggregated via distributed consensus to compute spectral moments From Subgraphs to Moments i i i j i Moments from local structural measurements Moments from subgraphs frequencies

13. ONR MURI: NexGeNetSci Key observation: The spectral moments are linear combinations of subgraphs embedding frequencies [Preciado, Jadbabaie, 2010]. The coefficients for all nonisomorphic connected subgraphs with 4 or less nodes are For example, the fifth moment can be computed as: mkmk k=424----8-- k=5--30--10--- k=621224126-4836- k=7--126--84-112- k=8228168723264264464528 Moments from Subgraphs Frequencies

14. ONR MURI: NexGeNetSci We propose a distributed technique to compute subgraph frequencies. Note that each subgraph can be ’discovered’ by a number of its nodes. For example, for 1 -hop neighborhoods: In general, if each node have access to its r -hops neighborhood, we can discover all subgraphs involved in moments of order up to 2r+1 (and part of the subgraphs involved in moments of higher order) II. Distributed Computation of Moments

15. ONR MURI: NexGeNetSci Subgraph with 2,404 nodes and 22,786 edges obtained from crawling the Facebook graph in a breadth-first search around a particular node We can compute the relevant quantities which allow us to compute moments Empirical Example

16. ONR MURI: NexGeNetSci So far, we have We now present an SDP-based approach to extract information from the spectral moments that 1.It is agnostic, in the sense that it does not make any assumption on the global network structure (no random model) 2.It allows to study the effect of arbitrarily complicated structural measurements in the network spectral properties III. Extracting Spectral Info from Moments Counting subgraph frequencies Computing spectral moments ?

17. ONR MURI: NexGeNetSci How can we extract information from spectral moments? The following problem, called the classical moment problem, is closely related to ours: Given a sequence of moments (m 1,…,m k ), and Borel measurable sets T   R, we are interested in computing where m in M(  ), M(  ) being the set of positive Borel measures supported by . Generalization of Markov and Chebyshev´s inequalities from probability theory, when a sequence of moments is available The Classical Moment Problem

18. ONR MURI: NexGeNetSci Using duality theory, we obtain the following formulation [Bertsimas, 2005] : This dual problem is a sum-of-squares program (SOSP) and can be formulated as a semidefinite program [Parrilo, 2006]. We define the spectral distribution of a graph as and define the r.v. X  G. Hence, using SOS, we can compute optimal bounds on Pr ( X  T )=#{ i  T } /n when we have access to a sequence of spectral moments Moment Problems, SOS and SDP

19. ONR MURI: NexGeNetSci From the set of spectral moments, we compute optimal bounds on #{ i  [ a,b ]}/ n, and #{ i  [ -c,c ]}/ n Notice that only those intervals [ a,b ] in region B and [ -c,c ] in C are able to support the whole set of eigenvalues. Hence, –We have a lower bound on the spectral radius  (A)>  * –We can also compute a bound on the Laplacian eigenratio from the Laplacian spectral moments Numerical Results a b B 1 ** c C

20. ONR MURI: NexGeNetSci IV Dynamical Implications: Spreading Processes We study a stochastic dynamical model of viral dissemination: - Each node has two possible states: 0. Susceptible (blue) 1. Infected (red) - Spreading parameters:  probability of contagion  probability of recovering Spectral results [Draief et al., 2008] : -  (A)>  /  is a necessary condition for a small infection to infect a significant part of the network - The larger  (A), the better a network disseminate a virus/rumor

21. ONR MURI: NexGeNetSci Spreading Processes: Simulations 0.05% initial infection  /  =35 < 45 <  (A)=60 0.2% initial infection  /  =65 >  (A)=60 Counting subgraph frequencies Computing spectral moments Bound on  (A)>45.0 Implications on Spreading # of infections tt

22. ONR MURI: NexGeNetSci Ongoing work: Design incentives for each individual to modify their local neighborhood in order to achieve a particular global spectral property. Some preliminaries results [Preciado et al., 2010]: Decentralized Network Design

23. ONR MURI: NexGeNetSci Adapt our framework to: 1.Evolving networks: Tracking the evolution of subgraphs frequencies and model their interactions 2.Links with weights and directions: Since the eigenvalues become complex, we have to work with 2D support 3.Nodes with attributes Incentive design: How can we drive nodes to take local actions that improve the global dynamical behavior of the network? Future Work

24. ONR MURI: NexGeNetSci Our work is devoted to study local structural properties and dynamical processes in large-scale complex networks There is a direct relationship between many dynamical processes in networks and the eigenvalues of the underlying graph There is plenty of information about the eigenvalue spectra from the distribution of local network measurements Our approach is agnostic, in which we do not assume any global structure (no random graphs) Our results can be of interest to analyze and design large- scale networks from a spectral point of view Conclusions

25. ONR MURI: NexGeNetSci F. Chung, L. Lu, and V. Vu, "The Spectra of Random Graphs with Given Expected Degrees," Internet Mathematics, vol. 1, pp. 257-275, 2003. M. Draief, A. Ganesh, and L. Massoulié, "Thresholds for Virus Spread on Networks," Annals of Applied Probability, vol. 18, pp. 359-378, 2008. L. Li, D. Alderson, J.C. Doyle, and W. Willinger, " Towards a Theory of Scale-Free Graphs,“ Interner Math, vol. 2, pp. 431-523, 2005. P. Parrilo, Algebraic Techniques and Semidefinite Optimization, Massachusetts Institute of Technology: MIT OpenCourseWare, Spring 2006. L.M. Pecora, and T.L. Carroll, "Master Stability Functions for Synchronized Coupled Systems," Physical Review Letters, vol. 80(10), pp. 2109-2112, 1998. I. Popescu and D. Bertsimas, "An SDP Approach to Optimal Moment Bounds for Convex Classes of Distributions," Mathematics of Operation Research, vol. 50, pp. 632-657, 2005. V.M. Preciado, and G.C. Verghese, "Synchronization in Generalized Erdös-Rényi Networks of Nonlinear Oscillators," IEEE Conference on Decision and Control, pp. 4628-4633, 2005. V.M. Preciado, and A. Jadbabaie, "Spectral Analysis of Virus Spreading in Random Geometric Networks," IEEE Conference on Decision and Control, pp. 4802-4807, 2009. V.M. Preciado, M.M. Zavlanos, A. Jadbabaie, and G.J. Pappas, “Distributed Control of the Laplacian Spectral Moments of a Network,” American Control Conference, 2010. V.M. Preciado and A. Jadbabaie, " From Local Measurements to Network Spectral Properties: Beyond Degree Distributions, " IEEE Conference on Decision and Control, 2010. Some References

26. ONR MURI: NexGeNetSci QUESTIONS?

27. ONR MURI: NexGeNetSci We study a collection of resistively coupled nonlinear oscillators IV.b Dynamical Implications: Synchronization   Spectral results [Pecora and Carrol]: L=D-A, and  i are its eigenvalues For stability of the synchronous state we need the Laplacian eigenration  n /  2 < , where  depends on the individual oscillator dynamics Network dynamics: Question: What values of  do make the network synchronize? 

28. ONR MURI: NexGeNetSci Using our SDP approach, we can also bound the Laplacian eigenratio from local structural properties via the Laplacian moments In the Laplacian moments, not only the frequencies of subgraphs are important, but also the degrees of the nodes involved. For example, for the 4th moment the following substructures are involved The Laplacian moments are functions of the frequencies of these structures and the degrees of the nodes involved: Laplacian Moments

29. ONR MURI: NexGeNetSci Synchronization: Simulations Counting frequencies of substructures Computing Laplacian moments Bound on  n /  2 Implications on Synchronization We simulate a network of 200 resistively coupled Rossler oscillators