MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005.

Slides:



Advertisements
Similar presentations
Routing Complexity of Faulty Networks Omer Angel Itai Benjamini Eran Ofek Udi Wieder The Weizmann Institute of Science.
Advertisements

Mobile Communication Networks Vahid Mirjalili Department of Mechanical Engineering Department of Biochemistry & Molecular Biology.
Topologically-Aware Overlay Construction and Server Selection Sylvia Ratnasamy, Mark Handly, Richard Karp and Scott Shenker Presented by Shreeram Sahasrabudhe.
Modeling Malware Spreading Dynamics Michele Garetto (Politecnico di Torino – Italy) Weibo Gong (University of Massachusetts – Amherst – MA) Don Towsley.
It’s a Small World by Jamie Luo. Introduction Small World Networks and their place in Network Theory An application of a 1D small world network to model.
The Connectivity and Fault-Tolerance of the Internet Topology
VL Netzwerke, WS 2007/08 Edda Klipp 1 Max Planck Institute Molecular Genetics Humboldt University Berlin Theoretical Biophysics Networks in Metabolism.
Farnoush Banaei-Kashani and Cyrus Shahabi Criticality-based Analysis and Design of Unstructured P2P Networks as “ Complex Systems ” Mohammad Al-Rifai.
1 Evolution of Networks Notes from Lectures of J.Mendes CNR, Pisa, Italy, December 2007 Eva Jaho Advanced Networking Research Group National and Kapodistrian.
The structure of the Internet. How are routers connected? Why should we care? –While communication protocols will work correctly on ANY topology –….they.
Network Topology Julian Shun. On Power-Law Relationships of the Internet Topology (Faloutsos 1999) Observes that Internet graphs can be described by “power.
Degree correlations in complex networks Lazaros K. Gallos Chaoming Song Hernan A. Makse Levich Institute, City College of New York.
Web as Graph – Empirical Studies The Structure and Dynamics of Networks.
The Internet’s Dynamic Geography Scott Kirkpatrick, School of Engineering, Hebrew University of Jerusalem and EVERGROW Collaborators (thanks, not blame…)
Jellyfish, and other Interesting creatures Of the Internet Scott Kirkpatrick, Hebrew University with Avishalom Shalit, Sorin Solomon, Shai Carmi, Eran.
Common Properties of Real Networks. Erdős-Rényi Random Graphs.
Sampling from Large Graphs. Motivation Our purpose is to analyze and model social networks –An online social network graph is composed of millions of.
The Internet as a computing surface Avogadro-scale Computing MIT, April 17, 2008 Scott Kirkpatrick Hebrew University of Jerusalem and CBA, MIT.
On Power-Law Relationships of the Internet Topology CSCI 780, Fall 2005.
Advanced Topics in Data Mining Special focus: Social Networks.
Physical Mechanism Underlying Opinion Spreading
The structure of the Internet. How are routers connected? Why should we care? –While communication protocols will work correctly on ANY topology –….they.
The structure of the Internet. The Internet as a graph Remember: the Internet is a collection of networks called autonomous systems (ASs) The Internet.
How is this going to make us 100K Applications of Graph Theory.
On Distinguishing between Internet Power Law B Bu and Towsley Infocom 2002 Presented by.
The Internet’s Physical Topology (or, Will the Internet ever measure itself?)‏ Scott Kirkpatrick, School of Engineering, Hebrew University of Jerusalem.
The Erdös-Rényi models
Optimization Based Modeling of Social Network Yong-Yeol Ahn, Hawoong Jeong.
(Social) Networks Analysis III Prof. Dr. Daning Hu Department of Informatics University of Zurich Oct 16th, 2012.
Analysis and Modeling of the Open Source Software Community Yongqin Gao, Greg Madey Computer Science & Engineering University of Notre Dame Vincent Freeh.
Measuring and Analyzing Networks Scott Kirkpatrick Hebrew University of Jerusalem April 12, 2011.
Topic 13 Network Models Credits: C. Faloutsos and J. Leskovec Tutorial
1 Pertemuan 20 Teknik Routing Matakuliah: H0174/Jaringan Komputer Tahun: 2006 Versi: 1/0.
Shai Carmi Bar-Ilan, BU Together with: Shlomo Havlin, Chaoming Song, Kun Wang, and Hernan Makse.
Author: M.E.J. Newman Presenter: Guoliang Liu Date:5/4/2012.
Boundary Recognition in Sensor Networks by Topology Methods Yue Wang, Jie Gao Dept. of Computer Science Stony Brook University Stony Brook, NY Joseph S.B.
MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005.
Random-Graph Theory The Erdos-Renyi model. G={P,E}, PNP 1,P 2,...,P N E In mathematical terms a network is represented by a graph. A graph is a pair of.
Topology and Evolution of the Open Source Software Community Advisors: Dr. Vincent W. Freeh Dr. Kevin Bowyer Supported in part by the National Science.
Self-Similarity of Complex Networks Maksim Kitsak Advisor: H. Eugene Stanley Collaborators: Shlomo Havlin Gerald Paul Zhenhua Wu Yiping Chen Guanliang.
Social Network Analysis Prof. Dr. Daning Hu Department of Informatics University of Zurich Mar 5th, 2013.
Chapter 3. Community Detection and Evaluation May 2013 Youn-Hee Han
3. SMALL WORLDS The Watts-Strogatz model. Watts-Strogatz, Nature 1998 Small world: the average shortest path length in a real network is small Six degrees.
Mobile Agent Migration Problem Yingyue Xu. Energy efficiency requirement of sensor networks Mobile agent computing paradigm Data fusion, distributed processing.
Efficient Labeling Scheme for Scale-Free Networks The scheme in detailsPerformance of the scheme First we fix the number of hubs (to O(log(N))) and show.
Analyzing the Vulnerability of Superpeer Networks Against Attack Niloy Ganguly Department of Computer Science & Engineering Indian Institute of Technology,
Robustness and Structure of Complex Networks Shuai Shao Boston University, Physics Department A dvisor: Prof. Gene Stanley Co-advisor: Prof. Shlomo Havlin.
Class 9: Barabasi-Albert Model-Part I
Concept Switching Azadeh Shakery. Concept Switching: Problem Definition C1C2Ck …
Application of statistical physics to random graph models of networks Sameet Sreenivasan Advisor: H. Eugene Stanley.
Percolation and diffusion in network models Shai Carmi, Department of Physics, Bar-Ilan University Networks Percolation Diffusion Background picture: The.
Teknik Routing Pertemuan 10 Matakuliah: H0524/Jaringan Komputer Tahun: 2009.
Recent developments in the study of transport in random networks Shai Carmi Bar-Ilan University Havlin group Minerva meeting Eilat, March 2009.
Jellyfish, and other Interesting creatures Of the Internet Scott Kirkpatrick, Hebrew University with Avishalom Shalit, Sorin Solomon, Shai Carmi, Eran.
Brief Announcement : Measuring Robustness of Superpeer Topologies Niloy Ganguly Department of Computer Science & Engineering Indian Institute of Technology,
1 CIS 4930/6930 – Recent Advances in Bioinformatics Spring 2014 Network models Tamer Kahveci.
Transport in weighted networks: optimal path and superhighways Collaborators: Z. Wu, Y. Chen, E. Lopez, S. Carmi, L.A. Braunstein, S. Buldyrev, H. E. Stanley.
Informatics tools in network science
Urban Traffic Simulated From A Dual Perspective Hu Mao-Bin University of Science and Technology of China Hefei, P.R. China
Percolation Percolation is a purely geometric problem which exhibits a phase transition consider a 2 dimensional lattice where the sites are occupied with.
1 Patterns of Cascading Behavior in Large Blog Graphs Jure Leskoves, Mary McGlohon, Christos Faloutsos, Natalie Glance, Matthew Hurst SDM 2007 Date:2008/8/21.
Hierarchical Organization in Complex Networks by Ravasz and Barabasi İlhan Kaya Boğaziçi University.
Response network emerging from simple perturbation Seung-Woo Son Complex System and Statistical Physics Lab., Dept. Physics, KAIST, Daejeon , Korea.
A Place-based Model for the Internet Topology Xiaotao Cai Victor T.-S. Shi William Perrizo NDSU {Xiaotao.cai, Victor.shi,
Data Center Network Architectures
Groups of vertices and Core-periphery structure
Empirical analysis of Chinese airport network as a complex weighted network Methodology Section Presented by Di Li.
The Watts-Strogatz model
Department of Computer Science University of York
Introducing complex networks into quantum regime
Presentation transcript:

MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005

Who we are Talk prepared by Shai Carmi. Graduate student in the Department of Physics, Bar-Ilan University, Israel. Supervised by Prof. Shlomo Havlin, who gives the talk.

Who we are Collaborators – Prof. Scott Kirkpatrick, Hebrew University of Jerusalem, Israel. Dr. Yuval Shavitt, Tel-Aviv University, Israel. Eran Shir, Ph.D. Student, Tel- Aviv University, Israel. Scott

Measuring the Internet Previous efforts to measure the Internet have used : One machine + Traceroute to many destinations. Many machines, specially deployed to traceroute to many destinations Sites restricted to academic or gov’t labs, on network backbone General perception was that Law of Diminishing Returns has set in.

Measuring the Internet DIMES : Distributed Internet MEasurement and Simulations ( seems to have made a breakthrough. Don’t manage machines, offer a very lightweight, limited purpose client, and collect its measurements centrally. 100 – 1000 clients via word-of-mouth (Sep04 to Apr05). >5000 clients now, achieved via Science article, slashdot. 82 countries represented. 2-3 M measurements per day.

The network we analyze We consider the Internet at the level of its autonomous systems (ASes), with roughly 20,000 nodes and 70,000 links. We use data gathered between March and June In the future, can study network dynamics, using intervals of months, weeks or even days.

k-shell method Use recursive pruning to peel network layers. To remove the 1-shell, keep removing all nodes with one link (degree=1) until only nodes with degree 2 or more remain. To remove the 2-shell, keep removing nodes with 2 links, until all degrees are >= 3. Keep going until all nodes are removed.

k-shell method - example Original Graph

k-shell method - example Pruning Degree 1

k-shell method - example Keep Pruning Degree 1

k-shell method - example Keep Pruning Degree 1

k-shell method - example Pruning Degree 2

k-shell method - example Keep Pruning Degree 2

k-shell method - example Pruning Degree 3

k-shell method Definitions : k-Core – union of all shells with indices >= k. k-Crust – union of all shells with indices <= k.

Applications Can use k-shell method to analyze the AS network. For example, color each node by its shell index to visualize the network. Next, plot quantities as a function of the shell index. Gain understanding of the network structure. More useful indicator then the degree.

AS graph colored by shells

Identification of a nucleus k-shell method enables us to identify the heart, or nucleus of the network as nodes in the last core. No parameters need to be fixed. (Topology dependent only). Stable over time. Significant ASes (tier-1) were verified to be in the nucleus. Most quantities show singular behavior at the last shell. Some examples -

Number of nodes and degrees in the shells Slope= ~2.6

Centrality vs. shell

Where links go

Distances (vs. crusts) Distances measured between all pairs in the largest cluster of the crust

Number of site-distinct paths in the nucleus At least 41 distinct paths between each pair 41 is the k-shell index of the nucleus The nucleus is k-connected!

Beyond the nucleus It is left to understand the role of the other nodes in the network. We look at the connectivity properties of the crusts. Incorporate this with observations from previously shown plots.

Clusters in the crusts Percolation Threshold

Structure of the AS network Nodes outside of the nucleus can be categorized into – The fractal part – nodes in the largest cluster of the one-before-last crust – contains ~70% of the nodes in the network. The rest of the nodes become then the isolated part.

Properties of the fractal part Connected (by construction), so that routing is possible without traversing and congesting the nucleus. Connections to the nucleus decrease path lengths significantly. Show fractal properties and power-laws.

Properties of the fractal part Fractal dimension calculated using the ‘box cover method’ (SHM 2005). Crossover behavior between non-fractal and completely fractal – at the percolation critical point. Percolation theory arguments predict

Properties of the fractal part Percolation theory prediction – slope = 2.5 The 6-crust is renormalized with box of size 4

Properties of the isolated part Contains ~30% of the ASes, not reachable without the nucleus. Low degree nodes, high clustering. Many small clusters. Contributions found in up to k=10 shell. Many nodes are connected directly to highly connected nodes in the nucleus.

AS network model We summarize – the AS network is composed of 3 main sub-components : 1. Nucleus – Nodes in last shell. 2. Fractal Part – Nodes in the largest cluster of the one-before-last crust. 3. Isolated Part – Nodes in all-but-largest clusters of the one-before-last crust.  We name this model Medusa because of its jellyfish like structure.  Some similarities to Faloutsous’ Jellyfish model but important differences.

Medusa model of the AS network Our view of the Internet

The End. Thank you for your attention.

Comments Some properties (such as percolation) are found in the “Random-Scale-Free-Model” Internet might not be so special. To have more insight must investigate navigation with commercial restrictions – Many properties change.

Clustering coefficient vs. shell

Nearest neighbor degree vs. shell

Derivation of the fractal dimension At the threshold, almost all the high degree nodes are removed, such that the network becomes similar to a random (Erdos-Renyi) network. Percolation in random networks is equivalent to percolation in an infinite dimensional lattice, in which we know the fractal dimension of the largest component is 4,. For infinite dimensional lattices,. Thus we conclude,,or the ''shortest-path'' fractal dimension is 2.

Faloutsos ’ Internet jellyfish model The Jellyfish Model : Identify core of network as maximal clique. (  not a very robust or reproducible approach) Shells around network labeled by hop count from core (a small world) Find large portion of peripheral sites connect to core.

Faloutsos ’ Internet jellyfish model Faloutsos’ view of the internet