MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005

Who we are Talk prepared by Shai Carmi. Graduate student in the Department of Physics, Bar-Ilan University, Israel. Supervised by Prof. Shlomo Havlin, who gives the talk.

Who we are Collaborators – Prof. Scott Kirkpatrick, Hebrew University of Jerusalem, Israel. Dr. Yuval Shavitt, Tel-Aviv University, Israel. Eran Shir, Ph.D. Student, Tel- Aviv University, Israel. Scott

Measuring the Internet Previous efforts to measure the Internet have used : One machine + Traceroute to many destinations. Many machines, specially deployed to traceroute to many destinations Sites restricted to academic or gov’t labs, on network backbone General perception was that Law of Diminishing Returns has set in.

Measuring the Internet DIMES : Distributed Internet MEasurement and Simulations ( seems to have made a breakthrough. Don’t manage machines, offer a very lightweight, limited purpose client, and collect its measurements centrally. 100 – 1000 clients via word-of-mouth (Sep04 to Apr05). >5000 clients now, achieved via Science article, slashdot. 82 countries represented. 2-3 M measurements per day.

The network we analyze We consider the Internet at the level of its autonomous systems (ASes), with roughly 20,000 nodes and 70,000 links. We use data gathered between March and June In the future, can study network dynamics, using intervals of months, weeks or even days.

k-shell method Use recursive pruning to peel network layers. To remove the 1-shell, keep removing all nodes with one link (degree=1) until only nodes with degree 2 or more remain. To remove the 2-shell, keep removing nodes with 2 links, until all degrees are >= 3. Keep going until all nodes are removed.

k-shell method - example Original Graph

k-shell method - example Pruning Degree 1

k-shell method - example Keep Pruning Degree 1

k-shell method - example Keep Pruning Degree 1

k-shell method - example Pruning Degree 2

k-shell method - example Keep Pruning Degree 2

k-shell method - example Pruning Degree 3

k-shell method Definitions : k-Core – union of all shells with indices >= k. k-Crust – union of all shells with indices <= k.

Applications Can use k-shell method to analyze the AS network. For example, color each node by its shell index to visualize the network. Next, plot quantities as a function of the shell index. Gain understanding of the network structure. More useful indicator then the degree.

AS graph colored by shells

Identification of a nucleus k-shell method enables us to identify the heart, or nucleus of the network as nodes in the last core. No parameters need to be fixed. (Topology dependent only). Stable over time. Significant ASes (tier-1) were verified to be in the nucleus. Most quantities show singular behavior at the last shell. Some examples -

Number of nodes and degrees in the shells Slope= ~2.6

Centrality vs. shell

Where links go

Distances (vs. crusts) Distances measured between all pairs in the largest cluster of the crust

Number of site-distinct paths in the nucleus At least 41 distinct paths between each pair 41 is the k-shell index of the nucleus The nucleus is k-connected!

Beyond the nucleus It is left to understand the role of the other nodes in the network. We look at the connectivity properties of the crusts. Incorporate this with observations from previously shown plots.

Clusters in the crusts Percolation Threshold

Structure of the AS network Nodes outside of the nucleus can be categorized into – The fractal part – nodes in the largest cluster of the one-before-last crust – contains ~70% of the nodes in the network. The rest of the nodes become then the isolated part.

Properties of the fractal part Connected (by construction), so that routing is possible without traversing and congesting the nucleus. Connections to the nucleus decrease path lengths significantly. Show fractal properties and power-laws.

Properties of the fractal part Fractal dimension calculated using the ‘box cover method’ (SHM 2005). Crossover behavior between non-fractal and completely fractal – at the percolation critical point. Percolation theory arguments predict

Properties of the fractal part Percolation theory prediction – slope = 2.5 The 6-crust is renormalized with box of size 4

Properties of the isolated part Contains ~30% of the ASes, not reachable without the nucleus. Low degree nodes, high clustering. Many small clusters. Contributions found in up to k=10 shell. Many nodes are connected directly to highly connected nodes in the nucleus.

AS network model We summarize – the AS network is composed of 3 main sub-components : 1. Nucleus – Nodes in last shell. 2. Fractal Part – Nodes in the largest cluster of the one-before-last crust. 3. Isolated Part – Nodes in all-but-largest clusters of the one-before-last crust.  We name this model Medusa because of its jellyfish like structure.  Some similarities to Faloutsous’ Jellyfish model but important differences.

Medusa model of the AS network Our view of the Internet

The End. Thank you for your attention.

Comments Some properties (such as percolation) are found in the “Random-Scale-Free-Model” Internet might not be so special. To have more insight must investigate navigation with commercial restrictions – Many properties change.

Clustering coefficient vs. shell

Nearest neighbor degree vs. shell

Derivation of the fractal dimension At the threshold, almost all the high degree nodes are removed, such that the network becomes similar to a random (Erdos-Renyi) network. Percolation in random networks is equivalent to percolation in an infinite dimensional lattice, in which we know the fractal dimension of the largest component is 4,. For infinite dimensional lattices,. Thus we conclude,,or the ''shortest-path'' fractal dimension is 2.

Faloutsos ’ Internet jellyfish model The Jellyfish Model : Identify core of network as maximal clique. (  not a very robust or reproducible approach) Shells around network labeled by hop count from core (a small world) Find large portion of peripheral sites connect to core.

Faloutsos ’ Internet jellyfish model Faloutsos’ view of the internet