Jellyfish, and other Interesting creatures Of the Internet Scott Kirkpatrick, Hebrew University with Avishalom Shalit, Sorin Solomon, Shai Carmi, Eran Shir, and Yuval Shavitt 25 May, 2005 Copyright, Pixar, Inc. 2003

– Internet topology map Previous efforts to measure the Internet have used: –One machine + Traceroute to many destinations –Many* machines, specially deployed to traceroute to many destinations * Many <= 50 because of management headaches Sites restricted to academic or gov’t labs, on network backbone General perception was that Law of Diminishing Returns has set in seems to have made a breakthroughhttp:// –Don’t manage machines, offer a very lightweight, limited purpose client, and collect its measurements centrally 100 – 1000 clients via word-of-mouth (Sep04 to April05) 1000 – 5000 clients achieved via press, slashdot, still in the geek community –(May05 -- ?) 5000 – clients in the general public by offering services in return –Somewhere in the client range, we have the network monitoring itself, and the possibility that it can also manage itself in real time.

Contribution distribution with 2K users, >3K agents

Exploring the DIMES AS-graph We consider the Internet at the level of its autonomous systems (ASes) Previous studies have used degree as indicator to decompose networks –In particular, the Faloutsos’ “jellyfish model” Identify core of network as maximal clique Shells around network labeled by hop count from core (a small world) Find unusual structure of peripheral sites We consider longer-range connectivity, using k-pruning. K-core, K-shell, and K-crusts result –K-shell is “derivative” of K-core, K-crust is union of K-shells –Near power-law structure of a new “inflow” region is observed –K-shells are not connected, but K-crusts have a giant cluster For Erdos-Renyi graphs, K-core is w.h.p. K-connected. For scale free? Result – focus attention on the capabilities of the inflow region, in support of P2P, chat, local traffic. Next steps – reachability is much harder than percolation.

How does original degree map into k-shell?

K-shell decomposition

K-shell decomposition, March 2005

K-shell decomposition, April 2005

May 20, slashdot doubles DIMES users

Now offering: monthly public stats

The K-core is at least K-connected

K-shell for network visualization Using LaNet-Vi

Michalis Faloutsos’ Jellyfish Highly connected nodes form the core Each Shell: adjacent nodes of previous shell, except 1- degree nodes Importance decreases as we move away from core 1-degree nodes hanging The denser the 1-degree node population the longer the stem Core Shells 1 2 3

Meduza (מדוזה) model In January, the inner core was at K = 30, but this picture persists to the present day, when core is >40.

K-crusts show percolation threshold Data from  These are the hanging tentacles of our (Red Sea) Jellyfish Largest cluster in each shell

Tendrils occur over a range of k-shells

Links per site of k-shells to k-core (above) and to k-crust (below)

Where do the links go in Medusa? Early shells (1-10) link to intermediate shells as well as to the core.

Average distance between sites in a crust

K-shell participation in giant clusters

Random scale-free graphs produce the same structure Seen in both Barabasi-style and Molloy-Reed models of scale free networks

Next steps New data permits reexamining the clustering behavior –Much data not seen in previous BGP-based studies –This is the major deviation from simple random models –Analyze as a function of k-shell, instead of simply degree Reachability is not percolation, but can be evaluated –Decision to transmit a message depends on sender and destination, not simply on the existence of a link –Cost of evaluating uphill-downhill reachability is comparable to shortest path

Preliminary reachability data (std data set)

Now add sideways steps at top of path

Now restrict to the 20-crust

Sideways step less effective inside crust