On the Many Facets of Neighbor Selection for Overlay Networks Nikolaos Laoutaris Researcher (aka “ Investigador ” ) Telefonica Research, Barcelona, Spain Homepage: Northeastern University, 10 Dec – Boston, MA

Georgios Smaragdakis Azer Bestavros John Byers Shang-Hua Teng Ravi Sundaram Rajmohan Rajaraman Mema Roussopoulos Joint work with: Documented in: -- Swarming on Optimized Graphs for N-way Broadcast IEEE INFOCOM Implications of Selfish Neighbor Selection in Overlay Networks IEEE INFOCOM A Bounded-Degree Network Formation Game arXiv-CoRR cs.GT/ EGOIST: Overlay Routing using Selfish Neighbor Selection BUCS-TR , Oct Uplink Allocation Beyond Choke/Unchoke or Why Divide Does Not Always Conquer Best submitted, Nov Pietro Michiardi Damiano Carra

3/33 Neighbors are important just find a way to get along and move on … in other situations neighbors can be changed … but it has a cost … 1. net admin 2. business development3. lawyers 4. London Internet Exchange5. net engineer Telco ’ s can change neighbors it only takes 1-2 years

4/33 Patching the Internet using Overlay Networks for routing (RON, Detour, QRON) for file sharing (Napset, Gnutella, KaZaA, BitTorrent) for Internet telephony (Skype) for IP TV (Joost) Online Games

5/33 Neighbors can be switched fast in an overlay net overlay node physical node (e.g., end-systems, router or AS) overlay link physical link An overlay network: nodes & (logical) overlay links weight ~ physical dist. messages routed on the overlay Neighbor selection: choose overlay nodes for the establishment of direct links O1O1 O2O2 O3O3 R1R1 R2R2 R3R3 R4R4 Goal: use neighbor selection handle network dynamics handle participation dynamics

6/33 5 case studies 1.Overlay routing with selfish nodes 2.Unstructured P2P with selfish nodes 3.BitTorrent-like (swarming) P2P 4.All-to-all broadcasting of bulk data 5.Network-initiated redirection of P2P

7/33 A design plane for neighbor selection scope of communication routing 1-to-1 P2P 1-to-many broadcasting many-to-many scope of utility selfish nodes cooperative nodes decision maker the network the node routing imperfect perfect no routing Shortest-pathChordFlooding/BitTorrent available information no information local network wide bootstrappingpings link-state protocol

8/33 Case study 1: Overlay Routing

9/33 Neighbor selection for Overlay Routing scope of communication routing P2Pbroadcasting available information no information local network wide scope of utility selfish nodes cooperative nodes decision maker the network the node routing imperfect perfect no routing

10/33 A selfish node in a (n choose k) situation vivi G -i =(V -i,S -i ) u w s i ={u,w} individual wiring S=S -i +{s i } global wiring residual wiring v i wants to minimize: over all s i  S i v i ’ s preference for v j v i ’ s residual network v i ’ s residual wiring pick k neighbors

11/33 How is the optimal comparing with simple heuristics? k-Random does not use any link information k-Closest uses only local information Under uniform overlay link weights (hop-count distance): Optimal selection  asymmetric k-median on the reversed distance function of the residual graph G -i : Consequently: Best-response is NP-hard Const. factor approx for metric k-median don ’ t apply here O(1)-approx with O(logn) blow-up in # medians (Lin and Vitter, ’ 92) Most likely the best we can do (Archer, 2000) expensive cheap d 12 <d 13 <d 14 How do you select neighbors optimally? u w since these cost the same w,u can be obtained from 2-median on reversed distances w u wrong right

12/33 Social Cost(ourEQ) < 2 * Social Cost(SO) Existence of pure Nash equilibria and performance uniform game  uniform preference, link budget (k), and link weights (1) Theorem: All (n,k)-uniform games have pure Nash equilibria. Theorem: There exist non-uniform games with no pure Nash equilibria. there exist asymmetric non-uniform games that have no pure Nash (we “implemented” on a graph the cost-structure of the matching-pennies game) there exists an equivalent symmetric non-uniform game for each one of them Theorem: Strong connectivity in O(n 2 ) turns from any initial state. Lemma: In any stable graph for the (n, k)-uniform game, the cost of any node is at most 2 + 1/k + o(1) times the cost of any other node. Lemma: The diameter of any stable graph for an (n, k)-uniform game is O(sqrt(n log k n)). [don ’ t know if it is tight] Theorem: For any k ≥ 2, no Abelian Cayley graph with degree k and n nodes is stable, for n ≥ c2 k, for a suitably large constant c.

13/33 EGOIST: Our prototype overlay routing system for 50 nodes around the world using the infrastructure of n1 n2n3 n4n5 n6 n7 n8 n9 n10 n11 Connecting a newcomer node v i bootstrap listen to overlay link-state protocol to get d G-i get d ij ’ s through active ( ping ) or passive measurmnt ( Pyxida,pathChirp ) wire according to (hybrid) Best-Response monitor and announce your links check it out live at:

14/33 EGOIST: Features supported metrics: delay (active/passive monitoring with ping / Pyxida ) available bandwidth (monitoring with pathChirp ) node load (monitoring with loadavg ) supported wiring strategies: k-Random k-Closest k-Regular Best-Response (delay and AvailBw formulations) Hybrid Best-Response BR-computation based on the full residual graph or just samples of it

15/33 Case study 2: Unstructured P2P (aka Gnutella or 1 st generation P2P)

16/33 The 1-slide intro to Gnutella random list of neighbors from bootstrap server flooding of queries first node that “ hits ” sends back the file

17/33 Neighbor selection for unstructured P2P scope of communication routing P2Pbroadcasting available information no information local network wide scope of utility selfish nodes cooperative nodes decision maker the network the node routing imperfect perfect no routing flooding

18/33 Current Gnutella in Gnutella neighbors are picked randomly or with local info Bootstrap Server if the TTL of scoped flooding is 2 this wiring will give you access to 4 nodes randomly

19/33 Gnutella with selfish nodes IDEA: we select neighbors to maximize the foothold of our scoped flooding Bootstrap Server for each candidate we compute the number of distinct nodes we can reach with TTL 2 this wiring gives us access to 6 nodes connect to those candidates that maximize the overall sum of uniques 5 6

20/33 Case study 3: BitMax (our hacked BitTorrent; 2 nd generation P2P)

21/33 The 1-slide intro to BitTorrent file broken to 256Kbyte chunks a tracker knows all the peers of a torrent and the seed 1 torrent  1 file new leacher peers get 40 random neighbors neighbors know each other ’ s chunks help each other with missing ones chunk selection: the Least Replicated First (LRF) algorithm request a missing chunk that is less represented uplink allocation: the choke/unchoke algorithm unchoke k=4 neighbors assigning equal share of the uplink capacity … the ones that have been uploading fastest to you rate-based tit-for-tat

22/33 WHY 4 ??? ask its inventor Bram Cohen he ’ ll give an amusing answer: … anyway, people say “ 4 is good ” … measurement studies: “ 90+% uplink utilization ” theory works: “ some contact process was asymptotically optimal under a ton of simplifications ” our simulations were showing otherwise … especially under real constraints: and we fixed most of our bugs and verified against a second simulator

23/33 Since when tit-for-tat became socially OPT? the 3 node example 1 seed, 2 leachers, 1 chunk of size 1, all upload rates=1 splitting gets the chunk delivered after 2 time slots focusing and then switching achieves the same … but the first leacher can start contributing earlier Lesson learned : you must saturate your uplink but partition it minimally allocated shares don ’ t have to be equal seedleachers A A B B

24/33 Using a fractional knapsack to get rid of the magic 4 a knapsack with capacity u(v) “ item ” v i \in V(v) with “ weight ” u(v,v i ) “ value ” U(v i,h) the value of unchoking v i for a horizon h time rate

25/33 simulator and scripts (will be) available online us them Here: median download time is halved worse case reduced by a fact. of 3

26/33 Case study 4: N-way broadcast

27/33 A native n 2 application each node has a large file upload it to all others download their respective ones bulk synchronous parallel processing n independent overlays is a BAD idea race conditions redundant re-monitoring should use one optimized overlay for all exchanges random neighborhoods is a BAD idea scale is small/mid should use more global info

28/33 Neighbor selection for N-way broadcast scope of communication routing P2Pbroadcasting available information no information local network wide scope of utility selfish nodes cooperative nodes decision maker the network the node routing imperfect perfect no routing swarming

29/33 Neighbor selection & broadcasting W=k= 4, 6, 8… we did this before BitMax node v i and a flow network G -i v i selects neighbor set V(v i ) to maximize its broadcast bandwidth (its minimum maxflow to any v j \in G -i ) cooperative node: forwards own and in-transit chunks indiscriminately selfish node: forwards its own nodes preferentially we have ways to detect and punish

30/33 Case study 5: P2P redirection at ISP peering points (only 2 slides left!)

P2P traffic can go through unwanted links 31 SPRINT/LEVEL3 Access ISP Telefonica Backbone Tel Spain Tel Brasil Other ISP Backbone Transit ISP Backbone Access ISPs Transit ($$$) Peering (zero cost) P2P

32/33 The “ box ” (codename Athena project) intercept BitTorrent ’ s signalling re-write the neighbor lists returned by the tracker can we both lower the cost for the ISP And improve the download times ?

33/33 Thank you Q ?

34/33 Social Cost(ourEQ) < 2 * Social Cost(SO) Existence of pure Nash equilibria and performance uniform game  uniform preference, link budget (k), and link weights (1) Theorem: All (n,k)-uniform games have pure Nash equilibria. Theorem: There exist non-uniform games with no pure Nash equilibria. there exist asymmetric non-uniform games that have no pure Nash (we “implemented” on a graph the cost-structure of the matching-pennies game) there exists an equivalent symmetric non-uniform game for each one of them Theorem: Strong connectivity in O(n 2 ) turns from any initial state. Lemma: In any stable graph for the (n, k)-uniform game, the cost of any node is at most 2 + 1/k + o(1) times the cost of any other node. Lemma: The diameter of any stable graph for an (n, k)-uniform game is O(sqrt(n log k n)). [don ’ t know if it is tight] Theorem: For any k ≥ 2, no Abelian Cayley graph with degree k and n nodes is stable, for n ≥ c2 k, for a suitably large constant c. …… …... … … h k …… k … t’t’ t t=n-n k,h t ’ =t+1 mod k gets either k roots or the k hubs gets t ’ roots and k-t ’ heaviest hubs unstable for large n …… …... … … h k k-1 k utopian disconnection utopian+1 disconnection