Download presentation
Presentation is loading. Please wait.
1
Copyright notice © 2008 Raul Jimenez - - This work is licensed under the Creative Commons Attribution 2.5 Sweden License. To view a copy of this license, visit or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. Copy, distribution, and reuse of this work is permitted, so long as the redistributed copy displays a link to The logos displayed on the title page belong to KTH and the P2P-Next consortium, respectively. The snapshot displayed on slide 4 is part of the animation movie “elephants dream” ( The figure displayed on slide 21 is part of the technical report referenced on slide 22. Although not mandatory, the author would be glad to hear from those who find this material useful. Of course, any comment which could complement and/or improve this work is very welcome.
2
Decentralized peer discovery performance in swarm-protocols
Raul Jimenez KTH, Sweden tslab.ssvl.kth.se/raul
3
Overview BitTorrent Content Discovery Data Exchange Peer Discovery
Centralized Tracker Decentralized Tracker (DHT) Gossiping (PEX) DHT Issues NAT effects Experiments
4
BitTorrent Data Exchange Content Discovery Peer Discovery
CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments BitTorrent Content Discovery Data Exchange Peer Discovery blender_foundation_-_ elephants_dream.torrent info_hash: c70bcf4657cac1253d12 acda95b04d1e4dcd23f4 Tracker (updated list of peers): 7070/announce
5
Decentralized Tracker (DHT) Gossiping (PEX)
BitTorrent BitTorrent CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Content Discovery Data Exchange Peer Discovery Centralized Tracker Decentralized Tracker (DHT) Gossiping (PEX) A tracker keeps track of every peer participating in the swarm Gossiping protocol Disconnected graphs Messages between peers Needs bootstrap Complements tracker
6
Centralized Tracker 1 P1 announces Tracker tracks P1
BitTorrent Centralized Tracker CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se peers: 1 P1 announces Tracker tracks P1 Tracker returns list
7
Centralized Tracker 1 2 P2 announces Tracker tracks P2
BitTorrent Centralized Tracker CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se 2 peers: 1 2 P2 announces Tracker tracks P2 Tracker returns list P2 contacts peers
8
Centralized Tracker 1 2 P3 announces Tracker tracks P3 3
BitTorrent Centralized Tracker CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se 2 peers: 1 2 3 P3 announces Tracker tracks P3 3 Tracker returns list P3 contacts peers
9
Decentralized Tracker (DHT) Gossiping (PEX)
BitTorrent BitTorrent CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Content Discovery Data Exchange Peer Discovery Centralized Tracker Decentralized Tracker (DHT) Gossiping (PEX) DHT Issues NAT effects Experiments Lookup times Dead-end lookups NAT behavior NAT types Effects on the DHT Measuring reachability with different methods
10
Distributed Tracker Distributed Hash Table
BitTorrent Distributed Tracker Distributed Hash Table CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Kademlia DHT (Mainline implementation) Totally decentralized - bootstrap? O(Log(n)) routing table size O(Log(n)) hops to find the list of peers Iterative routing algorithm Tracks every peer in the BitTorrent swarm
11
Distributed Hash Table (DHT)
BitTorrent Distributed Hash Table (DHT) CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Simple demo 4-bits identifiers (24 = 16 identifiers) 1 node per bucket 10 nodes Real world 160-bit identifiers 8 nodes per bucket 1 million nodes ?
12
DHT: Routing Table 11xx 0xxx 101x Routing table Level 0 0010 Level 1
BitTorrent DHT: Routing Table CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 11xx 0xxx 101x Routing table Level 0 0010 Level 1 1111 1001 Level 2 1011 1000 Level 3 ------ MyNode 1001
13
DHT: Find Peers 0000 0010 0011 0100 Level 0 0010 0101 torrentID
BitTorrent DHT: Find Peers CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 0000 0010 0011 0100 Level 0 0010 0101 torrentID Level 1 1111 Level 2 1011 Level 3 MyNode 1001 0110
14
DHT: Find Peers 0000 Level 0 1001 Level 1 0110 0010 Level 2 0000
BitTorrent DHT: Find Peers CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 0000 Level 0 1001 Level 1 0110 0010 Level 2 0000 Level 3 0011 0011 0100 0101 torrentID MyNode 1001 0110
15
DHT: Find Peers 0000 0010 0011 010x 0100 0101 torrentID 01x Level 0
BitTorrent DHT: Find Peers CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 0000 0010 0011 010x 0100 0101 torrentID 01x Level 0 1101 Level 1 0000 MyNode 1001 0110 Level 2 0100 Level 3
16
DHT: Find Peers 0000 0010 Level 0 1100 0011 Level 1 0011 Level 2 0110
BitTorrent DHT: Find Peers CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 0000 0010 Level 0 1100 0011 Level 1 0011 Level 2 0110 Level 3 0101 0100 0101 torrentID MyNode 1001 0110
17
DHT Issues Long lookup times (around a minute)
BitTorrent DHT Issues CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Long lookup times (around a minute) Especially long at the end of the lookup
18
Reported causes 20% dead nodes in 95% of the lookups
BitTorrent Reported causes CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 20% dead nodes in 95% of the lookups Too many dead nodes in the routing table: Levels 0-15: 15% dead nodes Levels 15-18: 65% dead nodes 57% of the nodes suffer “infant mortality” Reference: Scott A Crosby and Dan S Wallach An Analysis of BitTorrent's Two Kademlia-Based DHTs Technical Report TR-07-04, Department of Computer Science, Rice University, June 2007
19
Are reachability issues (most likely caused by NAT boxes)
BitTorrent CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Are reachability issues (most likely caused by NAT boxes) the main cause of BitTorrent's poor DHT performance?
20
Network Address Translation (NAT)
BitTorrent Network Address Translation (NAT) CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments NAT open partially open NAT temporarily open NAT timeout, or full table, or ...
21
BitTorrent 1 Peer discovery No peers to contact tracker.kth.se peers:
CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se peers: 1 Peer discovery No peers to contact
22
BitTorrent 1 2 Peer discovery P2 contacts P1 tracker.kth.se peers: 1 2
CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se 2 peers: 1 2 Peer discovery P2 contacts P1
23
BitTorrent 1 2 Peer discovery P3 contacts P1 3 P3 contacts P2
CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 tracker.kth.se 2 peers: 1 2 3 Peer discovery P3 contacts P1 3 P3 contacts P2
24
BitTorrent 1 4 2 Peer discovery P4 contacts P1 3 P4 contacts P2
CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 1 4 tracker.kth.se 2 peers: 1 2 3 4 Peer discovery P4 contacts P1 3 P4 contacts P2 P4 contacts P3
25
Temporal and partial reachability in DHT
BitTorrent Temporal and partial reachability in DHT CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Higher (apparent) churn Broken routes
26
Higher (apparent) churn
BitTorrent Higher (apparent) churn CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Churn rate: number of nodes joining and leaving the DHT during a given period. A new node joins Check reachability Level x N2 Add to the routing table 1 2 Dead node (remove) A new node joins ...
27
Broken routes Node 1 finds node 2 N1 checks N2's reachability
BitTorrent Broken routes CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Node 1 finds node 2 Level x N2 N1 checks N2's reachability N1 adds N2 to routing table 1 2 N3 sends message to N1 N2 is the next hop N2 is unreachable from N3 3
28
Experiments 3 PCs running DHT and local 1 PC running remote
BitTorrent Experiments CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments 3 PCs running DHT and local 1 PC running remote nodes checked DHT local remote
29
Experiments Check candidate node's reachability
BitTorrent Experiments CD PD DEX Tracker DHT PEX DHT Issues NAT Experiments Check candidate node's reachability from the DHT (same IP, same port) opening a new connection (same IP, different port) from another host (different IP) 33% U 61% U 63% U
30
What to do? Prevent my routing table from pollution
Check for global reachability before adding a node Don't pollute others' routing tables When I'm aware of being partially reachable, tell others: “don't add me to your routing table” Free riders?
31
Thank you!
Similar presentations
