P2P Computing

2/48 What is peer-to-peer (P2P)? “Peer-to-peer is a way of structuring distributed applications such that the individual nodes have symmetric roles. Rather than being divided into clients and servers each with quite distinct roles, in P2P applications a node may act as both a client and a server.” -- Charter of Peer-to-peer Research Group, IETF/IRTF, June 24, 2004 (

3/48 Client/Server Architecture GET /index.html HTTP/1.0 HTTP/ OK... Client Server

4/48 Disadvantages of C/S Architecture Single point of failure Strong expensive server Dedicated maintenance (a sysadmin) Not scalable - more users, more servers

5/48 The Client Side Today’s clients can perform more roles than just forwarding users requests Today’s clients have:  more computing power  more storage space Thin client  Fat client

6/48 Evolution at the Client Side IBM 4.77MHz 360k diskettes A 4GHz 100GB HD DEC’S VT100 No storage ‘70‘

7/48 What Else Has Changed? The number of home PCs is increasing rapidly Most of the PCs are “fat clients” As the Internet usage grow, more and more PCs are connecting to the global net Most of the time PCs are idle How can we use all this?

8/48 Resources Sharing What can we share?  Computer resources Shareable computer resources:  CPU cycles - GIMPS  Data - Napster, Gnutella  Bandwidth - PPLive, PPStream  Storage Space - OceanStore, CFS, PAST

9/48 SETI – Search for ExtraTerrestrial – On your own computer A radio telescope in Puerto Rico scans the sky for radio signals Fills a DAT tape of 35GB in 15 hours That data has to be analyzed

10/48 (cont.) The problem – analyzing the data requires a huge amount of computation Even a supercomputer cannot finish the task on its own Accessing a supercomputer is expensive What can be done?

11/48 (cont.) Can we use distributed computing?  YEAH Fortunately, the problem can be solved in parallel - examples:  Analyzing different parts of the sky  Analyzing different frequencies  Analyzing different time slices

12/48 (cont.) The data can be divided into small segments A PC is capable of analyzing a segment in a reasonable amount of time An enthusiastic UFO searcher will lend his spare CPU cycles for the computation  When? Screensavers

13/48 - Example

14/48 - Summary SETI reverses the C/S model  Clients can also provide services  Servers can be weaker, used mainly for storage Distributed peers serving the center  Not yet P2P but we’re close Outcome - great results:  Thousands of unused CPU hours tamed for the mission  3+ millions of users

15/48

16/48

17/48

18/48 Napster -- Shawn Fanning

19/48

20/48 History of Napster (1/2) 5/99: Shawn Fanning (freshman, Northeastern University) founds Napster Online (supported by Groove) 12/99: First lawsuit 3/00: 25% Univ. of Wisconsin traffic on Napster

21/48 History of Napster (2/2) 2000: estimated 23M users 7/01: simultaneous online users 160K 6/02: file bankrupt … 10/03: Napster 2 (Supported by Roxio) (users should pay $9.99/month) 1984~2000, 23M domain names are counted vs. 16 months, 23M Napster-style names are registered at Napster

22/48 Napster Sharing Style: hybrid center+edge “slashdot” song5.mp3 song6.mp3 song7.mp3 “kingrook” song4.mp3 song5.mp3 song6.mp3 song5.mp3 1. Users launch Napster and connect to Napster server 3. beastieboy enters search criteria 4. Napster displays matches to beastieboy 2. Napster creates dynamic directory from users’ personal.mp3 libraries Title User Speed song1.mp3 beasiteboy DSL song2.mp3 beasiteboy DSL song3.mp3 beasiteboy DSL song4.mp3 kingrook T1 song5.mp3 kingrook T1 song5.mp3 slashdot 28.8 song6.mp3 kingrook T1 song6.mp3 slashdot 28.8 song7.mp3 slashdot beastieboy makes direct connection to kingrook for file transfer song5 “beastieboy” song1.mp3 song2.mp3 song3.mp3

23/48 Gnutella -- Justin Frankel and Tom Pepper

24/48 Gnutella History Gnutella was written by Justin Frankel, the 21-year-old founder of Nullsoft. (Nullsoft acquired by AOL, June 1999) Nullsoft (maker of WinAmp) posted Gnutella on the Web, March 14, A day later AOL yanked Gnutella, at the behest of Time Warner. Too late: 23k users on Gnutella People had already downloaded and shared the program. Gnutella continues today, run by independent programmers.

25/48 The ‘Animal’ GNU: Either of two large African antelopes (Connochaetes gnou or C. taurinus) having a drooping mane and beard, a long tufted tail, and curved horns in both sexes. Also called wildebeest. Gnutella = GNUGNU: Recursive Acronym GNU’s Not Unix …. + Nutella: a hazelnut chocolate spread produced by the Italian confectioner Ferrero …. GNU Nutella

26/48 GNU GNU's Not Unix 1983 Richard Stallman (MIT) established Free Software Foundation and Proposed GNU Project Free software is not freeware Free software is open source software GPL: GNU General Public License

27/48 About Gnutella No centralized directory servers Pings the net to locate Gnutella friends File requests are broadcasted to friends  Flooding, breadth-first search When provider located, file transferred via HTTP History:  3/14/00: release by AOL, almost immediately withdrawn

28/48 Peer-to-Peer Overlay Network Focus at the application layer

29/48 Peer-to-Peer Overlay Network Internet End systems one hop (end-to-end comm.) a TCP thru the Internet

30/48 Topology of a Gnutella Network

31/48 xyz.mp3 ? Gnutella: Issue a Request

32/48 Gnutella: Flood the Request

33/48 xyz.mp3 Gnutella: Reply with the File Fully distributed storage and directory!

34/48 So Far Centralized : - Directory size – O(n) - Number of hops – O(1) Flooded queries: - Directory size – O(1) - Number of hops – O(n) n: number of participating nodes

35/48 We Want Efficiency : O(log(n)) messages per lookup Scalability : O(log(n)) state per node Robustness : surviving massive failures

36/48 How Can It Be Done? How do you search in O(log(n)) time?  Binary search You need an ordered array How can you order nodes in a network and data objects?  Hash function!

37/48 Example of Hasing Shark SHA-1 Object ID (key):DE11AC SHA-1 Object ID (key):AABBCC :8080

38/48 Basic Idea Hash key Object “y” Objects have hash keys Peer “x” Peer nodes also have hash keys in the same hash space P2P Network yx H(y)H(x) Join (H(x)) Publish (H(y)) Place object to the peer with closest hash keys

39/48 Mapping Keys to Nodes 0 M - an data object - a node

40/48 Viewed as a Distributed Hash Table Hash table Peer node Internet

41/48 DHT Distributed Hash Table Input: key (file name) Output: value (file location) Each node is responsible for a range of the hash table, according to the node’s hash key. Objects are placed in (managed by) the node with the closest key It must be adaptive to dynamic node joining and leaving

42/48 How to Find an Object? Hash table Peer node

43/48 Simple Idea Track peers which allow us to move quickly across the hash space  a peer p tracks those peers responsible for hash keys (p+2 i -1), i=1,..,m Hash table Peer node i+2 2 i+2 4 i+2 8 i

44/48 DHT example: Chord -- Ring Structure Circular 6-bit ID space N8 knows of only six other nodes. O(log n) states per node

45/48 Chord Lookup – with finger table StartInt.node 3[3,4)3 4[4,6)7 6[6,10)7 10[10,2) I’m node 2. Please find key 14! StartInt.node 11[11,12)12 [12,14)12 14[14,2)14 2[2,10)2 O(log n) hops (messages) for each lookup!! 14 ∈ [10,2) 14 ∈ [14,2) Circular 4-bit ID space

46/48 Hybrid P2P – Preserves some of the traditional C/S architecture. A central server links between clients, stores indices tables, etc  Napster Unstructured P2P – no control over topology and file placement  Gnutella, Morpheus, Kazaa, etc Structured P2P – topology is tightly controlled and placement of files are not random  Chord, CAN, Pastry, Tornado, etc Classification of P2P systems

47/48 What else in the future? P2P NEVs (Networked Virtual Environments)  P2P MMOGs (Massively Multiplayer Online Games)  P2P 3D Scene Streaming P2P Mobile Ad hoc Games

48/48 Q&A