Small-world Overlay P2P Network John C.S. Lui

Outline Describe our motivations on small world overlay P2P network. Introduce the background information of P2P network and small world network. Propose our Small-world Overlay Protocol (SWOP). Explain our Flash Crowd Handling Protocol Illustrate the experimental results

Motivation Fundamental Improvements in P2P network Improve the performance of Object Lookup in P2P Network Solve high traffic loading of a “popular” and “dynamic” object, i.e. under a Flash Crowd Scenario Small world is applied to achieve above criteria

Background --Structured P2P Network-- P2P network contains nodes (computers), which are acting as server as well as client. Structured P2P with two extra characteristics: Decentralized Structured To achieve this, consistent Distributed Hash Table (DHT) has been used. Two implementation issues for maintaining a logical structure: Unique key assignment scheme using DHT Characteristic routing table aims at reducing distance by at least half in each forwarding

Background --Structured P2P Network-- 1. Unique key assignment 1000 10000 217+100 217+10 212-1 6000 12000 216+1 224 x.x.x.b Item 1 x.x.x.a x.x.x.c Objects Item 3 Item 2 x.x.x.d x.x.x.e Nodes

Background --Structured P2P Network-- 2. Characteristic Routing Node 100 sends a message to node 217+100 Routing Table: … >217 : 217+10 …. >21 : 217+100 Item 1 Item 2 6000 1000 10000 12000 224 212-1 Item 3 Objects 216+1 217+100 217+10 Nodes

Background --Small world paradigm-- Small world networks represent two major properties: Two randomly chosen nodes are connected by short avg. distance Nodes are joined together in groups. The effect is called high clustering.

Main Ideas Apply Small world’s small average distance to structure P2P routing in order to improve the performance of object lookup Apply Small world’s high clustering coefficient to provide large traffic resolving solution

Small world Overlay Protocol (SWOP) -- Overview -- Small World Overlay Layer Topology Control Object Lookup Join Cluster (JCP) Leave Cluster (LCP) Object Lookup (OLP) DHT’s network

Small world Overlay Protocol (SWOP) -- Overview -- Terminologies Type of links Short links Long links Types of nodes Head nodes Inner nodes 31 1 30 2 29 3 28 4 27 5 26 6 25 24 7 23 8 22 9 21 10 20 11 19 12 18 17 13 16 14 15

Join Cluster Protocol Compute “distance” between predecessor and successor nodes Retrieve “group size” from nodes Select a group to join Update links information D1 = 3 D2 = 1 G1 = 2 G2 = 2 1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

Leave Cluster Protocol Contact short links neighbors for leaving If it is the “Head” node, hand over the short links neighbor and long links neighbors to next “Head”, and generate necessary new long links neighbors P.S. There exists boundary case, like only one node in a cluster. The solution is written in the thesis.

Leave Cluster Protocol Node 9 leaves Node 9 announces a new head 10 to 11 Node 10 gets the long links and short links Link from node 4 to node 9 fade out when this link being used by a lookup request. 31 1 30 2 29 3 28 4 27 5 26 6 25 24 7 23 8 22 9 21 10 20 11 19 12 18 17 13 16 14 15

Object Lookup Protocol Phase 1 Query short link neighbors if there exists one which contains desired object. If result is positive, the object lookup request ends. Otherwise, phase 2 begins. Phase 2 By using the “head” ‘s long link neighbor, forward the object lookup request to another cluster. Phase 1 continues by that node receives the object lookup request.

Object Lookup Example (1) Node 0 requests object 29, managed by node 31 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

Object Lookup Example (2) Node 0 requests object 16, managed by node 17 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

Performance analysis mathematical Worst case average link traversals E[X] ≤ (1 + log2(m/2)) (8ln(3m)/k) E[X] represents worst case expected number of link traversals m represents the number of clusters in the SWOP network k represents the number of long links The proof is conducted by randomized algorithm and it leaves in the thesis

Performance analysis simulation result Performance Metrics Probability density function of lookup hop count Simulation Setup We added nodes one by one and reform the topology according the construction protocol. Total about 1k-5k nodes were generated in the system. Each node generated data lookup request randomly certain times.

Performance analysis simulation result

Internet Flash Crowd Unpredictable huge amount of request for a popular object is generated towards the object owner This overwhelms the network and the CPU resources of the owner. e.g. CNN news server during 911

Types for Internet Flash Crowd Static Flash Crowd The popular object involved will remain unchanged after its first appearance, e.g. new movie. Dynamic Flash Crowd The popular object involved will change after its first appearance, e.g. news

Requirements for handling Internet Flash Crowd Loading Distributed Reduce the bottleneck by caching and replication Demand driven Replication scheme has to be demand driven, otherwise, it will be a flooding scheme. Dynamic compatibility Consider how to handle dynamic objects which are changed by original source

Protocols for handling Flash Crowd Algorithm Each node periodically records the access rates of objects stored. If the access rate of an object is greater than certain threshold, the owner of this object contacts the “head” of its cluster. “head” spread this object to all its long link neighbors

Protocols for handling Flash Crowd Each long link neighbor “caches” this object so that each neighbor can be acted as an image source of this object. Each long link neighbor keeps track of the rate of newly added object. Dynamic cases – Refresh Message Aims for reminding neighbor nodes to get the latest updated object.

Static Flash Crowd Example Assume, in node 4, an item’s hitting rate exceeds threshold 1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

Dynamic Flash Crowd Example 1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31 Original ver. New ver. Refresh msg.

Experiment results Study the performance on handling flash crowd scenario Performance metrics: Number of successful request In each interval, the nodes try to lookup the popular object. The metric counts the number of nodes can retrieve in this interval. Number of messages produced (traffic burden) Total number of messages in the system. Settings: Static flash crowd with fixed rate  req/s. Dynamic flash crowd with fixed rate  req/s with simulation time 25, 50 and 75 for object version update Dynamic flash crowd varying  with simulation time 25, 50 and 75 for object version update Static flash crowd with fixed rate  req/s

Experiment results Evaluation procedure We added 2000 nodes in the system using SWOP to form small world. One object was randomly chosen and was acted as the popular object. Each node generated that popular object’s lookup request traffic with rate . Each node has fixed  service rate and a queue for handling the item request.

Experimental results Static flash crowd

Experimental results Dynamic flash crowd

Experiment results Variation of object request rate

Experiment results Operation cost Replicating Region

Summary Build a protocol that applies Small world features on P2P network Improve the Object Lookup performance with the support of mathematical analysis. Propose an algorithm to handle massive traffics produced by “popular” and “dynamic” objects.

Thank you Questions & Answers

Small world Overlay Protocol Stabilize algorithm Method 1: Each node periodically probes their neighbors. Once a timeout event occurs (node failure) for a corresponding probing, the routing information is updated. Method 2: Each node does not perform a probing until an item lookup event occurs. When the lookup event fails, meaning there exists a node failure, the routing information is updated according to the failed node.

Experiment results Variation on number of long link neighbors k

Experiment results Variation of queue size

Mathematical Analysis Apply Markov Model Define each finite state as number of cached clusters Retrieve the transition probabilities Define Troop State Compute the troop state probability Calculate the expected time to troop state