1. Small-world Overlay P2P Network
John C.S. Lui

2. Outline Describe our motivations on small world overlay P2P network.
Introduce the background information of P2P network and small world network.
Propose our Small-world Overlay Protocol (SWOP).
Explain our Flash Crowd Handling Protocol
Illustrate the experimental results

3. Motivation Fundamental Improvements in P2P network
Improve the performance of Object Lookup in P2P Network
Solve high traffic loading of a “popular” and “dynamic” object, i.e. under a Flash Crowd Scenario
Small world is applied to achieve above criteria

4. Background --Structured P2P Network--
P2P network contains nodes (computers), which are acting as server as well as client.
Structured P2P with two extra characteristics:
Decentralized
Structured
To achieve this, consistent Distributed Hash Table (DHT) has been used.
Two implementation issues for maintaining a logical structure:
Unique key assignment scheme using DHT
Characteristic routing table aims at reducing distance by at least half in each forwarding

5. Background --Structured P2P Network--
1. Unique key assignment
1000
10000
217+10
212-1
6000
12000
216+1
224
x.x.x.b
Item 1
x.x.x.a
x.x.x.c
Objects
Item 3
Item 2
x.x.x.d
x.x.x.e
Nodes

6. Background --Structured P2P Network--
2. Characteristic Routing
Node 100 sends a message to node
Routing Table: …
>217 : ….
>21 :
Item 1
Item 2
6000
1000
10000
12000
224
212-1
Item 3
Objects
216+1
217+10
Nodes

7. Background --Small world paradigm--
Small world networks represent two major properties:
Two randomly chosen nodes are connected by short avg. distance
Nodes are joined together in groups. The effect is called high clustering.

8. Main Ideas
Apply Small world’s small average distance to structure P2P routing in order to improve the performance of object lookup
Apply Small world’s high clustering coefficient to provide large traffic resolving solution

9. Small world Overlay Protocol (SWOP) -- Overview --
Small World Overlay Layer
Topology Control
Object Lookup
Join Cluster (JCP)
Leave Cluster (LCP)
Object Lookup (OLP)
DHT’s network

10. Small world Overlay Protocol (SWOP) -- Overview --
Terminologies
Type of links
Short links
Long links
Types of nodes
Head nodes
Inner nodes 31 1 30 2 29 3 28 4 27 5 26 6 25 24 7 23 8 22 9 21 10 20 11 19 12 18 17 13 16 14 15

11. Join Cluster Protocol
Compute “distance” between predecessor and successor nodes
Retrieve “group size” from nodes
Select a group to join
Update links information
D1 = 3
D2 = 1
G1 = 2
G2 = 2 1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

12. Leave Cluster Protocol
Contact short links neighbors for leaving
If it is the “Head” node, hand over the short links neighbor and long links neighbors to next “Head”, and generate necessary new long links neighbors
P.S. There exists boundary case, like only one node in a cluster.
The solution is written in the thesis.

13. Leave Cluster Protocol
Node 9 leaves
Node 9 announces a new head 10 to 11
Node 10 gets the long links and short links
Link from node 4 to node 9 fade out when this link being used by a lookup request. 31 1 30 2 29 3 28 4 27 5 26 6 25 24 7 23 8 22 9 21 10 20 11 19 12 18 17 13 16 14 15

14. Object Lookup Protocol
Phase 1
Query short link neighbors if there exists one which contains desired object.
If result is positive, the object lookup request ends. Otherwise, phase 2 begins.
Phase 2
By using the “head” ‘s long link neighbor, forward the object lookup request to another cluster.
Phase 1 continues by that node receives the object lookup request.

15. Object Lookup Example (1) Node 0 requests object 29, managed by node 3115 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

16. Object Lookup Example (2) Node 0 requests object 16, managed by node 1715 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

17. Performance analysis mathematical
Worst case average link traversals
E[X] ≤ (1 + log2(m/2)) (8ln(3m)/k)
E[X] represents worst case expected number of link traversals
m represents the number of clusters in the SWOP network
k represents the number of long links
The proof is conducted by randomized algorithm and it leaves in the thesis

18. Performance analysis simulation result
Performance Metrics
Probability density function of lookup hop count
Simulation Setup
We added nodes one by one and reform the topology according the construction protocol.
Total about 1k-5k nodes were generated in the system.
Each node generated data lookup request randomly certain times.

19. Performance analysis simulation result

20. Internet Flash Crowd
Unpredictable huge amount of request for a popular object is generated towards the object owner
This overwhelms the network and the CPU resources of the owner.
e.g. CNN news server during 911

21. Types for Internet Flash Crowd
Static Flash Crowd
The popular object involved will remain unchanged after its first appearance, e.g. new movie.
Dynamic Flash Crowd
The popular object involved will change after its first appearance, e.g. news

22. Requirements for handling Internet Flash Crowd
Loading Distributed
Reduce the bottleneck by caching and replication
Demand driven
Replication scheme has to be demand driven, otherwise, it will be a flooding scheme.
Dynamic compatibility
Consider how to handle dynamic objects which are changed by original source

23. Protocols for handling Flash Crowd
Algorithm
Each node periodically records the access rates of objects stored.
If the access rate of an object is greater than certain threshold, the owner of this object contacts the “head” of its cluster.
“head” spread this object to all its long link neighbors

24. Protocols for handling Flash Crowd
Each long link neighbor “caches” this object so that each neighbor can be acted as an image source of this object.
Each long link neighbor keeps track of the rate of newly added object.
Dynamic cases – Refresh Message
Aims for reminding neighbor nodes to get the latest updated object.

25. Static Flash Crowd Example
Assume, in node 4, an item’s hitting rate exceeds threshold 1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31

26. Dynamic Flash Crowd Example1 15 8 2 3 7 6 5 4 9 14 13 12 11 10 23 16 17 18 19 20 21 22 24 25 26 27 28 29 30 31
Original ver.
New ver.
Refresh msg.

27. Experiment results
Study the performance on handling flash crowd scenario
Performance metrics:
Number of successful request
In each interval, the nodes try to lookup the popular object. The metric counts the number of nodes can retrieve in this interval.
Number of messages produced (traffic burden)
Total number of messages in the system.
Settings:
Static flash crowd with fixed rate  req/s.
Dynamic flash crowd with fixed rate  req/s with simulation time 25, 50 and 75 for object version update
Dynamic flash crowd varying  with simulation time 25, 50 and 75 for object version update
Static flash crowd with fixed rate  req/s

28. Experiment results Evaluation procedure
We added 2000 nodes in the system using SWOP to form small world.
One object was randomly chosen and was acted as the popular object.
Each node generated that popular object’s lookup request traffic with rate .
Each node has fixed  service rate and a queue for handling the item request.

29. Experimental results Static flash crowd

30. Experimental results Dynamic flash crowd

31. Experiment results Variation of object request rate

32. Experiment results Operation cost
Replicating Region

33. Summary
Build a protocol that applies Small world features on P2P network
Improve the Object Lookup performance with the support of mathematical analysis.
Propose an algorithm to handle massive traffics produced by “popular” and “dynamic” objects.

34. Thank you
Questions & Answers

35. Small world Overlay Protocol Stabilize algorithm
Method 1:
Each node periodically probes their neighbors. Once a timeout event occurs (node failure) for a corresponding probing, the routing information is updated.
Method 2:
Each node does not perform a probing until an item lookup event occurs. When the lookup event fails, meaning there exists a node failure, the routing information is updated according to the failed node.

36. Experiment results Variation on number of long link neighbors k

37. Experiment results Variation of queue size

38. Mathematical Analysis
Apply Markov Model
Define each finite state as number of cached clusters
Retrieve the transition probabilities
Define Troop State
Compute the troop state probability
Calculate the expected time to troop state