1. COMP/ELEC 429 Introduction to Computer Networks
Overlay networks
Slides used with permissions from Edward W. Knightly, T. S. Eugene Ng, Ion Stoica, Hui Zhang
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

2. Abstract View of the Internet
A collection of IP routers and point-to-point physical links connecting routers
Point-to-point links between two routers are physically as direct as possible
A copper wire, a coax cable or a fiber laid from one router to another
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

3. Reality Fibers and wires are laid with tremendous physical constraints
You can’t just dig up the ground everywhere and lay fibers
Right-of-way issue
Most fibers are laid along railroads
Physical fiber topology often very far from the topology you want
Internet is over-laid on top of this physical fiber topology
Internet topology is only logical!
Revelation: Internet is an overlay network
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

4. National Lambda Rail Project – Fiber Topology
IP logical link
Circuit
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

5. T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

6. Overlay Made Possible by Layer Abstraction
Layering hides the detail of lower layer from higher layer
IP operates on datalink layer (say ATM or SONET) logical topology
ATM or SONET creates point-to-point circuits on the fibers
Host A
Host B
Application
Application
Router
Transport
Transport
Network
Network
Network
Datalink
Datalink
Datalink
Physical
Physical
Physical
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

7. Overlay Overlay is clearly a general concept
You can keep overlaying one network on another, it’s all logical
IP Internet overlays on top of physical topology
Why stop here?
Something else can overlay on top of IP Internet
Use IP tunnel (or IP encapsulation) to create logical links for yet another logical topology
E.g. VPN link from home to Rice’s campus network
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

8. Advanced Reasons to Overlay on IP Internet
IP provides basic best effort datagram service
Many things you may want in a network but not widely supported
Like what?
Multicast
Performance-based routing
Web content distribution
Can you build an overlay network on IP Internet to provide QoS?
Overlay links must have guaranteed performance characteristics, otherwise, the overlay network cannot guarantee anything!
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

9. Overlay Multicast IP multicast is not widely deployed
Increase router complexity
Needs universal router support
Cannot incrementally deploy
Poor security control
Hard to multicast data reliably
Hard to perform congestion control on multicast data
Overlay multicast
Use end hosts as multicast tree nodes to distribute data
Most of the problems with IP multicast are eliminated
But overlay multicast has it’s own set of problems
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

10. Motivating Example: Conference Attendance
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

11. Solution based on Unicast
Stanford
Gatech
CMU
(Source)
Berkeley
Client-server architecture (the Web)
Does not scale well with group size
Source host is the bottleneck
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

12. End System Multicast Overlay Tree Stanford-LAN Gatech Stanford-Modem
CMU
Gatech
Stanford-Modem
Berkeley1
CMU
Stanford-Modem
Berkeley2
Berkeley1
Overlay Tree
Stanford-LAN
Gatech
Berkeley2
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

13. End System Multicast: Benefits
Scalability
Routers do not maintain per-group state
Easy to deploy
Works over the existing IP infrastructure
Can simplify support for higher level functionality
CMU
Stanford-LAN
Transcoding
Berkeley1
Unicast congestion
control
Gatech
Stanford-Modem
Berkeley2
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

14. Concerns with End System Multicast
Challenge to construct efficient overlay trees
Performance concerns compared to IP Multicast
Increase in delay
Bandwidth waste (packet duplication)
Gatech
Stanford
CMU
Berkeley
IP Multicast
Stanford-Modem
Gatech
Stanford-LAN
Berkeley1
Berkeley2
CMU
End System Multicast
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

15. More Challenges Overlays must adapt to network dynamics and congestion
CMU
Stanford-LAN
Stanford-Modem
Berkeley2
Gatech
Berkeley1
Overlays must adapt to network dynamics and congestion
CMU
Stanford-LAN
Stanford-Modem
Berkeley2
Gatech
Berkeley1
Group membership is dynamic: members can join and leave
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

16. Inefficient Overlay Trees
CMU
High latency
Berkeley2
Gatech
Stanford-LAN
Stanford-Modem
Berkeley1
Berkeley1
Berkeley2
Gatech
Stanford-LAN
CMU
Stanford-Modem
-Poor network usage
-Potential congestion near CMU
Gatech
CMU
Berkeley2
Stanford-Modem
Stanford-LAN
Berkeley1
Poor bandwidth
to members
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

17. An Efficient Overlay Tree
Gatech
CMU
Berkeley2
Stanford-LAN
Stanford-Modem
Berkeley1
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

18. Auto Tree Building is Very Hard
When a node joins, needs to quickly find a good parent to connect to tree
Good parent:
low delay to source
experience low packet loss
has enough spare outbound network capacity
has low delay, high bandwidth path to new node
When a node leaves, orphaned children nodes need to quickly find new parents
When loss rate or delay is high, need to decide whether to find a new parent
It the performance problem with this node, or with its parent, or in the middle of the network?
Are there actually better alternatives?
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

19. End System Multicast System
Focus on video broadcast applications
Implementation
Integrate with Apple QuickTime
Support for receiver heterogeneity
Support peers behind NAT and firewall
Run on Windows and Linux platforms
Showcase
SIGCOMM (max 60 simultaneous users)
Several CMU Distinguished Lectures
Slashdot (max 180 simultaneous users)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

20. Adapt to Receiver Heterogeneity
Congestion control: hop-by-hop TCP
Determine acceptable data rate
Prioritization of data streams
Parent maintains a priority queue
Drop video packets before audio packets
CMU
415Kbps
Type
Priority
Bandwidth
audio
high
15Kbps
LQ video
100Kbps
HQ video
low
300Kbps
MIT (Ethernet)
Source rate (415Kbps)
415Kbps
Berkeley (Ethernet)
Stanford (wireless)
115Kbps
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

21. Interface with Application (QT)
Mixer
Selects best viewable video stream dynamically
Switches between video streams for seamless playback A V
(Loopback)
Example: a peer with 2 children
Mixer A V1 V2
Child 1
ESM Protocol
(TCP)
(TCP)
Parent
Child 2
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

22. Group Dynamics conference end lunch folks not actively watching?
conference start
10am west coast
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

23. Overlay Tree at 2:09pm U.S. East Coast U.S. Central U.S. West Coast
Europe
Asia
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

24. Receiver Bandwidth (web download > 400Kbps are not shown)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

25. Transient Performance: Outages
Outage: loss rate exceeds 5%
95% of hosts have less than 3% outages for audio
3% outage: 2 second glitch every minute
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

26. Content Distribution Network
Refer to
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

27. Unicast Routing Overlay
Internet routing is built upon Intra-domain and Inter-domain router protocols
OSPF/RIP; BGP
OSPF/RIP routing based on shortest link weight routing
Link weights are typically very static
Does not necessarily give you best performing path (delay, throughput, loss rate)
BGP routing based mostly on policy
Policy may have nothing to do with performance
BGP very slow to react to failure (no reaction to high loss rate, e.g.)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

28. Resilient Overlay Network (RON)
Install N computers all over the place on the Internet
Each computer acts as an overlay network router
Between each overlay router is a IP tunnel (logical link)
Logical overlay topology is all-to-all (N^2)
Computers actively measure each logical link in real time for
Packet loss rate, latency, throughput, etc
Route overlay network traffic based on measured characteristics
Able to consider multiple paths in addition to the default IP Internet path given by BGP/OSPF
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

29. Example 1 Acts as overlay router
Default IP path determined by BGP & OSPF
Reroute traffic using red alternative overlay network path, avoid congestion point
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University 1

30. Potential Problems… Scalability of all these network measurements!
Overhead
Interference of measurements?
What if everyone has his/her own overlay network doing this?
Stability of the network? Oscillation? Keep rerouting back and forth?
How much can you really gain?
In delay/bandwidth, may not be that much, because for most users, the network bottleneck is near the end hosts
But is much faster to react to complete link failures than BGP
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

31. Introducing a New Idea
Multicast overlays, unicast overlays, content distribution overlays are meant to facilitate distribution of data
Route data to receivers
Basically the same fundamental function the Internet is designed for
Content addressable overlay are meant to facilitate the look-up of information!
Route questions to answers
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

32. In 1999, Napster was a huge hit
Hash Table
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

33. Distributed Hash Tables (DHT)
nodes
k1,v1
k2,v2
k3,v3
P2P overlay network
Operations:
insert(k,v)
lookup(k)
k4,v4
k5,v5
k6,v6
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

34. Why p2p overlays? Leverage pooled resources (storage, bandwidth, CPU)
Leverage resource diversity (geographic, ownership)
Leverage existing shared infrastructure
Scalability
Robustness
Self-organization
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

35. Pastry – An example system developed at Rice and Microsoft
Generic p2p location and routing substrate
Self-organizing overlay network
Lookup/insert object in < log16 N routing steps (expected)
O(log N) per-node state
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

36. Pastry: Object distribution
2128-1 O
128 bit circular ID space
nodeIds (uniform random)
objIds (uniform random)
Invariant: node with numerically closest nodeId maintains object
objId
nodeIds
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

37. Pastry
One primitive:
route(M, X): route message M to the live node with nodeId closest to key X
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

38. Pastry: Object insertion/lookup
2128-1 O
Msg with key X is routed to live node with nodeId closest to X X
Route(X)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

39. L – configurable parameter robust routing through neighbor nodes
Pastry: Leaf sets
Each node maintains IP addresses of the nodes with the L/2 numerically closest larger and smaller nodeIds, respectively
L – configurable parameter
robust routing through neighbor nodes
fault detection (use periodic hello msgs)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

40. Pastry: Routing d471f1 d467c4 d462ba d46a1c d4213f Route(d46a1c)
Properties
log16 N steps
O(log N) state
Route(d46a1c)
d13da3
65a1fc
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

41. Pastry: Routing table (# 65a1fcx)
Row 0
Row 1
Row 2
Row 3
log16 N
rows
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

42. Pastry: Routing Tradeoff O(log N) routing table size
O(log N) message forwarding steps
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

43. Pastry: Routing procedure
if (destination is within range of our leaf set)
forward to numerically closest member
else
let l = length of shared prefix
let d = value of l-th digit in D’s address
if (Rld exists)
forward to Rld
forward to a known node that
(a) shares at least as long a prefix
(b) is numerically closer than this node
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

44. Pastry: Node addition d471f1 d467c4 d462ba d46a1c New node: d46a1c
Each node has a randomly assigned 128-bit nodeId, circular namespace
Basic operation: A message with key X, sent by any Pastry node, is delivered
to the live node with nodeId closest to X in at most log16 N steps (barring node failures).
Pastry uses a form of generalized hypercube routing, where the routing tables are initialized and updated dynamically.
Route(d46a1c)
d13da3
65a1fc
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

45. Node departure (failure)
Leaf set members exchange keep-alive messages
Leaf set repair (eager): request set from farthest live node in set
Routing table repair (lazy): get table from peers in the same row, then higher rows
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

46. Application Example PAST: File storage
fileId
Insert fileId
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

47. PAST: File storage Storage Invariant: File “replicas” are
fileId
Insert fileId
k=4
Storage Invariant:
File “replicas” are
stored on k nodes
with nodeIds
closest to fileId
(k is bounded by the leaf set size)
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

48. PAST: File Retrieval C k replicas Lookup
fileId
file located in log16 N steps (expected)
usually locates replica nearest client C
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University

49. Application Example SCRIBE: Large scale multicast
topicId
Publish topicId
Subscribe topicId
T. S. Eugene Ng eugeneng at cs.rice.edu Rice University