1. 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

2. 12-21-2001 CS 838-22 Outline Motivation Design and Implementation Evaluation Conclusions

3. 12-21-2001 CS 838-23 Motivation Transferring large amount of data across Internet is challenging –Long period of transferring vs. Problematic underlying Internet paths [Paxson 96] Path/node failures Temporary path outages Rapid route alternation Temporary routing loops An overlay routing scheme can help

4. 12-21-2001 CS 838-24 Overlay Routing Scheme A group of application-layer routers –Build on existing Internet routing substrate Choosing “good” transferring path –Avoid problematic underlying paths Caching on intermediate routers –Help on retransferring

5. 12-21-2001 CS 838-25 The Generic Design Router $ $ $ $ File Transfer Server File Transfer Server Application

6. 12-21-2001 CS 838-26 File Transfer Routers Link-state based routing protocol –Send “Hello” periodically Exchange link-state info. Detect degraded path performance and failures –Build forward table dynamically Flexible path metrics –Latency, available throughput, packet loss rate –Application-specific metrics Network conditions fatal for one application, may not acceptable for another one

7. 12-21-2001 CS 838-27 File Transferring File transfer servers find the closest router –Propagate “who owns cache” queries and get metric info. Large files are split into chunks –Each chunk is transferred independently Over underlying Internet path directly Or, via transfer routers –“Best” path under current network situation Caching policy enforced

8. 12-21-2001 CS 838-28 Caching on Routers Chunks are cached on intermediate routers –Cache policies decided by application –Build cache info table on each router On retransferring –Cached chunks transferred from intermediate routers Caching policies – two layer –How to distribute chunks among routers –How to share the cache storage on each router

9. 12-21-2001 CS 838-29 Evaluation Limited experimental environment –Tux lab –Simulated network latency and degraded link performance Illustrate potential performance advantages of this routing scheme Experiments –Overcoming degraded performance –Caching improvement –Flexible caching policy

10. 12-21-2001 CS 838-210 Overcoming Degraded Performance Transferring a 10MB file Underlying links experiencing path outages or failures –10% of transferring time –Degraded performance: 10% - 100% Routing through intermediate routers during performance failures ( + 5% vs. direct link) Source Router Destination Direct link

11. 12-21-2001 CS 838-211 Overcoming Degraded Performance

12. 12-21-2001 CS 838-212 Caching Performance Limited caching capacity on intermediate routers Testing 10% - 100% data cached on the way –Caching improved transferring time greatly Need more flexible cache policy! –Spread cached chunks over multiple routers –Drag frequently accessed chunks near destination

13. 12-21-2001 CS 838-213 Caching Performance

14. 12-21-2001 CS 838-214 Application Cache Policy Lottery vs. Round Robin R R RRRR R R A B C A->B: Using Round Robin to leave caches (the first router caches seq#1, the second router caches seq#2… and wrap back) A->C: 73.9% improvement vs. w/o caching A->B: Using Lottery based on the distance to B (according to the hop number, generate possibility) A->C: 55.1% improvement vs. w/o caching

15. 12-21-2001 CS 838-215 Conclusions Using overlay routing can greatly improve the performance and reliability of transferring large files over problematic underlying Internet links Dynamically selecting path based on different metrics to adapt to application requirement Using cache to speed up multiple transferring Flexible cache policy

16. 12-21-2001 CS 838-216 Future Work How to setup the nodes on the Internet? –Real experiments How to get network metrics (bandwidth, loss rate etc.) accurately? How to share the cache storage on each node for files in an efficient way? –More caching policies How to recover transferring big files from interruptions?

17. 12-21-2001 CS 838-217 Thank You! Questions?

18. 12-21-2001 CS 838-218 Important Reference: “Resilient Overlay Networks” http://nms.lcs.mit.edu/ronhttp://nms.lcs.mit.edu/ron “End-to-End Routing Behavior in the Internet”, Paxson, 96 Sigcomm “The End-to-End Effects of Internet Path Selection”, U of Washington