CS An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

CS Outline Motivation Design and Implementation Evaluation Conclusions

CS Motivation Transferring large amount of data across Internet is challenging –Long period of transferring vs. Problematic underlying Internet paths [Paxson 96] Path/node failures Temporary path outages Rapid route alternation Temporary routing loops An overlay routing scheme can help

CS Overlay Routing Scheme A group of application-layer routers –Build on existing Internet routing substrate Choosing “good” transferring path –Avoid problematic underlying paths Caching on intermediate routers –Help on retransferring

CS The Generic Design Router $ $ $ $ File Transfer Server File Transfer Server Application

CS File Transfer Routers Link-state based routing protocol –Send “Hello” periodically Exchange link-state info. Detect degraded path performance and failures –Build forward table dynamically Flexible path metrics –Latency, available throughput, packet loss rate –Application-specific metrics Network conditions fatal for one application, may not acceptable for another one

CS File Transferring File transfer servers find the closest router –Propagate “who owns cache” queries and get metric info. Large files are split into chunks –Each chunk is transferred independently Over underlying Internet path directly Or, via transfer routers –“Best” path under current network situation Caching policy enforced

CS Caching on Routers Chunks are cached on intermediate routers –Cache policies decided by application –Build cache info table on each router On retransferring –Cached chunks transferred from intermediate routers Caching policies – two layer –How to distribute chunks among routers –How to share the cache storage on each router

CS Evaluation Limited experimental environment –Tux lab –Simulated network latency and degraded link performance Illustrate potential performance advantages of this routing scheme Experiments –Overcoming degraded performance –Caching improvement –Flexible caching policy

CS Overcoming Degraded Performance Transferring a 10MB file Underlying links experiencing path outages or failures –10% of transferring time –Degraded performance: 10% - 100% Routing through intermediate routers during performance failures ( + 5% vs. direct link) Source Router Destination Direct link

CS Overcoming Degraded Performance

CS Caching Performance Limited caching capacity on intermediate routers Testing 10% - 100% data cached on the way –Caching improved transferring time greatly Need more flexible cache policy! –Spread cached chunks over multiple routers –Drag frequently accessed chunks near destination

CS Caching Performance

CS Application Cache Policy Lottery vs. Round Robin R R RRRR R R A B C A->B: Using Round Robin to leave caches (the first router caches seq#1, the second router caches seq#2… and wrap back) A->C: 73.9% improvement vs. w/o caching A->B: Using Lottery based on the distance to B (according to the hop number, generate possibility) A->C: 55.1% improvement vs. w/o caching

CS Conclusions Using overlay routing can greatly improve the performance and reliability of transferring large files over problematic underlying Internet links Dynamically selecting path based on different metrics to adapt to application requirement Using cache to speed up multiple transferring Flexible cache policy

CS Future Work How to setup the nodes on the Internet? –Real experiments How to get network metrics (bandwidth, loss rate etc.) accurately? How to share the cache storage on each node for files in an efficient way? –More caching policies How to recover transferring big files from interruptions?

CS Thank You! Questions?

CS Important Reference: “Resilient Overlay Networks” “End-to-End Routing Behavior in the Internet”, Paxson, 96 Sigcomm “The End-to-End Effects of Internet Path Selection”, U of Washington