A WAN-in-LAB for Protocol Development Netlab, Caltech George Lee, Lachlan Andrew, David Wei, Bartek Wydrowski, Cheng Jin, John Doyle, Steven Low, Harvey Newman John Doyle, Steven Low, Harvey Newman

Outline What and why WAN-in-Lab? What and why WAN-in-Lab? Current projects Current projects Equipment Equipment Configuration Configuration External connections External connections Future plans Future plans How can I get access to WAN-in-Lab? How can I get access to WAN-in-Lab?

What is WAN-in-Lab? “Wide Area” Network in a laboratory “Wide Area” Network in a laboratory Real fibre delaysReal fibre delays Carrier-class routers, switches, …Carrier-class routers, switches, …

Why -- Spectrum of tools cost abstraction mathssimulationemulationlive netwkWANinLab NS2 SSFNet QualNet JavaSim Mathis formula Optimization Control theory Nonlinear model Stocahstic model DummyNet EmuLab ModelNet WAIL UltraLight PlanetLab Abilene NLR LHCNet CENIC etc ? All scales are important– WAN-in-Lab fills a gap

WAN in Lab WAN in Lab Capacity: 2.5 – 10 GbpsCapacity: 2.5 – 10 Gbps Delay: 0 – 120 ms round tripDelay: 0 – 120 ms round trip Configurable & evolvable Configurable & evolvable Topology, rate, delays, routeTopology, rate, delays, route Modular design stays up to dateModular design stays up to date Breakable Breakable Won’t take down real networkWon’t take down real network Flexible, active debugging Flexible, active debugging Passive monitoring, AQMPassive monitoring, AQM Integral part of R&A networks Integral part of R&A networks Transition from theory, implementation, demonstration, deploymentTransition from theory, implementation, demonstration, deployment Transition from lab to marketplaceTransition from lab to marketplace Global resource Global resource Part of global infrastructure UltraLight led by Harvey NewmanPart of global infrastructure UltraLight led by Harvey Newman Wind Tunnel of Networking

Projects TCP benchmarking TCP benchmarking FAST FAST Delay-based congestion controlDelay-based congestion control MaxNet MaxNet Explicit signalling congestion controlExplicit signalling congestion control MojaveFS MojaveFS New distributed file systemNew distributed file system University of Pittsburg University of Pittsburg TCP with small buffersTCP with small buffers University of Melbourne University of Melbourne Single-bit congestion markingSingle-bit congestion marking

Example project: MaxNet

Sample MaxNet results Achieves realistic delay at 1Gbit/s Achieves realistic delay at 1Gbit/s

Equipment 4 Cisco 7609 routers with OC48 line cards 4 Cisco 7609 routers with OC48 line cards 6 Cisco ONS switches 6 Cisco ONS switches A few dozen high speed servers A few dozen high speed servers 1G switch to routers/servers 1G switch to routers/servers Calient switch for OC48 Calient switch for OC48 2,400 kilometres of fibre, optical amplifiers, dispersion compensation modules 2,400 kilometres of fibre, optical amplifiers, dispersion compensation modules 63ms aggregate RTT delay, in two hops 63ms aggregate RTT delay, in two hops 120ms using IP loopbacks120ms using IP loopbacks

External connections Will link to Ultralight, 10Gbps Physics WAN Will link to Ultralight, 10Gbps Physics WAN Smooth migration testing -> deployment Smooth migration testing -> deployment Delay Delay longerlonger jitterjitter Cross traffic Cross traffic Monitor data routed through WiL Monitor data routed through WiL

Configuration -- Delays Want maximum delay from limited fibre Want maximum delay from limited fibre Signals traverse fibre 16 timesSignals traverse fibre 16 times 4 WDM wavelengths 4 WDM wavelengths 4 OC48 (2.5G) MUXed onto OC192 (10G) 4 OC48 (2.5G) MUXed onto OC192 (10G) Lots of transponders Lots of transponders WDM amplifier joins 100km spools  200kmWDM amplifier joins 100km spools  200km

Configuration – delays OC48 slot Amp WDM Wavelength Bidirectional 100km 16x200km

Configuration – delays Delay varied by adjusting the number of OC48 hops traversed Delay varied by adjusting the number of OC48 hops traversed Calient optical switch selects required hops Calient optical switch selects required hops Hop lengths 200km up to 1600km Hop lengths 200km up to 1600km Maximise granularity given limited switch portsMaximise granularity given limited switch ports Switch

Future plans Better control over capacities and buffers Better control over capacities and buffers Saturating links Saturating links Current servers 1Gbps, links 2.5GbpsCurrent servers 1Gbps, links 2.5Gbps Creative ways to emulate more topologies Creative ways to emulate more topologies Investigate cross-traffic generation Investigate cross-traffic generation Harpoon currently installedHarpoon currently installed Better monitoring Better monitoring

Using WAN-in-Lab Contact me – lachlan at caltech. Edu Contact me – lachlan at caltech. Edu Coarse timesharing Coarse timesharing Some users set up experiments while others run experimentsSome users set up experiments while others run experiments Software setup still being developed Software setup still being developed Your chance to influence our directions to tailor it to your needsYour chance to influence our directions to tailor it to your needs

Conclusion WAN-in-Lab fills the gap between emulation and live network experiments WAN-in-Lab fills the gap between emulation and live network experiments Seeks to be as realistic as possible Seeks to be as realistic as possible Long links, simple topologyLong links, simple topology Focus will be on TCP benchmarking Focus will be on TCP benchmarking We encourage people to use it We encourage people to use it

Spare Slides

WAN-in-Lab testbed Dummynet and simulation introduce artifacts Dummynet and simulation introduce artifacts Also need to test on real equipment Also need to test on real equipment WAN with real delays, located in a single room WAN with real delays, located in a single room Connected to an external WAN (Ultralight)Connected to an external WAN (Ultralight) Open for the community to use for benchmarking Open for the community to use for benchmarking OC-48

WAN-in-Lab capabilities CurrentPlanned Two 2.5G bottlenecks Multiple 1G bottlenecks Six 2.5G bottlenecks Two “real” delays (Emulate cross traffic delay) Up to six “real” delays End-to-end RTT, drop Per-router delay, drop (movable DAG cards)

Configuration -- delays OC48 slot Amp WDM Wavelength Bidirectional 100km