1. Building Virtual Networks for Experimentation and Profit Nick Feamster, Georgia Tech Andy Bavier, Lixin Gao, Mark Huang, Murtaza Motiwala, Jennifer Rexford, Larry Peterson, Yaping Zhu

2. 2 Original Internet Design Goals Interconnection/Multiplexing (packet switching) Resilience/Survivability (fate sharing) Heterogeneity Distributed management Cost effectiveness Ease of attachment Accountability Decreasing Priority

3. 3 Internet Faces New Demands High availability and performance –Path diversity, low-latency paths, fast reaction to failures, convergence… Security guarantees –Defense against unwanted traffic Manageability –Fault detection and localization Scaling Mobility

4. 4 Design Goal Redux Human costs, operational expenditure dominate –The network must provide support for manageability Accountability increasingly important –Mobility, security, etc. to the forefront …some of the most significant problems with the Internet today relate to lack of sufficient tools for distributed management, especially in the area of routing. Dave Clark, The Design Philosophy of the DARPA Internet Protocols Dr Cerf admits that with hindsight, there are two things he regrets not including: authentication, to ensure that internet packets really do come from where they claim to have originated, and support for mobile devices. ---The Economist, June 2006

5. 5 New Protocols to the Rescue Addressing: IPv6, 8+8, HIP, NIRA Security: Secure BGP (S-BGP), soBGP, SPV Routing: HLP, RCP, Compact/Valiant Routing Naming: DOA Unwanted traffic: Capabilities, SANE, DoS- Resistant Internet Archictecture,

6. 6 These Solutions Face Two Hurdles Deployment demonstrates feasibility Demonstrating feasibility requires deployment Deployment Dilemma Coordination Constraint Benefits only realized when large fraction of networks deploy No single network wants to deploy first

7. 7 Network Virtualization: Characteristics Multiple logical routers on a single platform Resource isolation in CPU, memory, bandwidth, forwarding tables, … Customizable routing and forwarding software General-purpose CPUs for the control plane Network processors and FPGAs for data plane Sharing Customizability

8. 8 Two Hurdles Deployment demonstrates feasibility Demonstrating feasibility requires deployment Deployment Dilemma Coordination Constraint Benefits only realized when large fraction of networks deploy No single network wants to deploy first VINI: Realistic and controlled network experimentation. Cabo: Concurrent Architectures are Better than One

9. 9 VINI Overview Runs real routing software Exposes realistic network conditions Gives control over network events Carries traffic on behalf of real users Is shared among many experiments Simulation Emulation Small-scale experiment Live deployment ? VINI Bridge the gap between lab experiments and live experiments at scale.

10. 10 Goal: Control and Realism Control –Reproduce results –Methodically change or relax constraints Realism –Long-running services attract real users –Connectivity to real Internet –Forward high traffic volumes (Gb/s) –Handle unexpected events Topology Actual network Arbitrary, emulated Traffic Real clients, servers Synthetic or traces Traffic Real clients, servers Synthetic or traces Network Events Observed in operational network Inject faults, anomalies

11. 11 VINI: Overview VINI characteristics –Fixed, shared infrastructure –Flexible network topology –Expose/inject network events –External connectivity and routing adjacencies PL-VINI: prototype on PlanetLab Preliminary Experiments Ongoing work

12. 12 Fixed Physical Infrastructure

13. 13 Shared By Many Parties

14. 14 Supports Arbitrary Virtual Topologies

15. 15 Supports Controlled Link Failures

16. 16 Carry Traffic for Real End Users s c

17. 17 Participate in Internet Routing s c BGP

18. 18 Why Is This Difficult? Creation of virtual nodes –Sharing of resources –Creating the appearance of multiple interfaces –Arbitrary software Creation of virtual links –Expose underlying failures of links –Controlled link failures –Arbitrary forwarding paradigms Embedding virtual topologies –Support for simultaneous virtual experiments –Must map onto available resources, account, etc.

19. 19 PL-VINI: Prototype on PlanetLab First experiment: Internet In A Slice –XORP open-source routing protocol suite –Click modular router Expose issues that VINI must address –Unmodified routing (and other) software on a virtual topology –Forwarding packets at line speed –Illusion of dedicated hardware –Injection of faults and other events

20. 20 PL-VINI: Prototype on PlanetLab PlanetLab: testbed for planetary-scale services Simultaneous experiments in separate VMs –Each has root in its own VM, can customize Can reserve CPU, network capacity per VM Virtual Machine Monitor (VMM) (Linux++) Node Mgr Local Admin VM 1 VM 2 VM n … PlanetLab node

21. 21 Internet In A Slice XORP Run OSPF Configure FIB Click FIB Tunnels Inject faults OpenVPN & NAT Connect clients and servers

22. 22 XORP: Control Plane BGP, OSPF, RIP, PIM- SM, IGMP/MLD Goal: run real routing protocols on virtual network topologies XORP (routing protocols)

23. 23 User-Mode Linux: Environment PlanetLab limitation: –Slice cannot create new interfaces Run routing software in UML environment Create virtual network interfaces in UML Challenge: Map these interfaces to the right tunnels XORP (routing protocols) UML eth1eth3eth2eth0

24. 24 Click: Data Plane Performance –Avoid UML overhead –Move to kernel, FPGA Interfaces tunnels –Click UDP tunnels correspond to UML network interfaces Filters –Fail a link by blocking packets at tunnel XORP (routing protocols) UML eth1eth3eth2eth0 Click Packet Forward Engine Control Data UmlSwitch element Tunnel table Filters

25. 25 Recent Developments: Independence from IP Solution: Forwarding should depend on MAC addresses in UML UML eth1eth3eth2eth0 Click Packet Forward Engine Control Data XORP (routing protocols) UmlSwitch element Tunnel table Forwarding cannot depend on IP New Routers and Protocols

26. 26 Demonstration planetlab1.csail.mit.edu planetlab3.csail.mit.edu planetlab6.csail.mit.edu planetlab4.csail.mit.edu 1 2 1 3

27. 27 Experiments IGP convergence: data and control plane RON and overlay experiments iBGP convergence Network troubleshooting

28. 28 Intra-domain Route Changes s c 1176 587 846 260 700 639 1295 2095 902 548 233 1893 366 856

29. 29 Ping During Link Failure Link down Link up Routes converging

30. 30 Close-Up of TCP Transfer Slow start Retransmit lost packet PL-VINI enables a user-space virtual network to behave like a real network on PlanetLab

31. 31 Attracting Real Users Could the experiment have been run in Emulab? –Maybe, though interconnection with real Internet could provide some advantages –Turning the dials between realism and contro Goal: Operate our own virtual network –Carrying traffic for actual users –We can tinker with routing protocols

32. 32 Two Hurdles Deployment demonstrates feasibility Demonstrating feasibility requires deployment Deployment Dilemma Coordination Constraint Benefits only realized when large fraction of networks deploy No single network wants to deploy first

33. 33 Overcoming the Coordination Constraint Key problem: Federation –No Internet Service Provider has control over an entire end-to-end path –This makes deployment, troubleshooting, accountability, etc. very dificult Idea: Make the infrastructure that supports the testbed the architecture itself –Separate providers of infrastructure and service

34. 34 Concurrent Architectures are Better than One (Cabo) Infrastructure: physical infrastructure needed to build networks Service: slices of physical infrastructure from one or more providers The same entity may sometimes play these two roles.

35. 35 End-to-End Services Multi-provider VPNs Paths with end-to-end performance guarantees TodayCabo Competing ISPs with different goals must coordinate Single service provider controls end-to-end path

36. 36 End-to-End Services Online BankingWeb Surfing RoutingSecure routing protocol (e.g., S-BGP) Lowest common denominator AddressingSelf-certifying addresses (optimized for persistence) Dynamic addresses (optimized for convenience) More Security More Complete Reachability Today: Deployment logjam –Deployment requires consensus and coordination Instead: Adopt pluralist approach –Determined service provider leases infrastructure and deploys technology end-to-end Example

37. 37 Application-Specific Networks Internal BGPLink-State Protocols DisseminationHierarchical, incrementalFlooding ComputationBGP-style decision processShortest Paths Better Scalability Faster Convergence Today: Optimize a single set of protocols Instead: Parallel deployment –Run multiple networks, each catered to specific applications Example

38. 38 Embedding Given: virtual network and physical network –Topology, constraints, etc. Problem: find the appropriate mapping onto available physical resources (nodes and edges) Many possible formulations –Specific nodes mapping to certain physical nodes –Generic requirements: three diverse paths from SF to LA with 100 MBps throughput –Traffic awareness, dynamic remapping, etc. –On-the-fly creation of links in the substrate

39. 39 Accounting and Provisioning Problem: Brokering of physical infrastructure –Discovery: Discovering physical infrastructure Autodiscovery of components and topology Decision elements that configure components –Provisioning: Creating virtual networks Requests to decision elements (initially out of band), which name virtual network components Turner et al., Substrate Control Metanet –Creation: Instantiating virtual networks

40. 40 Parallel Deployment: Questions Guaranteeing global reachability –Do we need an end-to-end global reachability service? Proliferation of protocols and architectures –Is low barrier to entry a good thing for an architecture? Security –Should parallel deployment imply isolation? –If so, how to implement it?

41. 41 Economic Refactoring Infrastructure providers: maintain physical infrastructure needed to build networks Service providers: lease slices of physical infrastructure from one or more providers

42. 42 Examples in Communications Networks Packet Fabric: share routers at exchange points FON: resells users wireless Internet connectivity Infrastructure providers: Buy upstream connectivity, broker access through wireless Nomads: Users who connect to access points Service provider: FON as broker Broker

43. 43 Economic Refactoring: Challenges Being a service provider: a great deal –Opportunity to add value by creating new services Infrastructure providers –Can this enterprise be profitable? Who will become infrastructure providers? http://www.cc.gatech.edu/~feamster/papers/cabo-tr.pdf Need to understand whether this refactoring would occur

44. 44 Summary Internet faces stronger demands from users –Not necessarily designed for those challenges –Difficult to deploy fundamentally new fixes Virtualization: help us move beyond paper design –Testbed –Architectural foundation Applications and Opportunities –Simultaneous operation –Substrate and interface

45. 45

46. 46 Can Other Industries Offer Clues? Infrastructure providers: Airports Infrastructure: Gates, hands and eyes, etc. Service providers: Airlines SFO ATL BOS ORD