1. VINI: Virtual Network Infrastructure
Nick Feamster Georgia Tech
Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University

2. VINI Overview Runs real routing software
Bridge the gap between “lab experiments” and live experiments at scale. ?
Emulation
VINI
Simulation
Small-scale
experiment
Live
deployment
Runs real routing software
Exposes realistic network conditions
Gives control over network events
Carries traffic on behalf of real users
Is shared among many experiments

3. Goal: Control and Realism
Traffic
Real clients, servers
Synthetic or traces
Goal: Control and Realism
Control
Reproduce results
Methodically change or relax constraints
Realism
Long-running services attract real users
Connectivity to real Internet
Forward high traffic volumes (Gb/s)
Handle unexpected events
Topology
Arbitrary, emulated
Actual network
Traffic
Synthetic or traces
Real clients, servers
Scientifically, what’s the point of such an infrastructure?
Network Events
Inject faults, anomalies
Observed in operational network

4. Overview VINI characteristics PL-VINI: prototype on PlanetLab
Fixed, shared infrastructure
Flexible network topology
Expose/inject network events
External connectivity and routing adjacencies
PL-VINI: prototype on PlanetLab
Preliminary Experiments
Ongoing work

5. Fixed Infrastructure Concretely, what is VINI Overlays! Aggregate Gb/s
Topology, IP address, BGP peering

6. Shared Infrastructure
Virtual nodes
Isolation between experiments (e.g., CPU reservation)

7. Arbitrary Virtual Topologies
P2P connectivity
Unique interfaces visible to each virtual node

8. Exposing and Injecting Failures
Concretely, what is VINI
Overlays!
Aggregate Gb/s
Topology, IP address, BGP peering

9. Carry Traffic for Real End Users
Talk about flexible routing and forwarding here s

10. Participate in Internet Routing
BGP c
Find out about external address blocks
In future: advertise address blocks s

11. PL-VINI: Prototype on PlanetLab
First experiment: Internet In A Slice
XORP open-source routing protocol suite (NSDI ’05)
Click modular router (TOCS ’00, SOSP ’99)
Clarify issues that VINI must address
Unmodified routing software on a virtual topology
Forwarding packets at line speed
Illusion of dedicated hardware
Injection of faults and other events

12. PL-VINI: Prototype on PlanetLab
PlanetLab: testbed for planetary-scale services
Simultaneous experiments in separate VMs
Each has “root” in its own VM, can customize
Can reserve CPU, network capacity per VM
Virtual Machine Monitor (VMM)
(Linux++)
Node
Mgr
Local
Admin
VM1
VM2
VMn …
PlanetLab node

13. XORP: Control Plane BGP, OSPF, RIP, PIM-SM, IGMP/MLD XORP
Goal: run real routing protocols on virtual network topologies
XORP
(routing protocols)

14. User-Mode Linux: Environment
Interface ≈ network
PlanetLab limitation:
Slice cannot create new interfaces
Run routing software in UML environment
Create virtual network interfaces in UML
UML
XORP
(routing protocols)
eth0
eth1
eth2
eth3

15. Click: Data Plane Performance XORP Interfaces  tunnels Filters Click
Avoid UML overhead
Move to kernel, FPGA
Interfaces  tunnels
Click UDP tunnels correspond to UML network interfaces
Filters
“Fail a link” by blocking packets at tunnel
UML
XORP
(routing protocols)
eth0
eth1
eth2
eth3
Control
Data
Packet
Forward
Engine
UmlSwitch
element
Tunnel table
Click
Filters

16. Intra-domain Route Changes
2095
856
700
260
233
1295 c
639
366
548
587
846
Talk about flexible routing and forwarding here
902
1893
1176

17. Ping During Link Failure
Routes converging
Link down
Link up

18. Close-Up of TCP Transfer
PL-VINI enables a user-space virtual network
to behave like a real network on PlanetLab
Slow start
Retransmit
lost packet

19. Challenge: Attracting Real Users
Could have run experiments on Emulab
Goal: Operate our own virtual network
Carrying traffic for actual users
We can tinker with routing protocols
Attracting real users

20. Conclusion http://www.vini-veritas.net/
VINI: Controlled, Realistic Experimentation
Installing VINI nodes in NLR, Abilene
Download and run Internet In A Slice

22. TCP Throughput
Link up
Link down
Zoom in

23. Ongoing Work Improving realism Improving control
Exposing network failures and changes in the underlying topology
Participating in routing with neighboring networks
Improving control
Better isolation
Experiment specification

24. Resource Isolation Issue: Forwarding packets in user space
PlanetLab sees heavy use
CPU load affects virtual network performance
Property
Depends On
Solution
Throughput
CPU% received
PlanetLab provides CPU reservations
Latency
CPU scheduling delay
PL-VINI: boost priority of packet forward process

25. Performance is bad
User-space Click: ~200Mb/s forwarding

26. VINI should use Xen

27. Experimental Results Is a VINI feasible?
Click in user-space: 200Mb/s forwarded
Latency and jitter comparable between network and IIAS on PL-VINI.
Say something about running on just PlanetLab? Don’t spend much time talking about CPU scheduling…

28. Low latency for everyone?
PL-VINI provided IIAS with low latency by giving it high CPU scheduling priority

29. Internet In A Slice XORP Run OSPF Configure FIB Click FIB Tunnels
Inject faults
OpenVPN & NAT
Connect clients and servers S S C C
Talk about flexible routing and forwarding here S C

30. PL-VINI / IIAS Router Blue: topology XORP Red: routing and forwarding
Virtual net devices
Tunnels
Red: routing and forwarding
Data traffic does not enter UML
Green: enter & exit IIAS overlay
UML
XORP
eth1
eth3
eth2
UmlSwitch
element
FIB
Encapsulation table
eth0
Control
Data
Click
tap0

31. PL-VINI Summary Flexible Network Topology
Virtual point-to-point connectivity
Tunnels in Click
Unique interfaces per experiment
Virtual network devices in UML
Exposure of topology changes
Upcalls of layer-3 alarms
Flexible Routing and Forwarding
Per-node forwarding table
Separate Click per virtual node
Per-node routing process
Separate XORP per virtual node
Connectivity to External Hosts
End-hosts can direct traffic through VINI
Connect to OpenVPN server
Return traffic flows through VINI
NAT in Click on egress node
Support for Simultaneous Experiments
Isolation between experiments
PlanetLab VMs and network isolation
CPU reservations and priorities
Distinct external routing adjacencies
BGP multiplexer for external sessions

32. PL-VINI / IIAS Router XORP: control plane UML: environment XORP
Virtual interfaces
Click: data plane
Performance
Avoid UML overhead
Move to kernel, FPGA
Interfaces  tunnels
“Fail a link”
UML
XORP
(routing protocols)
eth0
eth1
eth2
eth3
Control
Data
Packet
Forward
Engine
UmlSwitch
element
Tunnel table
Click