Adaptive Virtual Networking For Virtual Machine-based Distributed Computing Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University

2 People and Acknowledgements Students –Ashish Gupta, Ananth Sundararaj, Alex Shoykhet, Jack Lange Collaborators –In-Vigo project at University of Florida Renato Figueiredo, Jose Fortes Funders/Gifts –NSF through several awards, VMWare

3 IBM xSeries virtual cluster (64 CPUs), 1 TB RAID Northwestern Internet Interactivity Environment Cluster, CAVE (~90 CPUs), 8 TB RAID 2 Distributed Optical Testbed Clusters IBM xSeries (14-28 CPUs), 1 TB RAID Nortel Optera Metro Edge Optical Router Distributed Optical Testbed (DOT) Private Optical Network DOT clusters with optical connectivity IBM xSeries (14-28 CPUs), 1 TB RAID: Argonne, U.Chicago, IIT, NCSA, others

4 Users already know how to deal with this complexity at another level

5 Virtuoso: “The Dell Model” A. Shoykhet, J. Lange, and P. Dinda, Virtuoso: A System For Virtual Machine Marketplaces, Technical Report NWU- CS-04-39, July, R. Figueiredo, P. Dinda, J. Fortes, A Case For Grid Computing on Virtual Machines, Proceedings of the 23rd International Conference on Distributed Computing Systems (ICDCS 2003)

6 The Illusion User User’s LAN VM Your machines are sitting next to you.

7 Virtual Machines Language-oriented VMs –Abstract interpreted machine, JIT Compiler, large library –Examples: UCSD p-system, Java VM,.NET VM Application-oriented VMs –Redirect library calls to appropriate place –Examples: Entropia VM Virtual servers –Kernel makes it appear that a group of processes are running on a separate instance of the kernel or run OS at user-level on top of itself –Examples: Ensim, Virtuozzo, UML, VServer, FreeVSD … Microkernels designed to host OSes –Xeno VM Virtual machine monitors (VMMs) –Raw machine is the abstraction –VM represented by a single image –Examples: IBM’s VM, VMWare, Virtual PC/Server, Plex/86, SIMICS, Hypervisor, DesQView/TaskView. VM/386

8 Claim Virtual networking for VMs enables the broad application of dream techniques… –Adaptation –Resource reservation … using existing, unmodified applications and operating systems –So actual people can use the techniques

9 Questions Is there enough application information? –Resource demands –Goals Is there enough resource information? –Cycles –Bandwidth Are there sufficient adaptation and reservation mechanisms? Is the control loop fast enough?

10 Outline Motivation and claims VNET: A virtual network for virtual machines –And what it enables VTTIF: Application topology inference Dynamic topology adaptation –Combining VNET and VTTIF Current directions Conclusions

11 Why Virtual Networking? A machine is suddenly plugged into your network. What happens? –Does it get an IP address? –Is it a routeable address? –Does firewall let its traffic through? –To any port? How do we make virtual machine hostile environments as friendly as the user’s LAN?

12 VNET: A Layer 2 Virtual Network for the User’s Virtual Machines Why Layer 2? –Protocol agnostic –Mobility –Simple to understand –Ubiquity of Ethernet on end-systems What about scaling? –Number of VMs limited (1024/user) –Hierarchical routing possible because MAC addresses can be assigned hierarchically A. Sundararaj, P. Dinda, Towards Virtual Networks for Virtual Machine Grid Computing, USENIX VM 2004

13 A Simple Layer 2 Virtual Network ClientServer Remote VM Physical NIC VM monitor Virtual NIC Physical NIC SSH Hostile Remote NetworkFriendly Local Network

14 A Simple Layer 2 Virtual Network ClientServer Remote VM Physical NIC VM monitor Virtual NIC Physical NIC SSH Hostile Remote NetworkFriendly Local Network

15 A Simple Layer 2 Virtual Network ClientServer Remote VM Physical NIC VM monitorvnetd Virtual NIC Physical NIC UDP, TCP, TCP/SSL, or SSH tunnel Hostile Remote NetworkFriendly Local Network

16 More Details Host VM Proxy VNET Client vmnet0 ethx ethz“eth0” VNET ethy “eth0” Client LAN IP Network Ethernet Packet Tunneled over TCP/SSL Connection Ethernet Packet Captured by Promiscuous Packet Filter Ethernet Packet Injected Directly into VM interface “Host Only” Network VNET 0.9 available from A collection of such Proxy/Host connections forms a star network centered at the Proxy on the user’s network

17 Initial Performance Results (LAN) Faster than NAT approach Lots of room for improvement This version you can download and use right now

18 VNET 1.0: Bootstrapping the Virtual Network Star topology always possible Topology may change Links can be added or removed on demand Virtual machines can migrate Forwarding rules can change Forwarding rules can be added or removed on demand Host + VNETd Proxy + VNETd VM

19 Current Status Snapshots Pseudo proxy

20 VNET 1.0 Performance BW and latency similar to VNET 0.9 Add/Delete Link: 21 ms Add/Delete Rule: 16 ms IBM e1350 cluster, 100 mbit switch

21 VNET 1.0 Topology Manipulation (Eight VMs) SetupTeardown

22 VNET 1.0 Topology Manipulation (Eight VMs)

23 VNET 1.0 Topology Manipulation (Eight VMs)

24 Outline Motivation and claims VNET: A virtual network for virtual machines –And what it enables VTTIF: Application topology inference Dynamic topology adaptation –Combining VNET and VTTIF Current directions Conclusions

25 VM Layer Vnetd Layer Physical Layer

26 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load

27 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology

28 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency, sometimes topology; host load Vnetd layer can collect all this information as a side effect of packet transfers

29 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act

30 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act VM Migration

31 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act VM Migration Topology change

32 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act VM Migration Topology change Routing change

33 VM Layer Vnetd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act VM Migration Topology change Routing change Reservation

34 Outline Motivation and claims VNET: A virtual network for virtual machines –And what it enables VTTIF: Application topology inference Dynamic topology adaptation –Combining VNET and VTTIF Current directions Conclusions

35 VTTIF: Application Traffic Load Measurement and Topology Inference Parallel and distributed applications display particular communication patterns on particular topologies –Intensity of communication can also vary from node to node or time to time. –Combined representation: Traffic Load Matrix VNET already sees every packet sent or received by a VM Can we use this information to compute a global traffic load matrix? Can we eliminate irrelevant communication from matrix to get at application topology?

36 Traffic Monitoring and Reduction Host VM VNET vmnet0 ethz“eth0” “Host Only” Network Ethernet Packet Format: SRC|DEST|TYPE|DATA (size) VMTrafficMatrix[SRC][DEST]+=size Each VM on the host contributes a row and column to the VM traffic matrix Global reduction to find overall matrix, broadcast back to VNETs Each VNET daemon has a view of the global network load Packets observed here

37 Denoising The Matrix Throw away irrelevant communication –ARPs, DNS, ssh, etc. Find maximum entry, a Eliminate all entries below  a Very simple, but seems to work very well for BSP parallel applications Remains to be seen how general it is

38 Offline Results: Synthetic Benchmark

39 NAS IS Benchmark

40 NAS IS Benchmark h1h2h3h4h5h6h7h8 h h h h h h h h *numbers indicate MB of data transferred.

41 Online Challenges When to start? When to stop? –Traffic matrix may not be stationary! Synchronized monitoring –All must start and stop together

42 When To Start? When to Stop? Reactive MechanismsProactive Mechanisms Start when traffic rate exceeds threshold Stop when traffic rate exceeds a second threshold Non-uniform discrete event sampling Provide support for queries by external agent Keep multiple copies of the matrix, one for each resolution (1s, 2s, 4s, etc) What is the Traffic Matrix from the last time there was at least one high rate source? What is the Traffic Matrix for the last n seconds ?

43 Overheads (100 mbit LAN) Essentially zero latency impact 4.2 % throughput reduction versus VNET A. Gupta, P. Dinda, Inferring the Topology and Traffic Load of Parallel Programs Running In a Virtual Machine Environment, JSSPP 2004.

44 Online: NAS IS on 4 VMs

45 Outline Motivation and claims VNET: A virtual network for virtual machines –And what it enables VTTIF: Application topology inference Dynamic topology adaptation –Combining VNET and VTTIF Current directions Conclusions

46 Dynamic Topology Adaptation VTTIF reactive mechanism run continuously On topology change, adjust VNET topology, adding links in priority order Corresponding forwarding rules also added Measure performance (running time) of application (BSP patterns application) A. Sundararaj, A. Gupta, P. Dinda, Dynamic Topology Adaptation in a Virtual Network of Virtual Machines, In Submission

47 Inference and Adaptation (8 VMs, LAN)

48 Example Result (all-to-all, 8 VMs, LAN)

49 Outline Motivation and claims VNET: A virtual network for virtual machines –And what it enables VTTIF: Application topology inference Dynamic topology adaptation –Combining VNET and VTTIF Current directions Conclusions

50 Adaptation With Migration Learn how to adapt using Virtuoso’s VM migration capabilities Virtuoso migration times with rsync –~300 seconds (1.1 GB machine) –~50 seconds (100 MB machine) Versioning file system approaches Data point: CMU ISR project: seconds for personal windows VM

51 Policy Avoidance Routing Multi-site collaborations often stymied by interactions between per-site network security policies VNET opportunity: find a path on behalf of application where one exists, but is obscured Example: NAT Traversal –RFC 3489 / STUN (chownat) Example: Tunneling through initiation protocol –HTTP or SSH

52 Physical Network Measurement Use existing application traffic to measure underlying physical network Passive packet dispersion techniques –With Bruce Lowekamp, W&M Topology inference –With Bruce Lowekamp, W&M M. Zangrilli and B. Lowekamp, Using Passive Traces of Application Traffic in a Network Monitoring System, HPDC 2004.

53 Integration With Resource Prediction Visit rps.cs.northwestern.edu for more info and downloads

54 Improving the Backbone Replacing the proxy star with a multisource muliticast system for higher performance and resilience FatNemo protocol –Arrange nodes into fat tree S. Birrer, D. Lu, F. Bustamante, Y. Qiao, P. Dinda, FatNemo: Building a Resilient Multi-Source Multicast Fat-Tree, WCCD 2004

55 Extended Application Inference Offered computational load VM-internal performance data Synchronization points and waiting Inference of application goals Simple layered API for getting more application information into system

56 Making the Fast Path Fast Move VNET forwarder into kernel of host OS Guest OS device driver to directly communicate out of VM to VNET Forwarder Inference may make deposit message passing possible Goal: Minimal overhead BW and latency for using VNET, even on gigabit and faster networks

57 Leveraging Optical Networking Use inferred application topology to do light path setup on behalf of application Currently: ICAIR ODIN system, DOT network

58 Related Work Collective / Capsule Computing (Stanford) –VMM, Migration/caching, Hierarchical image files, Attestation Internet Suspend/Resume (CMU/Intel) –Your VM follows you around (will be deployed on CMU campus) Denali (U. Washington) –Highly scalable VMMs (1000s of VMMs per node) CoVirt (U. Michigan) Xenoserver (Cambridge) SODA (Purdue) –Virtual Server, fast deployment of services Ensim –Virtual Server, widely used for web site hosting –WFQ-based resource control released into open-source Linux kernel Virtouzzo (SWSoft) –Ensim competitor Available VMMs: IBM’s VM, VMWare, Virtual PC/Server, Plex/86, SIMICS, Hypervisor, DesQView/TaskView. VM/386

59 Conclusions Virtual machines on virtual networks as the abstraction for distributed computing Virtual network as a fundamental layer for measurement and adaptation Status –Virtuoso prototype running on our cluster –VNET 0.9 released. –VNET 1.0 (with VTTIF) in progress –Wayback versioning file system released

60 For More Information Prescience Lab – Virtuoso – Join our user comfort study! – Join our intrusion detection study! –