Towards Virtual Networks for Virtual Machine Grid Computing Ananth I. Sundararaj Peter A. Dinda Prescience Lab Department of Computer Science Northwestern.

Slides:



Advertisements
Similar presentations
All Rights Reserved © Alcatel-Lucent 2009 Enhancing Dynamic Cloud-based Services using Network Virtualization F. Hao, T.V. Lakshman, Sarit Mukherjee, H.
Advertisements

Computer Networking Components Chad DuBose ~ Assignment #3 ~ LTEC
1 Scoped and Approximate Queries in a Relational Grid Information Service Dong Lu, Peter A. Dinda, Jason A. Skicewicz Prescience Lab, Dept. of Computer.
Mobility Jennifer Rexford COS 461: Computer Networks Lectures: MW 10-10:50am in Architecture N101
Improving TCP Performance over Mobile Ad Hoc Networks by Exploiting Cross- Layer Information Awareness Xin Yu Department Of Computer Science New York University,
1 Virtual Machine Resource Monitoring and Networking of Virtual Machines Ananth I. Sundararaj Department of Computer Science Northwestern University July.
1 In VINI Veritas: Realistic and Controlled Network Experimentation Jennifer Rexford with Andy Bavier, Nick Feamster, Mark Huang, and Larry Peterson
Automatic Run-time Adaptation in Virtual Execution Environments Ananth I. Sundararaj Advisor: Peter A. Dinda Prescience Lab Department of Computer Science.
Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience.
Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience.
This work is supported by the National Science Foundation under Grant Number DUE Any opinions, findings and conclusions or recommendations expressed.
An Overlay Data Plane for PlanetLab Andy Bavier, Mark Huang, and Larry Peterson Princeton University.
Ashish Gupta, Marcia Zangrilli, Ananth I. Sundararaj, Peter A. Dinda, Bruce B. Lowekamp EECS, Northwestern University Computer Science, College of William.
Virtuoso: Distributed Computing Using Virtual Machines Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University
Virtuoso: Distributed Computing Using Virtual Machines Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University
Dynamic Topology Adaptation of Virtual Networks of Virtual Machines Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience Lab Department of Computer.
Hardness of Approximation and Greedy Algorithms for the Adaptation Problem in Virtual Environments Ananth I. Sundararaj, Manan Sanghi, John R. Lange and.
Virtuoso: Distributed Computing Using Virtual Machines Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University
An Optimization Problem in Adaptive Virtual Environments Ananth I. Sundararaj Manan Sanghi Jack R. Lange Peter A. Dinda Prescience Lab Department of Computer.
1 Automatic Dynamic Run-time Optical Network Reservations John R. Lange Ananth I. Sundararaj and Peter A. Dinda Prescience Lab Department of Computer Science.
Towards Virtual Networks for Virtual Machine Grid Computing Ananth I. Sundararaj Peter A. Dinda Prescience Lab Department of Computer Science Northwestern.
Topics 1.Security options and settings 2.Layer 2 vs. Layer 3 connection types 3.Advanced network and routing options 4.Local connections 5.Offline mode.
COS 461: Computer Networks
Networking in VMware Workstation 8
The Whats and Whys of Whole System Virtualization Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University
Adaptive Virtual Networking For Virtual Machine-based Distributed Computing Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University.
Free Network Measurement for Adaptive Virtualized Distributed Computing Ashish Gupta, Marcia Zangrilli, Ananth Sundararaj, Anne Huang, Peter A. Dinda,
Dynamic Topology Adaptation of Virtual Networks of Virtual Machines Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience Lab Department of Computer.
Inferring the Topology and Traffic Load of Parallel Programs in a VM environment Ashish Gupta Peter Dinda Department of Computer Science Northwestern University.
1 25\10\2010 Unit-V Connecting LANs Unit – 5 Connecting DevicesConnecting Devices Backbone NetworksBackbone Networks Virtual LANsVirtual LANs.
Jennifer Rexford Princeton University MW 11:00am-12:20pm SDN Software Stack COS 597E: Software Defined Networking.
And how they are used. Hubs send data to all of the devices that are plugged into them. They have no ability to send packets to the correct ports. Cost~$35.
Data Center Network Redesign using SDN
Networking Components Christopher Biles LTEC Assignment 3.
CECS 5460 – Assignment 3 Stacey VanderHeiden Güney.
Chapter 5 Networks Communicating and Sharing Resources
Wave Relay System and General Project Details. Wave Relay System Provides seamless multi-hop connectivity Operates at layer 2 of networking stack Seamless.
Common Devices Used In Computer Networks
Objectives Configure routing in Windows Server 2008 Configure Routing and Remote Access Services in Windows Server 2008 Network Address Translation 1.
Presented by Xiaoyu Qin Virtualized Access Control & Firewall Virtualization.
Module 4: Designing Routing and Switching Requirements.
Politecnico di Torino Dipartimento di Automatica ed Informatica TORSEC Group Performance of Xen’s Secured Virtual Networks Emanuele Cesena Paolo Carlo.
Virtual Private Ad Hoc Networking Jeroen Hoebeke, Gerry Holderbeke, Ingrid Moerman, Bard Dhoedt and Piet Demeester 2006 July 15, 2009.
S4-Chapter 3 WAN Design Requirements. WAN Technologies Leased Line –PPP networks –Hub and Spoke Topologies –Backup for other links ISDN –Cost-effective.
Advanced Computing and Information Systems laboratory IP over P2P: Enabling Self- configuring Virtual IP Networks for Grid Computing Arijit Ganguly, Abhishek.
Cisco S3C3 Virtual LANS. Why VLANs? You can define groupings of workstations even if separated by switches and on different LAN segments –They are one.
A machine that acts as the central relay between computers on a network Low cost, low function machine usually operating at Layer 1 Ties together the.
Chapter 3 - VLANs. VLANs Logical grouping of devices or users Configuration done at switch via software Not standardized – proprietary software from vendor.
Virtual Machines Created within the Virtualization layer, such as a hypervisor Shares the physical computer's CPU, hard disk, memory, and network interfaces.
SOFTWARE DEFINED NETWORKING/OPENFLOW: A PATH TO PROGRAMMABLE NETWORKS April 23, 2012 © Brocade Communications Systems, Inc.
+ Routing Concepts 1 st semester Objectives  Describe the primary functions and features of a router.  Explain how routers use information.
Networking Components William Isakson LTEC 4550 October 7, 2012 Module 3.
Internet Traffic Engineering Motivation: –The Fish problem, congested links. –Two properties of IP routing Destination based Local optimization TE: optimizing.
Intro to Distributed Systems Hank Levy. 23/20/2016 Distributed Systems Nearly all systems today are distributed in some way, e.g.: –they use –they.
Also known as hardware/physi cal address Customer Computer (Client) Internet Service Provider (ISP) MAC Address Each Computer has: Given by NIC card.
Model: DS-600 5x 10/100/1000Mbps Ethernet Port Centralized WLAN management and Access Point Discovery Manages up to 50 APs with access setting control.
KAPLAN SCHOOL OF INFORMATION SYSTEMS AND TECHNOLOGY IT375 Window Enterprise Administration Course Name – IT Introduction to Network Security Instructor.
Ad Hoc – Wireless connection between two devices Backbone – The hardware used in networking Bandwidth – The speed at which the network is capable of sending.
Connecting LANs, Backbone Networks
Planning and Troubleshooting Routing and Switching
GGF15 – Grids and Network Virtualization
Aled Edwards, Anna Fischer, Antonio Lain HP Labs
IS3120 Network Communications Infrastructure
Middleware for Grid Computing On Virtual Machines
Department of Computer Science Northwestern University
Computing Over Distance
NTHU CS5421 Cloud Computing
Ananth I. Sundararaj Ashish Gupta Peter A. Dinda Prescience Lab
System Models and Networking Chapter 2,3
An Optimization Problem in Adaptive Virtual Environments
Presentation transcript:

Towards Virtual Networks for Virtual Machine Grid Computing Ananth I. Sundararaj Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University

2 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET Adaptive virtual network Related Work Conclusions Current Status

3 Aim Grid Computing New Paradigm Traditional Paradigm Deliver arbitrary amounts of computational power to perform distributed and parallel computations Problem1: Grid Computing using virtual machines Problem2: Solution How to leverage them? Virtual Machines What are they? 6b 6a 5 4 3b 3a 2 1 Resource multiplexing using OS level mechanism Complexity from resource user’s perspective Complexity from resource owner’s perspective

4 Virtual Machines Virtual machine monitors (VMMs) Raw machine is the abstraction VM represented by a single image VMware GSX Server

5 Virtual machine grid computing Approach: Lower level of abstraction –Raw machines, not processes, jobs, RPC calls R. Figueiredo, P. Dinda, J. Fortes, A Case For Grid Computing on Virtual Machines, ICDCS 2003 Mechanism: Virtual machine monitors Our Focus: Middleware support to hide complexity –Ordering, instantiation, migration of machines –Virtual networking –remote devices –Connectivity to remote files, machines –Information services –Monitoring and prediction –Resource control

6 The Simplified Virtuoso Model Orders a raw machine User Specific hardware and performance Basic software installation available User’s LAN VM Virtual networking ties the machine back to user’s home network Virtuoso continuously monitors and adapts

7 User’s View in Virtuoso Model User User’s LAN VM

8 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET Adaptive virtual network Related Work Conclusions Current Status

9 User’s friendly LAN Foreign hostile LAN Virtual Machine Why VNET? A Scenario IP network User has just bought

10 User’s friendly LAN Foreign hostile LAN Virtual Machine VNET: A bridge with long wires Host Proxy X Why VNET? A Scenario VM traffic going out on foreign LAN IP network A machine is suddenly plugged into a foreign network. What happens? Does it get an IP address? Is it a routeable address? Does firewall let its traffic through? To any port?

11 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET Adaptive virtual network Related Work Conclusions Current Status

12 A Layer 2 Virtual Network for the User’s Virtual Machines Why Layer 2? –Protocol agnostic –Mobility –Simple to understand –Ubiquity of Ethernet on end-systems What about scaling? –Number of VMs limited (~1024 per user) –One VNET per user –Hierarchical routing possible because MAC addresses can be assigned hierarchically

13 Host VM Proxy VNET Client vmnet0 ethx ethz“eth0” VNET ethy “eth0” Client LAN IP Network Ethernet Packet Tunneled over TCP/SSL Connection Ethernet Packet Captured by Promiscuous Packet Filter Ethernet Packet Injected Directly into VM interface “Host Only” Network VNET operation Traffic outbound from the user’s LAN

14 Performance Evaluation Main goal Convey the network management problem induced by VMs to the home network of the user VNET’s performance should be In line with physical network Comparable to other options Sufficient for scenarios However Metrics LatencyBandwidth small transfer Interactivity Large transfer low throughput Why?How? Why? ping hour long intervals ttcp socket buffer 1 GB of data

15 VNET test configuration Proxy 100 mbit Switches Client 100 mbit Switch Firewall 1 Router Host 100 mbit Switches 100 mbit Switch Firewall 2 VM Local Local area configuration Proxy 100 mbit Switches Client 100 mbit Switch Firewall 1 Router Host 100 mbit Switch Router VM Local IP Network (14 hops via Abilene ) Wide area configuration Northwestern University, ILCarnegie Mellon University, PA

16 Average latency over WAN Proxy Client Host VM IP Network Northwestern University, ILCarnegie Mellon University, PA (Physical Network) Host - VM Client - Proxy Proxy - Host

17 Standard deviation of latency over WAN What:VNET increases variability in latency TCP connection between VNET servers trades packet loss for increased delay Why: (Physical Network)

18 Bandwidth over WAN What do we see: VNET achieves lower than expected throughput VNET’s is tricking TTCP’s TCP connection Why: Expectation: VNET to achieve throughput comparable to the physical network

19 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET Adaptive virtual network Related Work Conclusions Current Status

20 User’s friendly LAN Foreign hostile LAN 1 Host 2 + VNET Proxy + VNET VNET Overlay IP network Host 3 + VNET Host 4 + VNET Host 1 + VNET Foreign hostile LAN 3 Foreign hostile LAN 4 Foreign hostile LAN 2 VM 1 VM 4 VM 3 VM 2

21 Bootstrapping the Virtual Network Topology may change Links can be added or removed on demand Virtual machines can migrate VM Vnetd VM Host + VNETd Proxy + VNETd VM Star topology always possible Forwarding rules can change Forwarding rules can be added or removed on demand

22 VM Layer VNETd Layer Physical Layer Application communication topology and traffic load; application processor load Network bandwidth and latency; sometimes topology Vnetd layer can collect all this information as a side effect of packet transfers and invisibly act Reservation Routing change VM migrates Topology changes

23 Outline Virtual machine grid computing Virtuoso system Networking challenges in Virtuoso Enter VNET VNET Adaptive virtual network Related Work Conclusions Current Status

24 Related Work Collective / Capsule Computing (Stanford) –VMM, Migration/caching, Hierarchical image files Denali (U. Washington) –Highly scalable VMMs (1000s of VMMs per node) SODA and VIOLIN (Purdue) –Virtual Server, fast deployment of services VPN Virtual LANs, IEEE Overlay Networks: RON, Spawning networks, Overcast Ensim Virtuozzo (SWSoft) –Ensim competitor Available VMMs: IBM’s VM, VMWare, Virtual PC/Server, Plex/86, SIMICS, Hypervisor, VM/386

25 Conclusions There exists a strong case for grid computing using virtual machines Challenging network management problem induced by VMs in the grid environment Described and evaluated a tool, VNET, that solves this problem Discussed the opportunities, the combination of VNET and VMs present, to exploit an adaptive overlay network

26 Current Status Application traffic load measurement and topology inference [Ashish Gupta] Support for arbitrary topologies and forwarding rules Dynamic adaptation to improve performance

27 Current Status Snapshots Pseudo proxy

28 For More Information –Prescience Lab (Northwestern University) –Virtuoso: Resource Management and Prediction for Distributed Computing using Virtual Machines VNET is publicly available from