Globus Virtual Workspaces HEPiX Fall 2007, St Louis Kate Keahey Argonne National Laboratory University of Chicago

Slides:



Advertisements
Similar presentations
First Steps in the Clouds
Advertisements

Elastic Provisioning In Virtual Private Clouds
Open Science Grid Living on the Edge: OSG Edge Services Framework Kate Keahey Abhishek Rana.
A Scalable Approach to Deploying and Managing Appliances Kate Keahey Rick Bradshaw, Narayan Desai, Tim Freeman Argonne National Lab, University of Chicago.
Virtualization, Cloud Computing, and TeraGrid Kate Keahey (University of Chicago, ANL) Marlon Pierce (Indiana University)
On-Demand Virtual Workspaces: Quality of Life in the Grid Kate Keahey Argonne National Laboratory.
Virtual Appliances for Scientific Applications Kate Keahey Argonne National Laboratory University of Chicago.
Virtual Workspaces State of the Art and Current Directions Borja Sotomayor University of Chicago (Dept. of CS) Kate Keahey ANL/UC.
Virtualization: Towards More Flexible and Efficient Grids Kate Keahey Argonne National Laboratory.
Working Spaces: Virtual Machines in the Grid Kate Keahey Argonne National Laboratory Tim Freeman, Frank Siebenlist
Enabling Cost-Effective Resource Leases with Virtual Machines Borja Sotomayor University of Chicago Ian Foster Argonne National Laboratory/
Workspaces for CE Management Kate Keahey Argonne National Laboratory.
The VM deployment process has 3 major steps: 1.The client queries the VM repository, sending a list of criteria describing a workspace. The repository.
Wei Lu 1, Kate Keahey 2, Tim Freeman 2, Frank Siebenlist 2 1 Indiana University, 2 Argonne National Lab
Division of Labor: Tools for Growing and Scaling Grids Tim Freeman, Kate Keahey, Ian Foster, Abhishek Rana, Frank Wuerthwein, Borja Sotomayor.
From Sandbox to Playground: Dynamic Virtual Environments in the Grid Kate Keahey Argonne National Laboratory Karl Doering University.
Virtual Workspaces in the Grid Kate Keahey Argonne National Laboratory Ian Foster, Tim Freeman, Xuehai Zhang, Daniel Galron.
Cloud Computing and Virtualization with Globus Oakland, May 2008 Kate Keahey Tim Freeman University of Chicago.
Science Clouds: Early Experiences in Cloud Computing for Scientific Applications Chicago, October 2008 Kate Keahey, Renato Figueiredo, Jose Fortes, Tim.
Globus Virtual Workspaces An Update SC 2007, Reno, NV Kate Keahey Argonne National Laboratory University of Chicago
Nimbus or an Open Source Cloud Platform or the Best Open Source EC2 No Money Can Buy ;-) Kate Keahey Tim Freeman University of Chicago.
Cloud Computing with Nimbus FNAL, January 2009 Kate Keahey University of Chicago Argonne National Laboratory.
Infrastructure-as-a-Service Cloud Computing for Science
Virtual Machine Technology Dr. Gregor von Laszewski Dr. Lizhe Wang.
1/16 Distributed Systems Architecture Research Group Universidad Complutense de Madrid An Introduction to Virtualization and Cloud Technologies to Support.
System Center 2012 R2 Overview
Virtualization and Cloud Computing. Definition Virtualization is the ability to run multiple operating systems on a single physical system and share the.
The Case for Enterprise Ready Virtual Private Clouds Timothy Wood, Alexandre Gerber *, K.K. Ramakrishnan *, Jacobus van der Merwe *, and Prashant Shenoy.
An Approach to Secure Cloud Computing Architectures By Y. Serge Joseph FAU security Group February 24th, 2011.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Virtualization and the Cloud
Cloud Computing (101).
Kate Keahey Argonne National Laboratory University of Chicago Globus Toolkit® 4: from common Grid protocols to virtualization.
Copyright © 2010 Platform Computing Corporation. All Rights Reserved.1 The CERN Cloud Computing Project William Lu, Ph.D. Platform Computing.
Virtual Infrastructure in the Grid Kate Keahey Argonne National Laboratory.
 Cloud computing  Workflow  Workflow lifecycle  Workflow design  Workflow tools : xcp, eucalyptus, open nebula.
Nimbus & OpenNebula Young Suk Moon. Nimbus - Intro Open source toolkit Provides virtual workspace service (Infrastructure as a Service) A client uses.
+ CS 325: CS Hardware and Software Organization and Architecture Cloud Architectures.
Xen Overview for Campus Grids Andrew Warfield University of Cambridge Computer Laboratory.
Presented by: Sanketh Beerabbi University of Central Florida COP Cloud Computing.
1 Evolution of OSG to support virtualization and multi-core applications (Perspective of a Condor Guy) Dan Bradley University of Wisconsin Workshop on.
Large Scale Sky Computing Applications with Nimbus Pierre Riteau Université de Rennes 1, IRISA INRIA Rennes – Bretagne Atlantique Rennes, France
Globus Virtual Workspaces OOI Cyberinfrastructure Design Meeting, San Diego, October Kate Keahey University of Chicago Argonne National Laboratory.
Virtual Workspaces Kate Keahey Argonne National Laboratory.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
VMware vSphere Configuration and Management v6
Ian Gable University of Victoria 1 Deploying HEP Applications Using Xen and Globus Virtual Workspaces A. Agarwal, A. Charbonneau, R. Desmarais, R. Enge,
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
2012 Objectives for CernVM. PH/SFT Technical Group Meeting CernVM/Subprojects The R&D phase of the project has finished and we continue to work as part.
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Ian Gable HEPiX Spring 2009, Umeå 1 VM CPU Benchmarking the HEPiX Way Manfred Alef, Ian Gable FZK Karlsruhe University of Victoria May 28, 2009.
Predrag Buncic (CERN/PH-SFT) Software Packaging: Can Virtualization help?
1 Cloud Services Requirements and Challenges of Large International User Groups Laurence Field IT/SDC 2/12/2014.
36 th LHCb Software Week Pere Mato/CERN.  Provide a complete, portable and easy to configure user environment for developing and running LHC data analysis.
Unit 2 VIRTUALISATION. Unit 2 - Syllabus Basics of Virtualization Types of Virtualization Implementation Levels of Virtualization Virtualization Structures.
Claudio Grandi INFN Bologna Virtual Pools for Interactive Analysis and Software Development through an Integrated Cloud Environment Claudio Grandi (INFN.
Nimbus Update March 2010 OSG All Hands Meeting Kate Keahey Nimbus Project University of Chicago Argonne National Laboratory.
Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.
Towards Dynamic Database Deployment LCG 3D Meeting November 24, 2005 CERN, Geneva, Switzerland Alexandre Vaniachine (ANL)
Workspace Management Services Kate Keahey Argonne National Laboratory.
New Paradigms: Clouds, Virtualization and Co.
Cloud Computing with Nimbus
Management of Virtual Machines in Grids Infrastructures
Management of Virtual Machines in Grids Infrastructures
GGF15 – Grids and Network Virtualization
Cloud Computing with Nimbus
Managing Clouds with VMM
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Cloud Computing and Virtualization with Globus
Virtualization, Cloud Computing, and TeraGrid
Presentation transcript:

Globus Virtual Workspaces HEPiX Fall 2007, St Louis Kate Keahey Argonne National Laboratory University of Chicago

11/7/07, HEPiXVirtual Workspaces: Why Virtual Workspaces? l Quality of Service u We get: batch-style provisioning l One size fits all l Side-effect of job scheduling u We need: advance reservations, urgent computing, periodic, best-effort, and others l Separation of job scheduling and resource management l E.g. workflow-based apps and batch apps have different needs l Quality of Life u We have: I have a 100 nodes I cannot use u Complex applications l Hard to install l Require validation u Separation of environment preparation and resources leasing

11/7/07, HEPiXVirtual Workspaces: What are Virtual Workspaces? l A dynamically provisioned environment u Environment definition: we get exactly the (software) environment we need on demand. u Resource allocation: Provision the resources the workspace needs (CPUs, memory, disk, bandwidth, availability), allowing for dynamic renegotiation to reflect changing requirements and conditions. l Implementation u Traditional means: publishing, automated configuration, coarse-grained enforcement u Virtual Machines: encapsulated configuration and fine-grained enforcement Paper: Virtual Workspaces: Achieving Quality of Service and Quality of Life in the Grid

11/7/07, HEPiXVirtual Workspaces: Virtual Machines (Xen) l Open source l Paravirtualization u The Good: high-performance u The Bad: difficult to run proprietary OSs, and to mix 32-bit and 64-bit kernels (VT needed) l Xen terminology: u Domain0 (the host), u DomainU (user domain, the guest) LXVU SPEC INT2000 (score) LXVU Linux build time (s) LXVU OSDB-OLTP (tup/s) LXVU SPEC WEB99 (score) Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

11/7/07, HEPiXVirtual Workspaces: Deploying Workspaces Remotely Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Workspace -Workspace metadata -Pointer to the image -Logistics information -Deployment request -CPU, memory, node count, etc. VWS Service

11/7/07, HEPiXVirtual Workspaces: Interacting with Workspaces Pool node Trusted Computing Base (TCB) Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node The workspace service publishes information on each workspace as standard WSRF Resource Properties. Users can query those properties to find out information about their workspace (e.g. what IP the workspace was bound to) Users can interact directly with their workspaces the same way the would with a physical machine. VWS Service

11/7/07, HEPiXVirtual Workspaces: Workspace Service Components Pool node Trusted Computing Base (TCB) Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node VWS Service Workspace WSRF front-end that allows clients to deploy and manage virtual workspaces Resource manager for a pool of physical nodes Deploys and manages Workspaces on the nodes Contextualization creates a common context for a virtual cluster Each node must have a VMM (Xen) installed, as well as the workspace control program that manages individual nodes Workspace back-end:

11/7/07, HEPiXVirtual Workspaces: Workspace Service Components l GT4 WSRF front-end u Leverages GT core and services, notifications, security, etc. u Follows the OGF WS-Agreement provisioning model l Publishes available lease terms u Provides lease descriptions l Workspace Resource Manager (back-end) u Currently focused on Xen u Works with multiple Resource Managers u Workspace Control l Contextualization u Put the virtual appliance in its deployment context l Current release 1.3, available at: u

11/7/07, HEPiXVirtual Workspaces: Workspace Resource Managers l Default resource manager (basic slot fitting) u Commercial datacenter technology would also fit l Amazon Elastic Compute Cloud (EC2) u EC2: Selling cycles as Xen VMs u Software similar to Workspace Service l No virtual clusters, contextualization, fine-grain allocations, etc. u Grid credential admission -> EC2 charging model u STAR: 100 node VM run

11/7/07, HEPiXVirtual Workspaces: Virtual Workspaces for STAR l STAR image configuration u A virtual cluster composed of an OSG headnode and STAR worker nodes l Using the workspace service over EC2 to provision resources u Allocations of up to 100 nodes u Dynamically contextualized for out-of-the-box cluster

11/7/07, HEPiXVirtual Workspaces: Workspace Resource Managers l Default resource manager (basic slot fitting) u Commercial datacenter technology would also fit l Amazon Elastic Compute Cloud (EC2) u EC2: Selling cycles as Xen VMs u Software similar to Workspace Service l No virtual clusters, contextualization, fine-grain allocations, etc. u Grid credential admission -> EC2 charging model u STAR: 100 node VM run l Workspace Pilot u Integrating VMs into current provisioning models l Long-term solutions u Interleaving soft and hard leases u Providing better articulated leasing models u Developed in the context of existing schedulers

11/7/07, HEPiXVirtual Workspaces: Providing Resources: The Workspace Pilot l Challenge: find the simplest way to integrate VMs into current provisioning models l Glide-ins (Condor): poor mans resource leasing u Best-effort semantics: submit a job pilot that claims resources but does not run a job l The Workspace Pilot u Resources booted to dom0 u Pilot adjusts memory u VWS leases slots to VMs u Kill-all facility

11/7/07, HEPiXVirtual Workspaces: Workspace Control l VM control u Starting, stopping etc. u To be replaced by Xen API l Integrating into the network u Assigning MAC addresses and IP addresses u DHCP Delivery tool u Building up a trusted networking layer l VM image propagation l Image management and reconstruction u creating blank partitions l Talks to the workspace service via ssh

11/7/07, HEPiXVirtual Workspaces: Security Issues l Secure admission of appliances/workspaces u The appliance vendor configures the appliance, asserts its properties and signs them to the appliance l Security and other updates, configuration and versioning assertions, disallowing offsite root access, etc. u The appliance deployer validates the signature and matches the assertions to policies u SC05 Poster: Making your workspace secure: establishing trust with VMs in the Grid l Secure networking u Controlling spoofing u Isolating networks between different VM groups u Traffic monitoring

11/7/07, HEPiXVirtual Workspaces: So -- youve deployed some VMs… Now what? l Do they have public IP addresses? l Do they actually represent something useful? u I need an OSG cluster: l How do the VMs find out about each other? l Can they share storage? l Do they have host certificates? u And gridmapfile? u And all the other things that will integrate them into my VO?

11/7/07, HEPiXVirtual Workspaces: l Challenge: what is a virtual cluster? u A more complex virtual machine l Networking, shared storage, etc. that will be portable across sites and implementations u Available at the same time and sharing a common context u Example: l A set of worker nodes with some edge services in front and NFS-based shared storage l Solution: management of ensembles and sharing u Ensemble deployment, EPR management u Flexible, configurable cluster deployment u Networking l Edge Services have public IPs l Worker nodes are on a private network shared with the Edge Services u Exporting and sharing a common context l Configuring and joining context Paper: Virtual Clusters for Grid Communities, CCGrid 2006 Virtual Clusters

11/7/07, HEPiXVirtual Workspaces: Contextualization l Challenge: Putting a VM in the deployment context of the Grid, site, and other VMs u Assigning and sharing IP addresses, name resolution, application- level configuration, etc. l Solution: Management of Common Context Paper: A Scalable Approach To Deploying And Managing Appliances, TeraGrid conference 2007 u Configuration-dependent l provides&requires u Common understanding between the image vendor and deployer u Mechanisms for securely delivering the required information to images across different implementations contextualization agent Common Context IP hostname pk

11/7/07, HEPiXVirtual Workspaces: Where Do VM Images Come From? l Appliance providers u Appliance providers configure, manage, attest images u Contextualization: collaboration between appliance vendors and appliance deployers l Appliance providers u rPath l Recipe-style configuration (create a project, choose packages, cook, build the software appliance_ l Freely available online, many appliances l u Bcfg2 l Incrementally constructed configuration profiles l Configuration analysis capabilities l

11/7/07, HEPiXVirtual Workspaces: Image Management l Image partitions u Efficiency u Security u Flexibility l Partition management on deployment u Partition caching and generation u Partition sharing u Mounting System Layer Customization Layer Application Layer VO Layer

11/7/07, HEPiXVirtual Workspaces: Workspace Ecosystem Resource Providers: Local clusters, Grid resource providers (TeraGrid, OSG) Commercial providers: EC2, Sun, slicehost, Provisioning a resource, not a platform Appliance Providers: OSFarm, rPath, CohesiveFT, bcfg2, etc. marketplaces of all kinds Virtual Organizations: configuration, attestation, maintenance Middleware: appliances --> resources manage appliance deployment Combining networks and storage VWSEC2In-Vigo

11/7/07, HEPiXVirtual Workspaces: Parting Thoughts l VMs are the raw materials from which a working system can be built u But we still have to build it! u Technical challenges: taking one step at a time u Social/procedural challenges l Division of labor u Resource providers u Appliance providers u Can we build trust between these two groups? l If you have a specific problem, give us a call: u l In our copious spare time we also do research u Migration, fine-grained enforcement, resource management, load balancing, migration in time, lots of one-offs… u VTDC07 (co-located with SC07)

11/7/07, HEPiXVirtual Workspaces: Acknowledgements l Workspace team: u Kate Keahey u Tim Freeman u Borja Sotomayor l Funding u NSF SDCI Missing Links u NSF CSR Virtual Playgrounds u DOE CEDPS Project l With thanks to many collaborators: u Jerome Lauret (STAR, BNL), Doug Olson (STAR, LBNL), Marty Wesley (rPath), Stu Gott (rPath), Ken Van Dine (rPath), Predrag Buncic (Alice, CERN), Haavard Bjerke (CERN), Rick Bradshaw (Bcfg2, ANL), Narayan Desai (Bcfg2, ANL), Duncan Penfold-Brown (Atlas,uvic), Ian Gable (Atlas, uvic), David Grundy (Atlas, uvic), Ti Leggit (University of Chicago), Greg Cross (University of Chicago), Mike Papka (University of Chicago/ANL)

11/7/07, HEPiXVirtual Workspaces: Running jobs : 300 PDSF Fermi VWS/EC2BNL Running jobs : 230 Running jobs : 150 Running jobs : 50 Running jobs : 150 Running jobs : 300Running jobs : 282Running jobs : 243Running jobs : 221Running jobs : 195Running jobs : 140Running jobs : 76Running jobs : 0 Running jobs : 200 Running jobs : 50 Running jobs : 150Running jobs : 142Running jobs : 124Running jobs : 109Running jobs : 94Running jobs : 73Running jobs : 42 Running jobs : 195Running jobs : 183Running jobs : 152Running jobs : 136Running jobs : 96Running jobs : 54Running jobs : 37Running jobs : 0 Running jobs : 42Running jobs : 39Running jobs : 34Running jobs : 27Running jobs : 21Running jobs : 15Running jobs : 9Running jobs : 0 Job Completion : File Recovery : WSU with thanks to Jerome Lauret and Doug Olson of the STAR project

11/7/07, HEPiXVirtual Workspaces: Nersc PDSF EC2 (via Workspace Service) WSU Accelerated display of a workflow job state Y = job number, X = job state with thanks to Jerome Lauret and Doug Olson of the STAR project