1. From Sandbox to Playground: Dynamic Virtual Environments in the Grid Kate Keahey keahey@mcs.anl.gov Argonne National Laboratory Karl Doering University of California, Riverside Ian Foster Argonne National Laboratory

2. Grid 2004Kate Keahey Realizing the Grid Vision l Quality of Service u Protocol, agreement, advance reservation u The ability to enforce what was agreed on l Quality of Life u Being able to find the right configuration on the Grid

3. Grid 2004Kate Keahey Quality of Service l Some form of control over remote nodes l Enforcement of multiple qualities u CPU, disk, memory, network traffic… l More than per-process enforcement u Process group: a master process starts other processes l Dynamically modifiable to reflect changing policies and state in the Grid l Not just quality of service u Quality of Protection, etc… QoX

4. Grid 2004Kate Keahey Quality of Life l The right node configuration is hard to find u Operating system and architectural differences l Different Linux distributions l 64 bit vs 32 bit u Library signature and versioning u The ability to customize a remote execution environment l Effortless configuration of remote nodes u Subject to policies l Quality of Life for multiple groups of Grid users u Avoiding maintenance nightmare, etc.

5. Grid 2004Kate Keahey We Need a Sandbox l A configurable execution environment, container u Virtualizes Grid Node Configuration u Sandbox = Dynamic Virtual Environment (DVE) l We need to be able to create and manage it u Quota, termination, etc. requirements available technology solutions l How can DVEs be implemented? u Relevance to our needs, quality of solution, etc.

6. Grid 2004Kate Keahey DVE: Interfaces l Implemented as Grid Services u OGSI, WSRF l Factory u Creates and configures a DVE in implementation-specific way l e.g., dynamic account, deploys a VM u Writes/configures access and management policy l E.g., modify the GT3 gridmapfile l DVE Service u Interface providing DVE management l E.g., explicit or soft-state termination (implies policy updates) l Access policy management u Allows for inspecting and modifying DVE properties l E.g., hardware properties such as quota or software configuration

7. Grid 2004Kate Keahey DVE Implementations: Requirements l What is a container? l General u Not require users to e.g., use a specific language l Non-invasive u Proof-carrying code, etc. l Strong protection environment u Otherwise users wont trust sites and sites wont trust users u Isolate users from each other l Fine-grain enforcement l Configurable architecture, software, environment u Configurable environment throughout the software stack u Application software/libraries/licenses l Potentially: execution state u Allow migration

8. Grid 2004Kate Keahey DVEs and the Globus Toolkit Client (1) DN (4) GSH local DVE implementation setuid (3) gridmapfile (5) GRAM (6) Request+GSH (2) DVESservice PEP DVEFactory Service PEP

9. Grid 2004Kate Keahey DVE Implementations l Unix accounts u Pros: efficient, ubiquitous u Cons: very limited enforcement u Enforcement properties can be improved if used in conjunction with other technologies l setrlimit, DSRT, chroot, chown, and others l Sandboxes u VServer: protection, sharing and fine-grain enforcement u Pros: efficient, fine-grain enforcement, typically very lightweight u Cons: limited state enforcement, configuration flexibility u Adjustments needed to fully leverage fine-grain enforcement

10. Grid 2004Kate Keahey DVE Implementations (cntd) l Virtual Machines u VMware (not evaluated, but very promising: Xen) u Pros: l Flexibility (run linux on linux, 32 on 64-bit, etc.) l Enhanced security, audit forensics, etc. l Great user state management l Freezing/migration l Customized environment l A promising distribution/deployment tool u Cons: l Potential for being less efficient (emulation) l Potential for resource overhead l Poor implementation of sharing, relatively little enforcement (but can be combined with other technologies for enforcement) l Maturity issues u The potential is excellent, but needs more work

11. Grid 2004Kate Keahey The Need for Speed Comparison using the Fusion EFIT application

12. Grid 2004Kate Keahey Other efficiency concerns l Startup time l Resource usage overhead u Memory use: VMware: 24MB + 1 MB per 32 MB memory allocated u Disk use: large for VMware Table 1: DVE create/destroy times LinuxVServerVMware Create100 ms360 ms14-52 sec Destroy70 ms200 ms3-38 sec

13. Grid 2004Kate Keahey Enforcement Capabilities Unix accountVServerVMware CPU usage (sec)Via setrlimit()Not at present, but could be added Not enforced CPU usage (%)Not enforcedLimited: no VServer can starve another Not in VMware Workstation Disk space usageDynamically (per-user quotas) Dynamically (per context quotas) Statically (virtual disks) Memory usageNoNot at present, but could be added Statically Network usageNoDynamically

14. Grid 2004Kate Keahey DVE Comparison l Dynamic Accounts u Adduser versus pooled accounts u A limited but one that is here to stay… at least for now l VServer u Interesting: sharing and efficiency l VMware u No sharing u Least efficient u Migration, flexibility, etc. l General criteria u Efficiency: very acceptable, also see Xen u Enforcement: uneven, needs more research u Virtual Machines lead as far as configurability and user state representation u Sharing l Potential for replication l One VM per machine model?

15. Grid 2004Kate Keahey Implementation Status l Prototype available (GT 3.2) u Karl Doering: http://www- unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htmhttp://www- unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm l GT4 Implementation u adduser versus account pools u Better policy handling l Virtual machines and other implementations u Work in progress u SC04 poster: l P05: Quality of Life in the Grids: VMs Meet Bioinformatics Applications, with T. Freeman and D. Galron

16. Grid 2004Kate Keahey From Sandbox to Workspace l Virtual Workspaces u VWs are represented by an ontology description l Virtual resource characteristics, software stack, etc. l Potentially integrating community policy l They can be copied, etc. u They can be implemented using different technologies u They can be customized by the user u Deployed, managed and terminated in implementation-specific way l Entails some changes to the architecture

17. Grid 2004Kate Keahey Virtual Workspaces in the Grids Client request VW EPR inspect and manage deploy & suspend use existing VW Create VW VW Factory VW Repository VW Manager create new VW Resource VW start program

18. Grid 2004Kate Keahey From Sandbox to Playground l How will this affect interactions in the Grid? u Other than add many new capabilities l A larger role for the virtual organization u Account screening process: resource owner -> virtual organization l Should a VO be a legal entity? l Needs new privileges if takes on more responsibility u Administration of VWs l VW repository and other services, potentially VW certification l Sharing between VWs u More policies l Changes to many Grid services u May depend on the implementation we use u Security, networking, potentially others l Top-down model for building a Grid u Define a Grid in terms of requirements

19. Grid 2004Kate Keahey Conclusions l For Grids to scale we need a way to create and manage remote environments in the dynamically and effortlessly u Implementations will vary l Virtual is the new Real! u VMs present a very compelling solution… l Efficiency, flexibility, migration, etc. u …and introduce some new challenges l New services, different models of sharing, security, etc. l A growing role for Virtual Organizations l Policy, Policy, Policy… u Policy of resource owners, VOs, users… u Using WS-Agreement to negotiate virtual workspaces? u Have we exchanged one problem for another? l www.mcs.anl.gov/~keahey