Virtual Playground: from Quality of Service to Quality of Life in the Grids Katarzyna Keahey Argonne National Laboratory

06/26/04Katarzyna Keahey: PSNC Seminar What are computational Grids? l They are like power Grids: remote computing power can be accessible from your laptop in much the same way as electricity is delivered to your home

06/26/04Katarzyna Keahey: PSNC Seminar Are they really? l If power grids were like computational grids… u A simple task, like making a toast: l Where is/are your power station(s)? l How much power do they produce? l Does the power supply pattern vary through the day? l It is not clear who would be charged for your toast… l What if your power station goes down? u Unless you can handle these questions… no toast! l We need: u Quality of Service: reliable access to required amount of computational power when you need it u Quality of Protection: data secure in predictable ways u Quality of X: you name it u Last but not least: Quality of Life!

06/26/04Katarzyna Keahey: PSNC Seminar Enter Agreement-Based Interactions l Traditional Resource Management Model u Assumption of global ownership u Provider-centric: utilization as primary metric u Closed System l Dynamic Policy-driven Resource Management u Resources in a Global Context l Advertisement and dynamic negotiation/renegotiation l Resource maintains autonomy u Clients or Agents Negotiate for Resources l WS-Agreement u Dynamic, decentralized, policy creation and adjustment u Abstracts impact of other policies and users l Operating policies may be proprietary, global knowledge is hard to achieve, etc.

06/26/04Katarzyna Keahey: PSNC Seminar What is WS-Agreement? l An agreement defines a dynamically-established and dynamically-managed relationship between parties. The object of the relationship is the delivery of a service by one of the parties within the context of the agreement. The management of this delivery is achieved by agreeing on the respective roles, rights and obligations of the parties. The agreement may specify not only functional properties for identification or creation of the service, but also non-functional properties of the service such as performance or availability. Entities can dynamically establish and manage agreements via Web service interfaces. l How is agreement different from policy? u Policy is static u Agreements are dynamic: they provide mechanisms and protocols for creating and changing policies on the fly u Agreement may imply an action l Information about the action enters the system as well as a policy allowing that action

06/26/04Katarzyna Keahey: PSNC Seminar WS-Agreement (cntd) l What do agreements look like? u Agreements will often be finer-grained than policies u Agreements may represent an ephemeral or periodic policy u They may involve late binding u There may be a large volume of agreements l Other agreement properties u Simple way of expressing the aggregate of policies in the system u May hide proprietary policies from the user u Allow providers to gauge demand u Establish policies to reconcile conflicting terms and goals l Client: deadlines and availability goals l Applications: need coordinated resources l Resource owners: localized utilization goals, satisfying demand

06/26/04Katarzyna Keahey: PSNC Seminar WS-Agreement Model l Negotiation Layer u For managing Agreements u Complex dialogues u Coordination l Agreement Layer u Externalized policy u For managing service providers l Service Layer u Policy State l Affects service u Domain-specific Application Instance Factory Policycreate() foo() create() Agreement Ops: terminate(limits) inspect(query)... SDEs: TermsRelatedStatus Agrmts. inspect() Factory ConsumerProvider Manager Negotiation Ops: terminate(limits) negotiate(...)... SDEs: TermsRelatedStatus Agrmts. Factory create() negotiate() Negotiator

06/26/04Katarzyna Keahey: PSNC Seminar Agreement Abstractions l Agreement Factory u Persistent service u For example: façade to scheduler(s) u Advertises agreement template l Advertises what provider has to offer (contraints/offers) u Creates Agreements l Agreement u Transient resource u Allows inspection of terms u For example: job entry virtualized into a service u Lifetime maps to lifetime of terms of service u Allows the client to renegotiate

06/26/04Katarzyna Keahey: PSNC Seminar Agreement Negotiation l Agreement template -> acceptable agreement l Phases: u Negotiation admission u Iteration u Acceptance u Commitment l Fine-grained versus coarse-grained negotiation u Term-level versus agreement-level u Compromise between practicality and efficiency l Commitment models

06/26/04Katarzyna Keahey: PSNC Seminar Claiming Agreements l Claiming: associate an agreement with an entity u apply_agreement(A,X); u Considered domain-specific l Different claiming models u No claiming required: the actions to be taken are sufficiently well described by the agreement u Event-based claiming u Late binding l apply this resource reservation to this execution l Disassociate agreements u Bind an agreement to a different entity

06/26/04Katarzyna Keahey: PSNC Seminar Challenges l Synchronization u Commitment u When does a renegotiated agreement replace the current one? l Security and ownership u Transferability u Chaining agreements u Can agreements be tokens u Security roles l Agreement Terms l Composing agreements l Directionality l Other concerns u Tools and Usability u Performance

06/26/04Katarzyna Keahey: PSNC Seminar The National Fusion Collaboratory: Using Agreements for Science paper: Grids for Experimental Science: the Virtual Control Room CLADE 2004

06/26/04Katarzyna Keahey: PSNC Seminar Shared Application Real Time Data Display Video & Audio Between Pulse Data Shot Cycle Status Fusion Grid Remote control room experimental mock SC03 November, 2003

06/26/04Katarzyna Keahey: PSNC Seminar Grid service Usage (NFC) l Agreements for between-pulse execution u Dynamically created application-level agreements l whenever I run a certain configuration of EFIT tomorrow between 9am and 5pm I want it to run as fast as possible and no longer than 4 minutes u Late binding l Agreement claiming is triggered by event: experimental data becomes available l E2E Agreements u Job execution l Based on resource reservation and prediction of run-time u Resource reservation u Data Transfer l Based on prediction, resource management and adaptation u Aggregating agreements u paper: Agreement-Based Interactions for Experimental Science, Europar 2004 l Prototype implementation u Loosely compliant with the WS-Agreement spec

06/26/04Katarzyna Keahey: PSNC Seminar We Have an Agreement: Now What? l Providers role u Resource management l Advance reservation, rate limiting, etc. u Prediction l Is this really an agreement? u Adaptation u All of the above! l Paper: Providing Data Transfer with QoS as Agreement-Based Service, SCC 2004 l Resource management versus prediction u Confidence levels and guarantees

06/26/04Katarzyna Keahey: PSNC Seminar Agreement Enforcement l Effective agreement enforcement u Q: How can we enforce agreements? u A: Execution environment with QoS enforcement l Aspects of an execution environment u Configurable, virtual resource l Enforcement of CPU, disk memory usage l Sandboxes, virtual machines, etc. u Software configuration l Defined environment in terms of user-defined qualities: files, library signature, etc. l Dynamically, flexibly, remotely, automatically configurable… u To reflect potentially changing nature of agreements and policies in the Grid

06/26/04Katarzyna Keahey: PSNC Seminar Virtual Workspaces Virtual resource configuration Protection environment Software and file configuration state Execution state Virtual Workspace Grid Middleware Interface Grid client Interface Grid clients Grid middleware interface l Define interfaces and explore a variety of implementations l Describe a VW using an ontology u Properties of an implementation (e.g. how to deploy it on the Grid) u Content of the workspace

06/26/04Katarzyna Keahey: PSNC Seminar Properties Virtual Workspaces l Dynamic creation u Inherent property of Grid computation l Examples of existing mechanisms: single sign-on, GRAM u Avoiding a maintenance nightmare (automate administration) and potential security hazard l Dynamic configuration u To reflect changing policies in the Grid (implement agreements) l Strong protection environment u Otherwise users wont trust sites and sites wont trust users l Fine-grain enforcement l Configurable architecture, software, environment u Application software/libraries/licenses u Configurable environment u Running 32-bit programs on 64-bit architectures u Running a required version of the OS (Fedora vs. RH9) u Potentially even execution state

06/26/04Katarzyna Keahey: PSNC Seminar Technologies l Unix accounts and Unix account tools u setrlimit, DSRT, chroot, chown, etc. l Deployment & configuration tools u Pacman & pacman cache l Sandboxes u VServer, protection and fine-grain enforcement l Virtual Machines u VMware, Xen

06/26/04Katarzyna Keahey: PSNC Seminar Interacting with VWs Client request VW EPR inspect and manage deploy & suspend use existing VW Create VW VW Factory VW Repository VW Manager create new VW Resource VW start program

06/26/04Katarzyna Keahey: PSNC Seminar Services l Factory u Creates VW in terms of the implementation l e.g., VM image, pacman chache+ u May create based on an already created VW u Writes/configures access policy u May allow negotiation l VW Repository u Access to state describing a VW u Allows inspection, management, implementation-specific termination, potentially renegotiation, etc. u Soft-state lifetime management ensures termination l VW Manager u Lightweight infrastructure deploying VMs

06/26/04Katarzyna Keahey: PSNC Seminar Comparing VW Technologies l Unix accounts u Pros: efficient, ubiquitous u Cons: very limited functionality u Needs to be used in conjunction with other technologies l Pacman, system tools u Prototype available (GT 3.2) l l Sandboxes u Pros: efficient, fine-grain enforcement, typically very lightweight u Cons: limited state enforcement u Need to be used in conjunction with other technologies

06/26/04Katarzyna Keahey: PSNC Seminar Comparing VW Technologies (cntd) l Virtual Machines u Pros: l Flexibility (run linux on linux, 32 on 64-bit, etc.) l Enhanced security, audit forensics, etc. l Great user state management l Freezing/migration l Customized environment l A promising distribution/deployment tool u Cons: l Potential for being less efficient (emulation) l Resource inefficient l Poor implementation of sharing, relatively little enforcement (but can be combined with other technologies for enforcement) u The potential is excellent, but needs more work

06/26/04Katarzyna Keahey: PSNC Seminar What about efficiency? also see: Figuieredo et al, A case for Grid Computing on Virtual Machines

06/26/04Katarzyna Keahey: PSNC Seminar Other efficiency concerns l Startup time l Resource use efficiency u VMware memory use: 24MB + 1 MB per 32 MB memory allocated per VM l Upcoming paper Table 1: DVE create/destroy times LinuxVServerVMware Create100 ms360 ms14-52 sec Destroy70 ms200 ms3-38 sec

06/26/04Katarzyna Keahey: PSNC Seminar Toward Virtual Playground l More than one sandbox = playground l Managing VW-based execution in the Grids u A virtual Grid of interactions l New potential creates new challenges u Networking issues l VM networking issues l VNET u Sharing issues l How do we orchestrate sharing between VMs? l We have not even defined what sharing really means! u How do we represent and manage a playground as a Grid session?

06/26/04Katarzyna Keahey: PSNC Seminar Conclusions l Future Grids will rely on dynamic, decentralized, policy-driven resource management u Need for more flexibility u QoS issues l Agreements are a promising solution to creating policy on the fly l There will be more emphasis on enforcement technologies l Virtual Workspaces offer a comprehensive hardware/software solution u We enforce not only resource usage u Virtual is the new real!