1. Enabling Grids for E-sciencE www.eu-egee.org Agreement-based Workload and Resource Management Tiziana Ferrari, Elisabetta Ronchieri Mar 30-31, 2006

2. Enabling Grids for E-sciencE 2 Outline Motivation Use cases SLA and SLS WS-Agreement Agreement Service Agreement Service and WMS

3. Enabling Grids for E-sciencE 3 Motivation Problem: –Efficiency and fairness in case of multiple clients from different Virtual Organizations concurrently accessing Grid services and resources –Differentiated treatment of jobs at different levels of priority Proposal: –A novel resource-independent set of workload and resource management services allowing the establishment of Service Level Agreements (SLAs) between users and service providers  Resource allocation enforcement  SLA establishment and monitoring

4. Enabling Grids for E-sciencE 4 Resource allocation: use cases NETWORK: Data replication:  guaranteed average bandwidth  to optimize performance of a data transfer session (that otherwise would compete with other streams and would be subject to variable throughput)  to support file transfer with deadline (to synchronize job execution with input file transfer) COMPUTING:  to reserve computing resources (e.g. worker nodes, CPU cycles) in presence of a large number of other competing jobs STORAGE:  to guarantee that a sufficient amount of space is present in a data storage device (Storage Element) to save the output data produced by job “J”  the Storage Element is close to the selected computing resources (Computing Element), which are reserved to job “J”

5. Enabling Grids for E-sciencE 5 Service Level Agreement and Specification Service Level Agreement (SLA) –supplies administrative information (e.g., the identity of the entities involved in the contract, and the penalties applicable to the parties when the SLA guarantees are not honored) –quantitatively defines:  the performance level for the service requested  the obligations for the parties involved in the contract –is typically specified through a template containing both quantitative and qualitative information Service Level Specification (SLS) –is a set of attributes and values describing the profile of the requested performance level

6. Enabling Grids for E-sciencE 6 Agreement Services and Service Providers (GGF GRAAP WG) WS-Agreement Conceptual Layered Service Model Agreement Service (collective layer) Service Provider (resource layer)

7. Enabling Grids for E-sciencE 7 WS-Agreement defines a language and a protocol for –Advertising the capabilities of providers –Checking compliance to pre-defined templates –Creating agreements based on creational offers Agreement Layer: –Agreement initiator vs agreement responder –Responder: advertising, creation, delegation of provisioning, monitoring of agreements Service Layer –Service consumer vs service provider –is a resource-specific layer of a provided service Agreement Offer –XML Schema WS-Agreement specification WS-AG GRAAP WG, GGF

8. Enabling Grids for E-sciencE 8 Agreement initiators and service providers The Agreement Service is designed to interact with various initiators and Reservation and Allocation Service Providers (RASPs):

9. Enabling Grids for E-sciencE 9 Agreement Offer: XML Structure

10. Enabling Grids for E-sciencE 10 WS-AG Agreement template (1/2) <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:resourceType"> StorageElement <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:serviceProviderURI"> http://ladyoscar.cnaf.infn.it/ <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:serviceProviderURI"> http://testbed006.cnaf.infn.it/ Agreement Terms Guarantee Terms Context Name Service Description Terms

11. Enabling Grids for E-sciencE 11 WS-AG Agreement template (2/2) <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:duration"> 2000 <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:typeOfSpace"> durable <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:sizeOfTotalSpaceDesiredInBytes"> 2000000 <ns3:ServiceDescriptionTerm xsi:type="ns3:ServiceDescriptionTermType" ns3:ServiceName="spaceManagement:sizeOfGuaranteedSpaceDesiredInBytes"> 100000000

12. Enabling Grids for E-sciencE 12 Agreement Service (1/2) We propose a fully integrated Agreement Responder service named: Agreement Service (AS) Functionality: –Interacts with one or more service providers –Translates high-level service description terms (from initiator) to low level service-specific terms –Advertises the Service Provider capabilities through agreement templates. The template is an XML document that describes the contract skeleton. –Handles the agreement negotiation (agreement offer attributes from the initiator are tuned during the negotiation phase) –Provides information about: s tatus of the agreement negotiation process and attributes of a specific agreement instance

13. Enabling Grids for E-sciencE 13 Agreement Service (2/2) Benefits: 1.It hides the complexity of the service providers’ interfaces from the agreement initiator (the client). 2.It exposes a single operation which can be used independently of the nature of the agreement type (createAgreement). 3.Scalability: new Service Providers can be easily integrated by advertising the corresponding templates (clients can be notified when new templates are added).

14. Enabling Grids for E-sciencE 14 Co-Allocation Agreement Service Co-Allocation Agreement: a composition of dependent “atomic” agreements. Several types of dependency are possible: –Temporal: resources from different agreements need to be allocated in the same time interval; –Resource identity: resources allocated by different agreement need to satisfy some properties (e.g. Space is reserved on SE_i, computing is reserved on CE_j, and SE_i and CE_j are close to each other). Atomic transactions (?) Not currently addressed by the GRAAP WG at the GGF

15. Enabling Grids for E-sciencE 15 (Co-)Allocation and the WMS Reservation Manager: a proposed component of the WMS; it should handle: –Simple agreements (to start with) –Compound agreements (next in development roadmap) Why integrated in the WMS? –Agreement offer submission: similar to job submission –Agreement status: information from Logging and Bookeeping –Agreement Service discovery: a list of Agreement Services is tried (more powerful than submission to a single Agreement Service)  based on matchmaking library  takes advantage of the ISM –Agreement offers can be re-submitted periodically in case of failure

16. Enabling Grids for E-sciencE 16 Agreement Service and Workload Management System Information SuperMarket (ISM) collects data –It is periodically refreshed by the ISM Updater Logging and Bookkeeping collects information about jobs, events, tasks, resources, SLA status Matchmaker discovers the resources that satisfy the task requirements Task Queue stores user requests whose submission or execution failed