SoRTGrid: A Framework compliant with Soft Real Time requirements A. Merlo 1, V. Gianuzzi 1, A. Corana 2, A. Clematis 3, D. D’Agostino 3, A Quarati 3 1 DISI – Università di Genova, Italy. 2 IEIIT-CNR, Genova, Italy 3 IMATI-CNR, Genova, Italy.

Agenda Grid, Qos and Soft Real Time ◦ QoS over the Grid ◦ Grid and Time Constraints Presenting SoRTGrid ◦ Architecture ◦ “Physiology” ◦ Discovery phase Benefits of SoRTGrid Conclusions and future works

QoS over Grid Grid was born as a paradigm for sharing huge amount of resources for the execution of large batch job. ◦ Requests of huge amount of resources to use for large periods of time ◦ Sufficient support for discovery and selection of resources is provided by metascheduler systems that do not provide guarantees on:  Length of discovery phase  Success in finding and select resource However, since the first dawn of the paradigm, the demand of QoS for execution of different classes of applications has been noticed.

QoS over Grid /2 QoS: “harder issue” than in other distributed context: ◦ High Dynamism  constant change in the set of resource  A certain resource can be available when discovered and no more available when selected.  This produced unpredictable latency. ◦ Virtualization  There is a gap between virtual and real state of a resource  A free selected resource can have a state different than expected and be not suitable for the job requirements.  This provides no guarantees on successful job completion.

QoS over Grid /3 Currently, there are some proposals (Rana, Manascè et al.) for providing guarantees to the Grid User on the availability and quantity of resources. Such approaches underline that: ◦ QoS is a cross-layers issue. ◦ QoS can be granted only if some “guarantees” are provided at lower level. Unfortunately, it is still not possible to provide guarantees on time constraints from the discovery of resources to execution of a job within a deadline.

Grid and Time constraints The set of timed-based applications from the best effort to the Soft Real Time ones could take many advantages in running on the Grid. ◦ Cheapness (less expensive than buying proper machine for managing peaks of computation) ‏ ◦ Availability (resource availability due to pre- reservation) ‏ ◦ Fault-tolerance (redundancy due to the high number of possible available resources) ‏

Time-oriented application In our work, we considered job that are: Deterministic Time-constrained (till Soft Real Time) Our SoRTGrid aims to implement strategies to satisfy the “missing requirements” and make Grid accessible for tasks like: ◦ Coordination of swarms of robot ◦ Real-time management of sensor networks ◦ Advanced streaming applications over Grid (effective TV-on-demand, on-the-fly conversions, …) ‏

Presenting SoRTGrid A Grid framework able to manage time constraints must have: ◦ Resource discovery “in time” ◦ Pre-reservation mechanisms to avoid inconsistencies during the acquisition phase. ◦ Mechanism to assure the effective availability of selected resources The achievement of the above conditions depends strictly on a automatic and proficient interaction between Grid Users and Resource Owners without any centralized metascheduler.

Presenting SoRTGrid /2 User and Owner knows better than a metascheduler their respective realities. SortGrid requires that both parts possess some “abilities” User side: Evaluation of execution time Definition of execution deadline Owner side: Direct control of resources (up-to-date information on their state) Resolution of semantic gap typical of metaschedulers

Architecture of SoRTGrid Interaction between User and Owner are provided by a couple of autonomic agent classes, the User and the Owner one. ◦ User Agent: It uses SoRTGrid to retrieve resource and executes User jobs within their deadline. It provides information on job requirement to SoRTGrid thrugh a proper document, called JRM (Job Requirement Manifest). ◦ Owner Agent: It uses SoRTGrid to “offer” the resources possessed by the Owner to the User Agents under certain QoS level and time conditions. It produces set of SoRTBids that are documents containing a “proposal” on a set of resources.

Architecture of SoRTGrid /2 SoRTGrid act as a framework and provides basical service for sets of User’s and Owner’s Agents. It is composed by a set of independent Facilitator Agents, everyone operating on a certain subpart of the Grid.

Facilitator Agent Facilitator Agent is composed by: ◦ Singleton BidMan Grid Service ◦ Factory DiPe Service ◦ A local SoRTBid repository ◦ User and Owner Agents (and their respective resources) are registered on a certain Facilitator Agent and interact with it only. ◦ Operations (discovery, negotiation, etc) are executed by interaction of Facilitator Agent components. ◦ FA interacts with Grid middleware to obtain basic funcitonalities.

Services provided by SoRTGrid Owner Agent side: ◦ Publication of SoRTBids through the Local Repository. ◦ Offering of SoRTBids to User Agents ◦ Feedbacks on SoRTBids. User Agent side: ◦ Time-constrained discovery of SoRTBids ◦ Pre-selection of SoRTBids.

Owner Agent: SoRTBids and BidMan A SoRTBid contains: ◦ Amount of resources shared ◦ Minimum level of QoS offered ◦ Period of time in which the resource is offered (  P). ◦ Effective execution time  T  ◦ Expiration time. A SoRTBid is published through the BidMan service: ◦ It allows upload/removal of SoRTBids. ◦ It performs maintenance operations and provide statistics.

User Agent: JRM and DiPe A User Agent realizes for Grid User: Job requirements appraisal Job discovery, selection, megotiation and execution within the deadline. A JRM contains information on the job: ◦ Resource requirements ◦ QoS class ◦ Expiration Threeshold ◦ Appraisal Value ◦ Utility function parameters ◦ Number of required SoRTBids The JRM is provided to the DiPe Grid Service to start a discovery and pre-selection of SoRTBids.

SoRTGrid and QoS SoRTGrid (and related SoRTBids) supports three classes of QoS: ◦ Soft Real Time Class. The higher level of QoS; it guarantees the total availability of the resources for  T. ◦ High Level Class. The  T execution time is granted in  P but the job can be interrupted. ◦ Standard Class. The  T execution time is granted only if jobs with higher QoS level do not run on the same resources.

“Physiology” of SoRTGrid

Discovery phase in SoRTGrid It is composed by a Local and a Remote Discovery. Local Discovery provides a selection of SoRTBid on the Local Repository. Remote Discovery is performed only if enough “suitable” bids have not been retrieved and ends when the time allocated for discovery phase has passed. DiPe service provides the following services for the discovery phase: Local Discovery and PRE-RESERVATION Starts Remote discovery Delivery of suitable SoRTBids to UA.

Discovery phase in SoRTGrid /2

Benefits of SoRTGrid Autonomy. The two parts involved work independently for different aims and iteract to maximize the respective results and there is no centralization of any kind. Adaptability. SoRTGrid is defined by a set of parameters, so it it adabptable to different underlying Grid context. Consistency. Interaction between User and Owner Agent grants a flow of up-to-date information. A pre- reservation system assures that a resource remains effectively available for at least a certain time. Automation. The use of agents excludes human part to be involved directly, avoiding undefined latencies, accordingly with the time critical goal fo SoRTGrid.

Conclusions and future works A simulation of SoRTGrid has been implemented through GridSim. We’re simulating different application scenarios. Our research is oriented to the definition of classes of application suitable for the execution in SoRTGrid. Further implementation of negotiation and execution phase.