A Resource Management Architecture for Metacomputing Systems Karl Czajkowski Ian Foster Nicholas Karonis Carl Kesselman Stuart Martin Warren Smith Steven Tuecke

Plan : Introduction The addressed problems Related work The proposed architecture Emplementation Conclusion

Introduction Globus metacomputing toolkit problems related to locating and allocating computational resources and the activities required to prepare a resource for use. No concern with scheduling or the management of other resources

Resource management problems Site autonomy: no control over local administration Heterogeneous substrate: many different platforms Policy extensibility: application specific allocation requirements Co-allocation:simultaneous access of resources Online control:need of negotiations to adopt application requirement to resource availability

Related work Systemexample Site autonomy Heterog- eneity Policy extensi- bility On-line control Co- allocation Networked batch queuing systems NQE LSF PBS LOADLEVE LER No LimitedNoyes Wide-area scheduling systems Gallop Yes No Yes Legion No YesNo PRM No Yes No Condor Yes LimitedNo

Proposed architecture

1.Information service Metacomputing Directory Service (MDS):Access to information about current availability,capacity and characteristics of resources. The data presentation and API are adopted from Lightweight Directory Access protocol (LDAP) MDS information can be used by other tools as the Globus View tool

1.Recourse specification language to communicate requests for resources between components of the system The syntax:

Recourse specification language Resource brokers,co-allocators and resource managers can each define a set of parameter- names Tow types of parameter-name for the resource manager: – MDS attribute names – Scheduler parameters Example:

Recourse Brokers Use information maintained locally from MDS or the information contained in the specification to specialize the specification Composing the behaviors of several brokers leads to identify a specific resource manager

Recourse Brokers

A Graphical Recourse Selector

Recourse Co-allocatation Some applications require several resources to be allocated simultaneously(mulirequest) The co-allocater  split a mulirequest into its constituent components  Submit each component to a resource manager  Allows the manipulation of the set of resources as a whole

Recourse Co-allocatation Co-allocater services: – Mirror current GRAM semantics:all or nothing – Allocate at least N out of M requested resources then return – Return immediately,but gradually return more resources as they become avaliable

Local Resource Management Globus Resource Allocation Manager GRAM 1. Processing RSL representing resource requests 2. Enabling remote monitoring and management of jobs created in response to a resource request 3. Periodically updating the MDS information service with information about the current availability and capabilities of the resource that it manages

Local Resource Management GRAM Scheduling Model

Local Resource Management GRAM imlementation GRAM implementation

Implementation GUSTO testbed :15 sites,330 computers,3600 processors Local schedulers: LSF,NQE,LoadLeveler, EASY,Fork,CONDOR SF-Express:a large scale distributed interactive simulation application(852 computer nodes on thre different architectures located at six different computer centers controlled by three local resource managers

Conclusion The ability to include constraints on MDS attribute values in RSL specifications provides a powerful mechanism for controlling how an RSL specification is interpreted Need of more sophisticated resource broker and resource co-allocater services Extension of the architecture to encompass other resources