University of Catania, Engineering Faculty Department of Computer Science and Telecommunications Report about the Development of Scheduling Algorithm in the Resource Broker S. Cavalieri and S. Monforte

Introduction State of the implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks

Introduction State of the implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks

Introduction State of the implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks Current GRIS/GIIS Information Architecture u Information Services (i.e. grid-info-host, globus-version etc.) on GRISes publish LDIF objects representing their resources in LDAP directories. u None of these informations are sent to the GIIS. u The GIIS maintains a directory (grid-info-site-regdir) as an index of registered GRISes (reg...conf) GIIS GRIS gridct1.ct.infn.it GRIS gridct2.ct.infn.it GRIS gridct3.ct.infn.it globus-version grid-info-host globus-gram-scheduler globus-version grid-info-host globus-gram-schedulerglobus-versiongrid-info-host globus-gram-scheduler grid-info-site-regdirreg.2163.gridct1.ct.infn.it.confreg.2163.gridct2.ct.infn.it.confreg.2163gridct3.ct.infn.it.conf

Design Issues One ClassAd One LDAP Entry All in One Final Remarks The Resource Broker u Implementation of the scheduling functions performed by the Resource Broker, is based on the following architecture: GIIS GRIS GRIS GRIS Resource Broker Resources Published in LDAP trees ClassAd Description of GRISes Resources JDL ClassAd Description of Job Requirements Best Matching GRIS Introduction State of the implementation

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks Implemented Functions u Search of all the available resources. Through the analysis of the content of the directory “grid-info-site-regdir” where GIIS maintains the list of GRIS referring to it, information useful to localisation and access to each GRIS is obtained. u For each GRIS that can be reached from the GIIS, information published are retrieved through LDAP. u Collected resources are converted in terms of ClassAds and maintained by an ad hoc data structure.

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks Implemented Functions u Two objects have been developed in order to provide an abstraction over the resources published in a LDAP directory tree of a GRIS as well as regarding the architecture designed for connecting GRISes and GIIS. u A GRISResourceWrapper is instantiated for each GRIS supplying the GIISResourceWrapper with a ClassAd View of its resources. u The GIISResourceWrapper provides for a coherent and easy to query view over the entire site resources, ready to be used for matching.

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks Design Issues u What is the best design for the GRISResourceWrapper ? u How many ClassAds should be mantained for each GRISResourceWrapper ? u Which attributes should be mantained ? u How and when a GRISResourceWrapper should be updated ?

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks What is the best design for the GRISResourceWrapper ? Three Solutions: u One ClassAd, One LDAP Entry u One ClassAd for all the LDAP Entries u Nested ClassAd (not discussed in the presentation)

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks One ClassAd, One LDAP Entry… u A GRISResourceWrapper maintains a collection of ClassAds, one for each entry of the LDAP directory, which contains the resources published by a globus information service. [ … schedulertype=”condor”…]globus-gram-scheduler grid-info-host ostype=”linux” …] [ … ostype=”linux” …] globus-version [ … releaseDate=2000/04/11 …] [ … releaseDate=2000/04/11 …] ClassAd View of Resources

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks One ClassAd, One LDAP Entry… u The ClassAd expression of the job requirements may contain attributes published by different information services, thus owned by different ClassAd of the collection the GRISResourceWrapper stores. [… requirements= other.ostype==”linux” && other.schedulertype==”condor …][… …] JobAd [ … schedulertype=”condor”…] ostype=”linux” …] [ … ostype=”linux” …] [ … releaseDate=2000/04/11 …] [ … releaseDate=2000/04/11 …] GRIS ResourceAds

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks One ClassAd, One LDAP Entry… u Since the MatchClassAd specialization performs a bilateral match between two and only two ClassAds, it will return FALSE for every ClassAd describing the GRIS resources. GIIS Resource Broker ? [… requirements= other.ostype==”linux” && other.schedulertype==”condor …][… …] JobAd [ … schedulertype=”condor”…] ostype=”linux” …] [ … ostype=”linux” …] [ … releaseDate=2000/04/11 …] [ … releaseDate=2000/04/11 …] GRIS ResourceAds No Such GRIS

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks All in One… u The second approach is based on the use of just one ClassAd describing all the resources published by a given GRIS. [ … releaseDate=2000/04/11schedulertype=”condor”ostype=”linux”…] ResourceAdglobus-versiongrid-info-host globus-gram-scheduler

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks All in One… u On the other hand, it should be kept in mind that two different entries of the LDAP directory might contain one or more attributes with the same name, but different values. u Therefore, any new value for an attribute already inserted in the ClassAd must appended, thus creating a list of values. grid-info-host globus-gram-schedulerCONDOR [ … releaseDate=2000/04/11schedulertype={”condor”,”lsf”}ostype=”linux” … ] ResourceAdglobus-versionglobus-gram-schedulerLSF

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks All in One… u Nevertheless, the match between the ResourceAd and the JobAd does not work yet. GIIS Resource Broker ? GRIS No Such GRIS [… requirements= other.ostype==”linux” && other.schedulertype==”condor …][… …] JobAd ResourceAd [ … releaseDate=2000/04/11schedulertype={”condor”,”lsf”}ostype=”linux” … ]

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks All in One… u The problem is within the description of the job requirements in ClassAd, that is, the operator == does not work when applied to an attribute whose value is a list. The member function must be rather used. GIIS Resource Broker ? GRIS Match Ok [… requirements= other.ostype==”linux” && Member(“condor”,other.schedulertype)…][… Member(“condor”,other.schedulertype)…] JobAd ResourceAd [ … releaseDate=2000/04/11schedulertype={”condor”,”lsf”}ostype=”linux” … ] u It should be pointed out that the JDL user must know which attribute requires the member function for a feasible matching.

Introduction State of the Implementation Design Issues One ClassAd One LDAP Entry All in One Final Remarks u In the author’s point of view the best solution is to use only one ClassAd representing a subset of the more meaningful attributes, rather than the entire attributes published. u Problem to be discussed: what information is needed for the Broker and who has to publish it