Some ideas for the future Francesco.Giacomini@cnaf.infn.it WP1 Meeting Milano, 11th September 2001

Overview Complex jobs Dependency on Condor Use of existing frameworks

Complex Jobs: Definition A complex job is a set of jobs with dependencies among them. Dependencies are temporal and are given, for example, by I/O requirements. subjob B needs as input the output of subjob A The definition is very vague and generic.

Complex Jobs: (initial) Assumptions Dependencies are not cyclical Otherwise subjobs have to execute concurrently (co-allocation problem) The definition is not recursive A subjob cannot be a complex job Otherwise the semantics of a dependency is more complicated

Complex Jobs: Example A B F C D E The complex job can be represented as a Direct Acyclical Graph (DAG)

Complex Jobs: Simple Solution Represent a DAG with a nested ClassAd (?) “Linearise” the DAG for the example: A C D B E F use a “topological sort” algorithm the solution is not unique For each subjob send it to the Resource Broker independently Data representing a dependency is saved in a “temporary” and “anonymous” file

Complex Jobs: Other Issues Job identifiers Interaction with LB What to do in case a subjob fails Various types of dependencies

Dependency on Condor In my opinion we should get rid as soon as possible of our dependency on those parts of Condor which will not likely become Open Source (TM) Starting with Condor-G job submission is not particularly difficult directly on top of Globus job monitoring should be done based on info in the LB

Implementing ComputingElement Implemented the ComputingElement interface directly on top of Globus, so that one can do: ComputingElement ce(“pcfg.cnaf.infn.it”); Job job; job.set_attribute(“executable”, “/bin/sleep”); job.set_attribute(“arguments”, “10”); ce.submit(job); ce.cancel(job);

Using Existing Frameworks We should evaluate and possibly adopt existing (OO) application frameworks, instead or reinventing the wheel there was a comment on this from one of the D1.2 reviewers One possibility is the Adaptive Communication Environment (ACE) (www.cs.wustl.edu/~schmidt/ACE.html)