Leading the pervasive adoption of grid computing for research and industry © 2005 Global Grid Forum The information contained herein is subject to change without notice JSDL Stephen McGough With thanks to: Ali Anjomshoaa, Fred Brisard, Michel Drescher, Donal K. Fellows, William Lee, An Ly, Darren Pulsipher, Andreas Savva, Chris Smith 10 May 2006

2 Describing a Job Submission: JSDL Job Submission Description Language (JSDL) −A language for describing the requirements of jobs for submission −Declarative description A JSDL document describes the job requirements −Job identification information −Application (e.g., executable, arguments) −Required resources (e.g., CPUs, memory) −Input/output filesJob IT Infrastructure JSDL

3 OGSA BES system WS Gateway WS Clients Job Manager A Grid Information Service Local Information Service Local resource (e.g., Supercomputer) Existing DRM Super Scheduler, or Broker, or … JSDL Here And Here JSDL Here And Here JSDL Document Usage

4 JSDL Introduction JSDL stands for Job Submission Description Language −A language for describing the requirements of computational jobs for submission to Grids and other systems. A JSDL document describes the job requirements −What to do, not how to do it JSDL does not define a submission interface or what the results of a submission look like −Or how resources are selected, or … The JSDL-WG is now considering its next steps. JSDL 1.0 is published as GFD-R-P.56 −Includes description of JSDL elements and XML Schema −Available at

5 Why do we need JSDL? There are many languages out there for submitting jobs to the Grid – so why do we need JSDL? −This is the problem. We didn’t have a standard for job submission −Each system uses its own language −Hence interoperability is difficult (if not impossible) We set out to develop one language for all to use −A standard language that we can use between systems −Or for submitting to any system This is especially important for Workflow

6 JSDL Document (1) A JSDL document is an XML document It may contain −Generic (job) identification information −Application description −Resource requirements (main focus is computational jobs) −Description of required data files Out of scope, for JSDL version 1.0 −Scheduling −Workflow −Security −…

7 JSDL Document (2) A JSDL document is a template … −It can be submitted multiple times; and therefore can be used to create multiple job instances … so JSDL 1.0 does not define attributes to describe the state of a running job −No start time, end time, submission status, or even JobID A JSDL document can be composed with other languages (open content model) −For example to express scheduling, security, etc, requirements in more detail −Within other documents - Workflows

8 Workflow Job JSDL: Conceptual relation with other standards JSDL RRL SDL WS-A JLM JPL … … Job JSDL RRL SDL WS-A JLM JPL … … Job JSDL RRL SDL WS-A JLM JPL … … Job JSDL RRL SDL WS-A JLM JPL … … SDL – Scheduling Description Language JPL – Job Policy Language RRL - Resource Requirements Language JLM – Job Lifetime Management WS-A – WS-Agreement

9 BES JSDL Document Life Cycle A JSDL document may be −Abstract Only the minimum information necessary −For example, application name and input files Runnable at sites that understand this level of description −Refined More detail provided −Target site, number of CPUs, which data source May be refined several times −Tied to a specific site/system Incarnated (Unicore speak); or Grounded (Globus speak) This model is supported/allowed but not required by JSDL

10 A few words on JSDL and BES JSDL is a language −No submission interface defined (on purpose) −JSDL is independent of submission interfaces BES is defining a Web Service interface which consumes JSDL documents −This is not the only use of JSDL −Though we do like it BES Container JSDL

11 JSDL Document Structure Overview ? * Note: −None[1..1] −?[0..1] −* [0..n] −+ [1..n]

12 Data Staging Requirement (1) Previous statements included: −“A JSDL document describes the job requirements What to do, not how to do it *” −“Workflow is out of scope.” But … data staging is a common requirement for any meaningful job submission −Especially for batch job submission −No standard to describe such data movements −Don’t want to do a ‘workflow’ language.

13 Data Staging Requirement (2) Our solution −Assume simple model: Stage-in – Execute – Stage-Out −Files required for execution Files are staged-in before the job can start executing −Files to preserve Files are staged-out after the job finishes execution More complex approaches can be used −But this is outside JSDL −You don’t need to use the JSDL Data Staging Stage-InExecuteStage-Out

14 JSDL & Workflow We hope that JSDL can be used as core elements within a workflow language −Such as BPEL −Etc… Action1 Action2 Action3 Action4 JSDL

15 Example Case: GridSAM & Sedna JSDL Job Submission

16 JSDL Adoption The following projects have presented at GGF JSDL sessions and are known to have implementations of some version of JSDL; not necessarily 1.0. −Business Grid −Grid Programming Environment (GPE) −GridSAM −HPC-Europa −Market for Computational Services −NAREGI −UniGrids The following groups also said they are or will be implementing JSDL: −DEISA −GridBus Project (see OGSA Roadmap, section 8) −gridMatrix (Cadence) (presentation) −Nordugrid Also within GGF a number of groups either use directly or have a strong interest or connection with JSDL: −BES-WG, CDDLM-WG, DRMAA-WG, GRAAP-WG, OGSA-WG, RSS-WG An up-to-date version of this list is on Gridforge:

17 JSDL Mappings ARC (NorduGrid) Condor eNANOS Fork Globus 2 GRIA provider Grid Resource Management System (GRMS) JOb Scheduling Hierarchically (JOSH) LSF Sun Grid Engine Unicore

18 Questions

19 GGF Full Copyright Notice Copyright (C) Global Grid Forum (2006). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the GGF or other organizations, except as needed for the purpose of developing Grid Recommendations in which case the procedures for copyrights defined in the GGF Document process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the GGF or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE GLOBAL GRID FORUM DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."