1. Alexandre Duarte CERN Fifth EELA Tutorial Santiago, 06/09-07/09,2006
Special Jobs
Alexandre Duarte
CERN
Fifth EELA Tutorial
Santiago, 06/09-07/09,2006

2. Outline MPI jobs on gLite DAG Job Collection Parametric jobs
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

3. MPI Overview
Execution of parallel jobs is an essential issue for modern informatics and applications.
Most used library for parallel jobs support is MPI (Message Passing Interface)
At the state of the art, parallel jobs can run inside single Computing Elements (CE) only;
several projects are involved into studies concerning the possibility of executing parallel jobs on Worker Nodes (WNs) belonging to different CEs.
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

4. Requirements & settings
In order to guarantee that MPI job can run, the following requirements MUST BE satisfied:
the MPICH software must be installed and placed in the PATH environment variable on each WNs of the CE.
Some MPI’s applications require a shared filesystem among the WNs to run.
The variable VO_<name_of_VO>_SW_DIR will contain the name of a directory in case of SHARED filesystem.
The variable VO_<name_of_VO>_SW_DIR will contain “.” if there is NO SHARED filesystem.
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

5. From the user’s point of view, jobs to be run as MPI are specified setting the JDL JobType attribute to MPICH and specifying the NodeNumber attribute as well.
E.g.: …
JobType = “MPICH”;
NodeNumber = 4;
This attribute defines the required number of CPUs needed for the application
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

6. When the previous two attributes are included in a JDL, the User Interface (UI) automatically adds the following expression:
to the JDL Requirements expression in order to find out the best resource where the job can be executed.
(other.GlueCEInfoTotalCPUs >= NodeNumber) &&
Member (“MPICH”,other.GlueHostApplicationSoftwareRunTimeEnvironment)
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

7. MPI exercise
Create the file “mpi-test.jdl” inside $HOME/gLite/Other and put this contents inside the file: [
Type = "Job";
JobType = "MPICH";
Executable = “cpi";
NodeNumber = 2;
StdOutput = “cpi.out";
StdError = “cpi.err";
InputSandbox = {"cpi"};
OutputSandbox = {“cpi.err",“cpi.out"};
RetryCount = 0;
Requirements = Other.GlueCEUniqueID=="gildace01.roma3.infn.it:2119/jobmanager-lcgpbs-infinite"; ]
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

8. MPI submission
Other]$ glite-job-submit -o id mpi-test.jdl
Selected Virtual Organisation name (from proxy certificate extension): gilda
Connecting to host santiago01.reuna.cl, port 7772
Logging to host santiago01.reuna.cl, port 9002
======================= glite-job-submit Success ==========================
The job has been successfully submitted to the Network Server.
Use glite-job-status command to check job current status. Your job identifier is: -
The job identifier has been saved in the following file:
/home/santiago05/gLite/Other/id
======================================================================
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

9. MPI status and output
Query the status of the job using the following command:
Other]$ glite-job-status -i id
…………………………………………….
When the status of the job is “DONE”, you can retrieve output with
the following command:
Other]$ glite-job-output -i id
……………………………………………
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

10. MPI on the web… LCG-2 User Guide Manuals Series
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

11. Workload Manager Proxy
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

12. WMProxy overview WMProxy (Workload Manager Proxy)
is a new service providing access to the gLite Workload Management System (WMS) functionality through a simple Web Services based interface.
has been designed to efficiently handle a large number of requests for job submission and control to the WMS
the service interface addresses the Web Services and SOA architecture standards, in particular adhering to WS-I
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

13. WMProxy : submission & monitoring
In order to submit jobs with WMProxy, it’s mandatory to delegate credentials:
The submission/monitoring commands are slightly different, but most of the “old” options are supported
glite-wms-job-delegate-proxy -d del_ID
glite-wms-job-submit -d del_ID collection.jdl
glite-wms-job-status jobID
glite-wms-job-output jobID
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

14. DAG job
A DAG job is a set of jobs where input, output, or execution of one or more jobs can depend on other jobs
Dependencies are represented through Directed Acyclic Graphs, where the nodes are jobs, and the edges identify the dependencies
nodeA
nodeB
nodeC
NodeF
nodeD
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

15. JDL structure
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

16. Attribute: Nodes
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

17. Attribute: Dependencies
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

18. DAG jdl [
type = "dag";
max_nodes_running = 4;
nodes = [
nodeA = [
file ="nodes/nodeA.jdl" ; ];
nodeB = [
file ="nodes/nodeB.jdl" ;
nodeC = [
file ="nodes/nodeC.jdl" ;
nodeD = [
file ="nodes/nodeD.jdl";
dependencies = {
{nodeA, nodeB},
{nodeA, nodeC},
{ {nodeB,nodeC}, nodeD } } ]
Node description could also be done here, instead of using separate files
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

19. Job Collection
A job collection is a set of independent jobs that user wants to submit and monitor via a single request
Jobs of a collection are submitted as DAG nodes without dependencies
JDL is a list of classad, which describes the subjobs [
Type = "collection";
VirtualOrganisation = “gilda";
nodes = {
[ <job descr 1 >],
[ <job descr 2 >], … }; ]
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

20. ‘Scattered’ Input Sandboxes
Input Sandbox can contain
file paths on the UI machine (i.e. the usual way)
URI pointing to files on a remote gridFTP/HTTPS server
InputSandbox = {
"gsiftp://neo.datamat.it:2811/var/prg/sim.exe",
"
"file:///home/pacio/myconf“ };
A base URI to be applied to all sandbox files can also be specified
InputSandboxBaseURI = "gsiftp://matrix.datamat.it:2811/var";
Only local files (file://) are uploaded to the WMS node
File pointed by URIs are directly downloaded on the WN by the JobWrapper just before the job is started
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

21. ‘Scattered’ Output Sandboxes
JDL has been enriched with new attributes for specifying the destinations for the files listed in the OutputSandbox attribute list
OutputSandbox = { "jobOutput",
"run1/event1",
"jobError" };
OutputSandboxDestURI = {
"gsiftp://matrix.datamat.it/var/jobOutput",
"
"gsiftp://matrix.datamat.it/var/jobError" };
A base URI to be applied to all sandbox files can also be specified
OutputSandboxBaseDestURI = "gsiftp://neo.datamat.it/home/run1/";
Files are copied when the job has completed execution by the JobWrapper to the specified destination without transiting on the WMS node
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

22. Job collection example[
type = "collection";
InputSandbox = {"date.sh"};
RetryCount = 0;
nodes = {
file ="jobs/job1.jdl" ; ],
Executable = "/bin/sh";
Arguments = "date.sh";
Stdoutput = "date.out";
StdError = "date.err";
OutputSandbox ={"date.out", "date.err"}; ]
file ="jobs/job3.jdl" ; };
All nodes will share this Input Sandbox
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

23. Parametric Job
A parametric job is a job where one or more of its attributes are parameterized
Values of attributes vary according to a parameter
Job monitoring / managing is always done through an unique jobID, as if the job was single (see submission of collection [
JobType = "Parametric";
Executable = "/bin/sh";
Arguments = "md5.sh input_PARAM_.txt";
InputSandbox = {"md5.sh", "input_PARAM_.txt"};
StdOutput = "out_PARAM_.txt";
StdError = "err_PARAM_.txt";
Parameters = 4;
ParameterStart = 1;
ParameterStep = 1;
OutputSandbox = {"out_PARAM_.txt", "err_PARAM_.txt"}; ]
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

24. Parametric job / 2 Parameter can be also a list of string
InputSandbox (if present) has to be coherent with parameters
[ui-test] /home/giorgio/param > cat param2.jdl [
JobType = "Parametric";
Executable = “/bin/cat";
Arguments = “input_PARAM_.txt”;
InputSandbox = "input_PARAM_.txt";
StdOutput = "myoutput_PARAM_.txt";
StdError = "myerror_PARAM_.txt";
Parameters = {earth,moon,mars};
OutputSandbox = {“myoutput_PARAM_.txt”}; ]
[ui-test] /home/giorgio/param > ls
inputEARTH.txt inputMARS.txt inputMOON.txt param2.jdl
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006

25. References JDL attributes specification for WM proxy
WMProxy quickstart
WMS user guides
5th EELA TUTORIAL - USERS - Santiago, 06/09-07/09,2006