EDGI European Desktop Grid Initiative gLite job submission to EDGI EDGI is supported by the FP7 Capacities Programme under contract nr RI

ContentsContents Single job submission to EDGI through gLite Single job submission to EDGI through gLite Metajob submission to EDGI through gLite Metajob submission to EDGI through gLite GBAC submission to EDGI through gLite GBAC submission to EDGI through gLite For each type of submission, there will be a step-by-step explanation in order to ease the understanding. 2

Single job submission 3

Step 1: Choose application from the EDGI AR 4 Let us select the dsp app…

Step 2: Get the gridftp url of its glite binary 5 gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp

Step 3: Get the VO and site list and select 6 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

Step 4: create jdl Create your dsp.jdl: Create your dsp.jdl: Executable = "dsp"; Arguments = "-f 22 -i 22 -p 723 -n pools.txt"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp", "pools.txt"}; OutputSandbox = {"cost.txt"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo“; 7 Executable reference in the EDGI AR Input file

Step 5: submit jdl Proxy creation: Proxy creation: > glite-voms-proxy-init -voms edgiprod.vo.edgi-grid.eu:/edgiprod.vo.edgi-grid.eu/Role=edgidemo Submit: Submit: > glite-wms-job-submit -a -o id dsp.jdl Connecting to the service ================== glite-wms-job-submit Success =================== The job has been successfully submitted to the WMProxy Your job identifier is: The job identifier has been saved in the following file: /home/lpds/smith/TEST-EDGIDEMO/DSP/id======================================================== 8

Step 6: get status Status: Status: glite-wms-job-status -i id ================== glite-wms-job-status Success ================= BOOKKEEPING INFORMATION: Status info for the Job : Current Status: Running Status Reason: unavailable Destination: cr1.edgi-grid.eu:8443/cream-pbs-edgidemo Submitted: Wed May 16 14:59: CEST ===================================================== 9

Step 7: logging Logging: Logging: glite-wms-job-logging-info -v 2 -i id Check UserTags! It returns: 3gbridge_internal_id3gbridge_dg_url3gbridge_dg_id The last one is a URL pointing to the WU! 10

Step 8: track the job in BOINC Inspect BOINC WU Inspect BOINC WU 11

Step 9: get the result 12

Metajob submission 13

Step 0: prepare your inputs Upload your individual input files to web server: Upload your individual input files to web server: … Create the description of your metajob: Create the description of your metajob: %Required 100% %SuccessAt 100% %Comment pools1.txt Arguments = "-i 0 -n pools.txt -f 22 -p 723“ Input = pools.txt= Queue[….] %Comment pools10000.txt Input = pools.txt= Queue 14

Step 1: Choose application from the EDGI AR 15 Let us select the dsp app…

Step 2: Get the gridftp url of its glite binary 16 gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp

Step 3: Get the VO and site list and select 17 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

Step 4: create and submit jdl Create your jdl: Create your jdl: Executable = "dsp"; Arguments = "-f 22 -i 22 -p 723 -n pools.txt"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1001/1102/dsp", "pools.txt", "_3gb-metajob-dsp-10000" }; OutputSandbox = {"cost.txt"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo"; Submit: Submit: glite-wms-job-submit -o id my-metajob jdl 18 Reference to the app in the EDGI AR Basic input file Extra input file

Step 5: Query the logging and get the url 19 Location of the status description

Step 6: Query the status 20 Metajob handling rules Actual status of jobs Location of the mapping

Step 7: Download (and extract) the results 21 Download result as usual: glite-wms-job-output -i id --dir outputs =>./outputs/cost.txt Extract it: tar zxvf cost.txt =>./outputs/ /cost.txt See the mapping between your individual job definition and the jobids (which gives the name of directories storing the output files of your app) => one subjob id one job

GBAC job submission 22

EDGIDemo BOINC project … Overview 3G Bridge … BOINC Client GBAC Application and Inputs BOINC Client GBAC Application and Inputs 1.A User submits her application to EDGIDemo trough gLite. Using GBAC as a virtualization framework requires the real binaries and inputs defined as GBAC inputs. 2.Clients (who have VirtualBox installed) will download the BOINC native GBAC application with the submitted application and its inputs. 3.GBAC starts a Linux Virtual Machine (using VirtualBox). 4.GBAC copies the applications and inputs into the Virtual Machine. 5.The application is executed in the Linux VM. 6.The result is fetched from the VM by GBAC. 7.The VM is shut down and discarded. 8.GBAC finishes and the result is returned to EDGIDemo from the Client. 9.The results are returned to gLite through 3G Bridge from EDGIDemo. 10. User can retrieve the results by using gLite commands

Step 1: Get the gridftp URL of GBAC To submit a virtualised app choose the GBAC app from the EDGI AR To submit a virtualised app choose the GBAC app from the EDGI AR Then query the Grid FTP url of the file “gbac.nobin” under the gLite implementation (as you did it during the single submission) Then query the Grid FTP url of the file “gbac.nobin” under the gLite implementation (as you did it during the single submission) Gridftp url: Gridftp url:gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1355/1512/gbac.nobin 24

Step 2: Get the VO and site list and select 25 cr1.edgi-grid.eu:8443/cream-pbs-edgidemo

Step 3: Create “helloworld” application Notes: package must have the name X.app.tgz, where X is the name of the executable which must be first launched after unpackaging. Notes: package must have the name X.app.tgz, where X is the name of the executable which must be first launched after unpackaging. 26

Step 4: create jdl Create your jdl: Create your jdl: Executable = “gbac.nobin"; Arguments = “anything"; InputSandbox = { "gsiftp://edgi-repo.cpc.wmin.ac.uk:2811/srv/edgi/1355/1512/gbac.nobin", "myhelloworld.sh.app.tgz" }; OutputSandbox = {"output"}; SubmitTo = "cr1.edgi-grid.eu:8443/cream-pbs-edgidemo"; 27 GBAC reference in the EDGI AR Your hello world applicationName of GBAC app framework

Step 5: Submit jdl Submitting the job: Submitting the job: glite-wms-job-submit –a –o id helloworld.jdl 28

Step 6a: Check the status of the job under BOINC 29

Step 6b: Waiting for the job to finish Get the status of the job: Get the status of the job: glite-wms-job-status -i id 30

Step 7: get the result Get the result: Get the result: glite-wms-job-output -i id --dir outputs 31

Controlled hands-on has finished at this point, please play with the tools you learnt or try any of the following tasks: 1.Try another application in the EDGI AR 2.Assume dsp is an MC type application, submit it 20 times with the same input (“queue 20”) 3.Take a real application and submit it using GBAC 4.For advanced users: combine MetaJob and GBAC: submit the helloworld application 10 times a)With different inputs b)With the same input 32

Thank you for your attention! 33