The gLite API – PART I Giuseppe LA ROCCA INFN Catania ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia

W M P r o x y J a v a A P I ver Software Requirements Java Class org.glite.wms.wmproxy.WMProxyAPI Support to cog-jglobus-1.4 Hands-on

Installation guide API JAVA /1 0. Start from a fresh install of Scientific Linux CERN –other platforms are also compatible but yet not properly tested 1.Axis ( where the needed jars available are: –axis jar –axis jar –axis-jaxrpc jar –axis-wsdl4j jar 2. Geronimo J2EE libraries: –geronimo-activation_1.0.2_spec jar –geronimo-j2ee_1.4_spec jar

3. Crypto libraries: –cryptix-asn1.jar –cryptix32.jar –bcprov.jar –puretls.jar 4. gLite libraries: –glite-jdl-api-java.jar –glite-wms-wmproxy-api-java.jar –glite-security-delegation-java.jar –glite-security-trustmanager.jar –glite-security-util-java.ja 5. GLOBUS and other libraries: –cog-jglobus.jar –classad.jar –commons-discovery jar –commons-logging-api-1.1.jar –log4j jar W A R N I N G Make sure all the listed jar(s) are included in your CLASSPATH environment variable !! Installation guide API JAVA /2

Warning!! All the needed gLite packages can be found here All the needed external packages can be found here Installation guide API JAVA /3

The JDL to be matched must contains some mandatory Requirements and Rank attributes; –when using command line interface these are added by default by the UI, but in our case user has to take care to check if his/her JDL file contains them: Requirements = (other.GlueCEStateStatus == "Production"); Rank = (-other.GlueCEStateEstimatedResponseTime); Basic Requirement

The main Java class of this API is: org.glite.wms.wmproxy.WMProxyAPI; A client object can be created using one of these following constructor(s): public WMProxyAPI(String url, String proxyFile) public WMProxyAPI(String url, String proxyFile, String CAcerts) public WMProxyAPI(String url, InputStream proxyStream) public WMProxyAPI(String url, InputStream proxyStream, String CAcerts) org.glite.wms.wmproxy.WMProxyAPI

where: –url: the WMProxy server URL to be contacted (e.g. : /glite_wms_wmproxy_server); –proxyFile: the pathname to a valid user proxy; for the default value (/tmp/x509up_u ) –CAcerts: the path of a local CA directory –proxyStream: a valid proxy passed as input stream;

Therefore, using one of the four constructors WMProxyAPI Client = new WMProxyAPI( … ); a new instance is created. This instance will be created to invoke any other WMProxy services. For example: WMProxyAPI Client = new WMProxyAPI( 01.ct.infn.it:7443/glite_wms_wmproxy_server, “/tmp/x509up_u504”);

In case of failure, one of the following exceptions is thrown: org.glite.wms.wmproxy.AuthenticationFaultType org.glite.wms.wmproxy.JobUnknownFaultType org.glite.wms.wmproxy.AuthorizationFaultType org.glite.wms.wmproxy.NoSuitableResourcesFaultType org.glite.wms.wmproxy.GetQuotaManagementFaultType org.glite.wms.wmproxy.ProxyFileException org.glite.wms.wmproxy.GrstDelegationException org.glite.wms.wmproxy.GenericFaultType org.gridsite.

Before calling submission or list match services, client proxy credentials are requested to be transferred from the client to the server through a delegation process. The following Java code performs this delegation process: The obtained delegation identifier can be reused for several calls to the WMProxy server; Delegation of user credential

The following Java code submits a job and returns a JobID Submit a job to WMProxy

Submission of simple jobs, DAGs, collections and parametric jobs to the WMProxy service requires as input a job description file in which job characteristics and requirements are expressed by means of Condor class-ad-like expressions. In this description the users can specify some files that are needed by the submitted jobs during the execution on the remote Computing Elements. These files are listed in the JDL InputSandbox attribute specifying for each of them the complete URI of its location. If the job does NOT have any file in the InputSandbox to be transferred from the submitting machine to the WMProxy node, the submission can be performed also calling the jobSubmit service: jobId = client.jobSubmit(jdlString, delegationID); Difference between submission and registration /1

Otherwise, these following steps are needed: a preliminary server registration: jobId = client.jobRegister(jdlString, delegationID); transfer of files in the InputSandbox from the client machine to the WMS node; call the jobStart service to trigger the submission: client.jobStart(jobId); Difference between submission and registration /2

In order to allow grid user to invoke, within Java code, the UrlCopy class to copy files from the WMS Server to the UI and vice versa the following Java package is requested cog-jglobus-1.4-bin.tar.gz The above package, with all the needed documentation, can be downloaded from this web page Software Requirements

Executable = "/bin/sh"; Arguments = "start_hostname.sh"; StdOutput = "hostname.out"; StdError = "hostname.err"; InputSandbox = {"start_hostname.sh"}; OutputSandbox = {"hostname.err","hostname.out"}; Requirements = (other.GlueCEStateStatus == “Production”); Rank = (-other.GlueCEStateEstimatedResponseTime); RetryCount = 3; #!/bin/sh sleep 5 hostname -f hostname.jdl start_hostname.sh

$ java WMProxyGetProxyAndSubmit \ \ hostname.jdl start_hostname.sh | grep -v "ERROR" Connecting to the service ====================== glite-wms-job-submission Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: =============================================================================== Submit a new request to WMProxy /1 Usage : java WMProxyGetProxyAndSubmit -h[elp] java WMProxyGetProxyAndSubmit [ ] where:... the entry point of the WMProxy Server to contact (e.g. : (e.g. : the jdl file to submit to the grid... The list of file(s) to transfer

The script, thanks to the UrlCopy Class, performs the copy of the InputSandbox files to the reduced path of the WMS as you can see: cd /var/glite/SandboxDir/P-/ ll https_3a_2f_2fgilda-wms-01.ct.infn.it_3a9000_2fP-H1wp0RpTnos2qGyQ2ETQ/input/ -rwxrwxr-x 1 gilda001 glite 30 Jan 11 09:05 start_hostname.sh Submit a new request to WMProxy /2

When the job finishs you can retrieve the output file(s) as follow: $ java WMProxyGetOutputAndPurge \ \ \ \ $PWD/ | grep -v "ERROR" Connecting to the service ====================== glite-wms-job-output Success ====================== JOB GET OUTPUT OUTCOME Output sandbox files for the job: have been successfully retrieved and stored in the directory: /home/larocca/Api-Java-WMProxy/ =========================================================================== Retrieve output file(s)

The following Java code cancels a list of jobIDs Cancelling a job

A job can be canceled before it ends using the following Java class: $ java WMProxyCancelJobs \ \ \ | grep -v "ERROR" Connecting to the service ======================== glite-wms-job-cancel Success ============================= The cancellation request has been successfully submitted for the following job(s): - ==================================================================================== Cancelling a job

The following Java code returns a list of CEs available for job submission Listing CE ID(s) that match a job

It is possible to see which CEs are eligible to run a job described by a given JDL using the following Java API: $ java WMProxyGetProxyAndListMatch \ \ hostname.jdl start_hostname.sh | grep -v "ERROR" Connecting to the service COMPUTING ELEMENT IDs LIST ========================================================================== The following CE(s) matching your job requirements have been found: - gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - gn0.hpcc.sztaki.hu:2119/jobmanager-lcgpbs-gilda - grid010.ct.infn.it:2119/jobmanager-lcgpbs-infinite - grid010.ct.infn.it:2119/jobmanager-lcgpbs-long - grid010.ct.infn.it:2119/jobmanager-lcgpbs-short - grid011f.cnaf.infn.it:2119/jobmanager-lcgpbs-gilda - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-infinite - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-long - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-short - ce.hpc.iit.bme.hu:2119/jobmanager-lcgpbs-gilda ========================================================================== Listing CE ID(s) that match a job

Java API Documentation

References Datamat – WMProxy quickstart ProxyClient.QuickStart JDL Attributes guide for WMProxy WMProxy user guide GILDA wiki

Hands-on Connect to the training infrastructure using the information reported in the tutorial sheet Run the hands-on available in this web link: Enjoy! $ cd ${HOME}/Api-Java-WMProxy To compile Java code run: $ source wmproxy.sh Submit and collect results with the two Java classes: WMProxyGetProxyAndSubmit WMProxyGetOutputAndPurge