F.Pacini - Milan - 7 maggio, n° 1 UI Interactions and Interfaces with the Workload Manager Components DataGrid WP1 F. Pacini

F.Pacini - Milan - 7 maggio, n° 2 Summary u Interactions between UI and RB u Interactions between UI and M&C u Other Points

F.Pacini - Milan - 7 maggio, n° 3 Interactions between UI and RB (1/10) u The Job Submission UI contacts the RB when the following commands are issued by the user: s dg-job-submit s dg-list-job-match s dg-job-cancel u Communication is performed via socket (TCP/IP) u An agreement on the Communication protocol is needed

F.Pacini - Milan - 7 maggio, n° 4 Interactions between UI and RB (2/10)  dg-job-submit dg-job-submit [-resource res_id] [-notify e_mail_address] n the information flow from the UI to the RB consists of a job class-ad built from the job description file plus a variable indicating the request type. The job class-ad object consists in a list of entries “attribute = expression”. The following attributes are always present in the job class-ad (jobAd): s UserID s CertificateSubject s ExecutableName s Input s Output s Constraints

F.Pacini - Milan - 7 maggio, n° 5 Interactions between UI and RB (3/10) u If the dg-job-submit has been issued with the “-resource” option, then the job-ad contains the attribute: n ResourceID = res_id and the RB shall submit the job to the resource identified by “res_id” without going through the match-making process. u If the dg-job-submit has been issued with the “-notify” option, then the job class-ad contains the attribute: n UserContact = e_mail_address and the RB shall send an notification to e_mail_address at each job status transition. Content is the same as output from dg- job-status command (TBD).

F.Pacini - Milan - 7 maggio, n° 6 Interactions between UI and RB (4/10) u The variable indicating the request type, say requestType, is an enumerative with the following values: s JOB_SUBMIT s LIST_MATCH and is passed in this case with the JOB_SUBMIT value. u Summarising the UI passes to the RB a structure with the two following fields: n requestType n jobAd

F.Pacini - Milan - 7 maggio, n° 7 Interactions between UI and RB (5/10) u The expected response from the RB consists of: n ReturnCode, a numeric code indicating the operation result n ReturnMessage, a string describing the operation result n dg_jobId, a string identifying the submitted job UI to RBRB to UI - requestType (JOB_SUBMIT) - jobAd - ReturnCode - ReturnMessage - dg_jobId

F.Pacini - Milan - 7 maggio, n° 8 Interactions between UI and RB (6/10)  dg-list-job-match dg-list-job-match n the information flow from the UI to the RB consists of a job class-ad built from the job description file plus a variable indicating the request type. The job class-ad object consists in a list of entries “attribute = expression”. The following attributes are always present in the job class-ad (jobAd): s UserID s CertificateSubject s ExecutableName s Input s Output s Constraints

F.Pacini - Milan - 7 maggio, n° 9 Interactions between UI and RB (7/10) u The variable indicating the request type, requestType, is is passed in this case with the LIST_MATCH value. u The jobAd associated with this requestType will never contain neither the ResourceID nor the UserContact attributes. u In this case the RB does not submit the job but only searches for resources compatible with the input jobAd.

F.Pacini - Milan - 7 maggio, n° 10 Interactions between UI and RB (8/10) u The expected response from the RB consists of: n ReturnCode, a numeric code indicating the operation result (0 stays for success) n ReturnMessage, a string describing the operation result n ResourceList, a list of ResourceIds, i.e. strings identifying the resources matching with the jobAd. UI to RBRB to UI - requestType (LIST_MATCH) - jobAd - ReturnCode - ReturnMessage - ResourceList

F.Pacini - Milan - 7 maggio, n° 11 Interactions between UI and RB (9/10)  dg-job-cancel dg-job-cancel n the information flow from the UI to the RB consists of s userID, a string representing the user identifier s certSubject, a string containing the user certificate subject s dg_jobIdList, a list of string representing the identifiers of jobs to be canceled as specified by the user n if the dg-job-cancel command has been issued with the “-all” input parameter, then the dg_jobIdList passed to the RB will be empty indicating that all jobs submitted by the user identified by userID have to be canceled.

F.Pacini - Milan - 7 maggio, n° 12 Interactions between UI and RB (10/10) u The expected response from the RB consists of: n ReturnCode, a numeric code indicating the operation result n ReturnMessage, a string describing the operation result n dg_jobIdList, a list of dg_jobIds, i.e. strings identifying the jobs effectively deleted UI to RBRB to UI - userID - certSubject - dg_jobIdList (can be empty) - ReturnCode - ReturnMessage - dg_jobIdList

F.Pacini - Milan - 7 maggio, n° 13 Interactions between UI and M&C (1/11) u The Job Submission UI contacts the M&C when the following commands are issued by the user: s dg-job-status s dg-get-logging-info u to serve this request the UI uses the bookkeeping and logging APIs made available by the M&C component (see the Cesnet L&B Service Document) u APIs implementation shall encompass network communication

F.Pacini - Milan - 7 maggio, n° 14 Interactions between UI and M&C (2/11)  dg-job-status dg-job-status | -all > [full] n The UI will use the provided L&B server API with the following input information: s UserId, a string representing the user identifier s dg_jobIdList, a list of dg_jobIds s InformationLevel, indicating the required information level (SHORT/FULL) according to the command option “-full” n If the dg-job-status command has been issued with the “-all” input parameter, then the dg_jobIdList will be empty indicating that status information about all jobs submitted by the user identified by userID are requested.

F.Pacini - Milan - 7 maggio, n° 15 Interactions between UI and M&C (3/11) u Returned information should encompass: n ReturnCode, a numeric code indicating the operation result n ReturnMessage, a string describing the operation result n JobsStatusInfo, consisting of (TBD): s userID s dg_jobID s jobStatus s ResourceID s Executable s input s output s submissionTime (when the job has been submitted from the UI) s scheduledTime (when the job has been submitted to the resource) s startRunningTime (when the job has started its execution) s StopRunningTime (when the job has completed its execution) (if the InformationLevel is SHORT).

F.Pacini - Milan - 7 maggio, n° 16 Interactions between UI and M&C (4/11) n JobsStatusInfo, consisting of (TBD): s ResourceManagementType s ResourceManagementVersion s Gramversion s Architecture s OpSys s traversalTime s TotalCpus s FreeCpus s RunningJobs s IdleJobs s MaxTotalJobs s MaxRunningJobs s Status s userID s dg_jobID s jobStatus s ResourceID s ResourceName s Executable s input s output s submissionTime s scheduledTime s startRunningTime s StopRunningTime s CpuTime s Rank s Constraint (if the InformationLevel is FULL)

F.Pacini - Milan - 7 maggio, n° 17 Interactions between UI and M&C (5/11)  Summarising with the dg-job-status command we have: UI to M&CM&C to UI - userID - dg_jobIdList (can be empty) - InformationLevel - ReturnCode - ReturnMessage - JobsStatusInfo

F.Pacini - Milan - 7 maggio, n° 18 Interactions between UI and M&C (6/11) u dg-get-logging-info dg-get-logging-info | -all > [-from T1] [-to T2] [-full] n The UI will use the provided L&B server API with the following input information: s UserId, a string representing the user identifier s dg_jobIdList, a list of dg_jobIds s fromTime, timestamp s toTime, timestamp s InformationLevel, indicating the required information level (SHORT/FULL) according to the command option “-full” n If the dg-get-logging-info command has been issued with the “-all” input parameter, then the dg_jobIdList will be empty.

F.Pacini - Milan - 7 maggio, n° 19 Interactions between UI and M&C (7/11) u Returned information should encompass: n ReturnCode, a numeric code indicating the operation result n ReturnMessage, a string describing the operation result n JobsLogInfo according to the requested InformationLevel.  Summarising with the dg-get-logging-info command we have: UI to M&CM&C to UI - userID - dg_jobIdList (can be empty) - fromTime - toTime - InformationLevel - ReturnCode - ReturnMessage - JobLogInfo

F.Pacini - Milan - 7 maggio, n° 20 Interactions between UI and M&C (8/11) n userID n jobID n jobStatus n submissionTime n scheduledTime n startTime n finishTime n executable n executableSize n input data LFN n output data LFN n pendingReasons n Constraint n Rank n NumCpus n CpuTime n swapSpace n totalI/O n totalDataSpace n WallClockTime u JobLogInfo has to be defined:

F.Pacini - Milan - 7 maggio, n° 21 Interactions between UI and M&C (9/11) u RunWindows u ResourcePriority u MaxCpuTime u MaxWallTime u networkReq u fromTime u toTime u RunWindows u queuePriority u MaxCpuTime u MaxWallTime u ResourceID u ResourceName u ResourceManagementType u ResourceManagementVersion u Gramversion u executingHost u Architecture u OpSys u traversalTime u TotalCpus u MaxTotalJobs u MaxRunningJobs u ResourceStatus

F.Pacini - Milan - 7 maggio, n° 22 Interactions between UI and M&C (10/11) u Some points from the L&B service document (pages 2,3,5): n SUBMITTED status: does it mean that the job is still in the UI? n The UI does not know the dg_jobId before the job is in the RB. How can use the logging service? n CHKPT status: is checkpoint supported for PM9? n CLEARED status: who triggers transition 8? Who is going to log the JobClearedEvent? n GridScheduler, Condor-G and Globus job-manager are expected to log: the UI too? n Modification to Globus job-manager for logging: which effort? Who does it? n JobDoneEvent: does Globus provides the job exit status?

F.Pacini - Milan - 7 maggio, n° 23 Interactions between UI and M&C (11/11) u Some points from the L&B service document (pages 10,11,13): n L&B API seems not to provide a way to select info by time s Is it foreseen for PM9? s Shall the UI filter by time locally? n jobLog(): “level” vs “InformationLevel” n jobStatus(): “appropriate details” are already defined? n Encapsulation of network communication in the L&B server API is needed JobSubmitEvent: source of this log event is the GUI. Again jobID is needed but the UI does not know it.

F.Pacini - Milan - 7 maggio, n° 24 Other Points (1/2) u The information flow between the RB and the SS (the Condor-G wrapper) consists of class-ad objects n The RB maintain a persistent queue of jobs submitted by the users through the UI. Jobs are described by jobAd objects and identified by the dg_jobIds. n Once a suitable resource for a job has been found by the RB through the match making process, the jobAd is enriched with the found ResourceId and passed to the SS for submission u Which is the information returned to teh RB? n a job handle (the condor_jobId) and the jobStatus (condor_rm command)?

F.Pacini - Milan - 7 maggio, n° 25 Other Points (2/2) u How to detect job status transitions? Who does it? n Inspection of the Condor-G log files n globus_gram_client_job_status function n Other ways?