High-Performance Computing Lab Overview: Job Submission in EDG & Globus November 2002 Wei Xing

High-Performance Computing Lab Outline 1. Job Submission User Interface Architecture 2. Job Status 3. Job Submission Procedure 4. Interaction with RB, CE, LB 5. Job Submission in Globus 6. Pro-prototype of GB Portal Design

High-Performance Computing Lab EDG: Job Submission User Interface Architecture (1) ●Motivation Providing basic access functionality to WMS (Workload Management System) services. ● 4 interacting components of WMS – the User Interface (UI), – the Resource Broker (RB), – the Job Submission Service (JSS), – and the Logging and Booking (LB) service.

High-Performance Computing Lab EDG: Job Submission User Interface Architecture (2) UI RB L&B JSS Globus GRAM Here we are list dg-jobIdJSS-jobId local Q

High-Performance Computing Lab Job submission: Job Status ● Job Status : During its lifetime, a job may go through following states, and triggered by 10 events. User SubmittedWaitingReadyScheduled ClearedChkpt Done AbortedRunning

High-Performance Computing Lab A Job Submission Example UI JDL Resource Broker Job Submission Service Storage Element ComputeElement Information Service Job Status Replica Catalogue Job Submit Event Input Sandbox submitted waitingReady Brokerinfo Scheduled Input Sandbox running Output Sandbox done Job Status Output Sandbox cleared

High-Performance Computing Lab Job Submission Procedure (UI) ● User client – Describe their jobs. – Submit their jobs. – Control the submitted jobs. – Monitor the status of the submitted jobs. – Get the output results.

High-Performance Computing Lab Job Submission Procedure (UI) ● Job description with JDL. The general JDL format: attribute = value; values can be of different types: numeric, string, booleans, timestamps etc. ➢ Some attributes are used to describe the tecnical characteristics of the job. Executable = ''sim.exe''; StdInput = ''dataset.in'' ➢ Others are used to specify requirements for a CE. Requirements = other.OpSys == ''RH 6.2'' && other Arch == ''INTEL'';

High-Performance Computing Lab Job Submission Procedure (UI) ● Example of Hello World JDL file : Executable = "/bin/echo"; Arguments = "Hello World !"; StdOutput = "Messagge.txt"; StdError = "stderr.log"; OutputSandbox = "Message.txt"; ● User submission : dg-job-submit HelloWorld.jdl

High-Performance Computing Lab Job Submission Procedure (UI) ● Job identifier (grid-wide unique identifier) among UI, RB, and LB. / / / / / /**LB server name & port*/ firefox.esrlin.esa.it/ /** UI machine name */ / /** UI machine time; UI process ID; 007 random number */ datagrid.esrin.esa.it /** RB hostname & port */

High-Performance Computing Lab Job Submission Procedure (RB) ● Main task: Find the best suitable CE. – Direct job submission; ● UI contains a CEId ● RB checks JDL syntax ● Pass to JSS – Job submission without data-access requirements; ● Checks JDL syntax ● RB starts the actual matchmaking algorithm to find the suitable Grid resources matches the job requirements. ● Pass to JSS – Job submission with data-access requirements. ● Checks JDL syntax ● Interacts with RM services to find suitable CE & SE. ● Pass to JSS

High-Performance Computing Lab Job Submission Procedure (JSS) ● Job Submission Service (JSS) is responsible for the actual job management operations, in particular to manage the job submission and job removal requests, interacting with CE. – Job sumission: ● receives JDL sent to RB by UI. ● builds a ''wrapper'' of the user job and assign a jss- jobId to this wrapper job, insert it in a local quene. ● Performs the actual job submission to the chosen remote CE.

High-Performance Computing Lab Job Submission Procedure (JSS) ● Job Removal – the RB requires the removal of a job; – the JSS receives a dg-jobId, finds the correspondent jss-jobId; – inform the remote CE; – when it has been removed from CE, it is then removed from the local quene. At last step, RB is notified.

High-Performance Computing Lab Interaction with RB, CE, LB ● JSS (server) listens the requests from RB (client) continously. ● JSS interact with CE to request submission/ removal of jobs, query the status of submitted jobs, be informed on the completion of jobs. ● JSS pushes some events to LB. – JobAcceptedEvent – JobRefusedEvent –......

High-Performance Computing Lab Job Submission in Globus: GRAM & Globus Grid Security Infrastructure Job Manager MDS client API calls to locate resources Query current status of resource Create RSL Library Parse Request Allocate & create processes Process Monitor & control Client MDS: Grid Index Info Server Gatekeeper MDS: Grid Resource Info Server Local Resource Manager MDS client API calls to get resource info GRAM client API state change callbacks

High-Performance Computing Lab Pro-prototype of GB Portal Design

High-Performance Computing Lab Use Case Diagram

High-Performance Computing Lab Class Definition & UML Diagram ● Waiting …. ---:)

High-Performance Computing Lab Thanks for Ur Patience

High-Performance Computing Lab ● GASS/RSL Example ● &(executable= (stdin= (stdout=/home/bester/output) (stderr=