EGEE-II INFSO-RI Enabling Grids for E-sciencE International Summer School on Grid Computing 2006 gLite Information System and Workload Management System Diego Scardaci INFN Catania International Summer School on Grid Computing Ischia, 9-21 July, 2006

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Outline Information System Architecture −Berkeley DB Information Index (BDII) −The Relational Grid Monitoring Architecture (RGMA) Workload Management System −WMS Architecture −Job Description Language Overview −WMProxy Overview −Special Jobs: DAG, Collections, Parametric and MPI

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Information System

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Information System What is? –System to collect information on the state of resources Why? –To discover resources of the grid and their nature –To have useful data in order to who is in charge of managing the workload to do it more efficiently. –To check for health status of resources. How? –Monitoring state of resources locally and publishing fresh data on the information system. –Adopting a data model that MUST be well known to all components that want to access monitored information –Using different approaches that we are going to investigate in the next slides

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Adopted Information Systems The BDII (Berkeley DB Information Index) –has been adopted in LCG middleware as the Information System provider. –It is an evolution of the Globus Meta Directory System (MDS) –It is based on Lightweight Directory Access Protocol (LDAP) servers. The Relational Grid Monitoring Architecture (R-GMA) –It is an implementation of the Grid Monitoring Architecture (GMA) standardized by the Global Grid Forum (GGF, now OGF) –It is a relational implementation of the GMA –It is strongly Web Services Oriented –It uses standard SQL query syntax

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing GRISs, local BDII and BDII Each site can run a BDII. It collects the information given by the local BDIIs At each site, a *local* BDII collects the information given by the GRISs Local GRISes run on CEs and SEs at each site and report dynamic and static information Abbreviations: BDII: Berkeley DataBase Information Index GIIS: Grid Index Information Server GRIS: Grid Resource Information Server

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing The IS in gLite RB Local GRIS SE Local GRIS CE Local GRIS BDII-A BDII-B SE Local GRIS SE Local GRIS CE Local GRIS SE Local GRIS BDII-C CE Local GRIS CE Site BDII CE Local GRIS CE Site BDII CE Local GRIS CE Site BDII Site 1 Site 2 Site 3

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing BDII Users and other Grid services (such as the RB) can interrogate BDIIs to get information about the Grid status. Each BDII collects information from the site GIISes (or local BDII) defined in a configuration file, which it accesses through a web interfaces. Every two minutes a cron-job runs a script and collects information (pull model) from all the GIIS (local BDII) listed in the configuration file

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing R-GMA The Relational Grid Monitoring Architecture (R-GMA) –It is the relational implementation of GMA defined by the GGF –Adopts a database model with tables and relations between tables –Implements a virtual database –The user queries the R-GMA as he/she was querying to a classical database (SQL string) –Implements different type of queries The information –The Producer stores its location (URL) in the Registry. –The Consumer looks up producer URLs in the Registry. –The Consumer contacts the Producer to get all the data or to listen for new data. PRODUCER CONSUMER REGISTRY Store location Lookup location Transfer Data

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Workload Management System

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Overview of WMS Architecture Job Description Language Overview WMProxy Overview Special Jobs –DAG jobs –Job collections –Parametric jobs –MPI jobs Outline

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing The Workload Management SystemThe Workload Management System (WMS) comprises a set of Grid middleware components responsible for distribution and management of tasks across Grid resources. The purpose of the Workload Manager (WM) is accept and satisfy requests for job management coming from its clients –meaning of the submission request is to pass the responsibility of the job to the WM.  WM will pass the job to an appropriate CE for execution taking into account requirements and the preferences expressed in the job description file matchmakingThe decision of which resource should be used is the outcome of a matchmaking process. WMS Objectives

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing WMS Architecture Job management requests (submission, cancellation) expressed via a Job Description Language (JDL) Finds an appropriate CE for each submission request, taking into account job requests and preferences, Grid status, utilization policies on resources Keeps submission requests Requests are kept for a while for a while if no resources are immediately available Repository of resource information information available to matchmaker Updated via notifications and/or active polling on resources Performs the actual job submission and monitoring

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Job Description Language

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Job Description Language Job Description Language (JDL)In gLite Job Description Language (JDL) is used to describe jobs for execution on Grid. CLASSified Advertisement language (ClassAd)The JDL adopted within the gLite middleware is based upon Condor’s CLASSified Advertisement language (ClassAd). A ClassAd is a record-like structure composed of a finite number of attributes separated by semi-colon (;) A ClassAd is highly flexible and can be used to represent arbitrary services match-making process The JDL is used in gLite to specify the job’s characteristics and constrains, which are used during the match-making process to select the best resources that satisfy job’s requirements.

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing JDL syntaxThe JDL syntax consists on statements like: Attribute = value; Comments must be preceded by a sharp character # ( # ) or have to follow the C++ syntax WARNING: The JDL is sensitive to blank characters and tabs. No blank characters or tabs should follow the semicolon at the end of a line. Job Description Language (cont.)

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing JDL: an example Type = "Job"; JobType = "Normal"; Executable = "startGen4.sh"; Environment = {"CLASSPATH=./gfal.jar:./gint.jar","LD_LIBRARY_PATH=.:$LD_LIB RARY_PATH","LCG_GFAL_VO=gilda","LCG_RFIO_TYPE=dpm"}; Arguments = " aliserv6.ct.infn.it lfn:/grid/gilda/valeria/2000pillar.dat /gilda/ischia06/vardizzo"; StdOutput = "sample.out"; StdError = "sample.err"; InputSandbox = {"startGen4.sh","gint.jar","gfal.jar","libGFalFile.so"}; OutputSandbox = {"sample.err","sample.out"}; Requirements = Member("GLITE- 3_0_0",other.GlueHostApplicationSoftwareRunTimeEnvironment);

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Workload Manager Proxy

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing WMProxy WMProxy (Workload Manager Proxy) –is a new service providing access to the gLite Workload Management System (WMS) functionality through a simple Web Services based interface. –has been designed to handle a large number of requests for job submission  gLite 1.5 => ~180 secs for 500 jobs  goal is to get in the short term to ~60 secs for 1000 jobs –it provides additional features such as bulk submission and the support for shared and compressed sandboxes for compound jobs. –It’s the natural replacement of the NS in the passage to the SOA approach.

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing New request types Support for new types strongly relies on newly developed JDL converters and on the DAG submission support –all JDL conversions are performed on the server –a single submission for several jobs All new request types can be monitored and controlled through a single handle (the request id) –each sub-jobs can be however followed-up and controlled independently through its own id “Smarter” WMS client commands/API –allow submission of DAGs, collections and parametric jobs exploiting the concept of “shared sandbox” –allow automatic generation and submission of collections and DAGs from sets of JDL files located in user specified directories on the UI

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Special Jobs

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Outline DAG Job Collection Parametric jobs MPI jobs on gLite

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing DAG job A DAG job is a set of jobs where input, output, or execution of one or more jobs can depend on other jobs Dependencies are represented through Directed Acyclic Graphs, where the nodes are jobs, and the edges identify the dependencies nodeA nodeBnodeC NodeF nodeD

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing JDL structure

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Attribute: Nodes

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Attribute: Dependencies

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing DAG jdl [ type = "dag"; max_nodes_running = 4; nodes = [ nodeA = [ file ="nodes/nodeA.jdl" ; ]; nodeB = [ file ="nodes/nodeB.jdl" ; ]; nodeC = [ file ="nodes/nodeC.jdl" ; ]; nodeD = [ file ="nodes/nodeD.jdl"; ]; dependencies = { {nodeA, nodeB}, {nodeA, nodeC}, { {nodeB,nodeC}, nodeD } } ]; ] Node description could also be done here, instead of using separate files

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Job Collection A job collection is a set of independent jobs that user wants to submit and monitor via a single request Jobs of a collection are submitted as DAG nodes without dependencies JDL is a list of classad, which describes the subjobs [ Type = "collection"; VirtualOrganisation = “gilda"; nodes = { [ ], … }; ]

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing ‘Scattered’ Input Sandboxes Input Sandbox can contain –file paths on the UI machine (i.e. the usual way) –URI pointing to files on a remote gridFTP/HTTPS server A base URI to be applied to all sandbox files can also be specified Only local files ( file:// ) are uploaded to the WMS node File pointed by URIs are directly downloaded on the WN by the JobWrapper just before the job is started InputSandbox = { "gsiftp://neo.datamat.it:2811/var/prg/sim.exe", " "file:///home/pacio/myconf“ }; InputSandboxBaseURI = "gsiftp://matrix.datamat.it:2811/var";

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing ‘Scattered’ Output Sandboxes JDL has been enriched with new attributes for specifying the destinations for the files listed in the OutputSandbox attribute list A base URI to be applied to all sandbox files can also be specified Files are copied when the job has completed execution by the JobWrapper to the specified destination without transiting on the WMS node OutputSandbox = {"jobOutput", "run1/event1", "jobError"}; OutputSandboxDestURI = { "gsiftp://matrix.datamat.it/var/jobOutput", " "gsiftp://matrix.datamat.it/var/jobError"}; OutputSandboxBaseDestURI = "gsiftp://neo.datamat.it/home/run1/";

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Job collection example [ type = "collection"; InputSandbox = {"date.sh"}; RetryCount = 0; nodes = { [ file ="jobs/job1.jdl" ; ], [ Executable = "/bin/sh"; Arguments = "date.sh"; Stdoutput = "date.out"; StdError = "date.err"; OutputSandbox ={"date.out", "date.err"}; ] ], [ file ="jobs/job3.jdl" ; ] }; ] All nodes will share this Input Sandbox

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Parametric Job A parametric job is a job where one or more of its attributes are parameterized Values of attributes vary according to a parameter Job monitoring / managing is always done through an unique jobID, as if the job was single (see submission of collection [ JobType = "Parametric"; Executable = "/bin/sh"; Arguments = "md5.sh input_PARAM_.txt"; InputSandbox = {"md5.sh", "input_PARAM_.txt"}; StdOutput = "out_PARAM_.txt"; StdError = "err_PARAM_.txt"; Parameters = 4; ParameterStart = 1; ParameterStep = 1; OutputSandbox = {"out_PARAM_.txt", "err_PARAM_.txt"}; ]

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing Parametric job / 2 Parameter can be also a list of string InputSandbox (if present) has to be coherent with parameters [ui-test] /home/giorgio/param > cat param2.jdl [ JobType = "Parametric"; Executable = “/bin/cat"; Arguments = “input_PARAM_.txt”; InputSandbox = "input_PARAM_.txt"; StdOutput = "myoutput_PARAM_.txt"; StdError = "myerror_PARAM_.txt"; Parameters = {earth,moon,mars}; OutputSandbox = {“myoutput_PARAM_.txt”}; ] [ui-test] /home/giorgio/param > ls inputEARTH.txt inputMARS.txt inputMOON.txt param2.jdl

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing MPI Overview Execution of parallel jobs is an essential issue for modern informatics and applications. Most used library for parallel jobs support is MPI (Message Passing Interface) At the state of the art, parallel jobs can run inside single Computing Elements (CE) only; –several projects are involved into studies concerning the possibility of executing parallel jobs on Worker Nodes (WNs) belonging to different CEs.

Enabling Grids for E-sciencE EGEE-II INFSO-RI International Summer School on Grid Computing References gLite 3.0 User Guide – R-GMA overview page – GLUE Schema – JDL attributes specification for WM proxy – WMProxy quickstart – wm/wmproxy_client_quickstart.shtmlhttp://egee-jra1-wm.mi.infn.it/egee-jra1- wm/wmproxy_client_quickstart.shtml WMS user guides –