INFSO-RI Enabling Grids for E-sciencE Padova site report Massimo Sgaravatto On behalf of the JRA1 IT-CZ Padova group

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 2 CEMon Code for Glite 1.2 has been finalized a while ago No major changes since that –Some minor fixes and improvements –Unit tests finalized –User guide completed No much feedback so far Ready to address CEMon issues when GLite 1.2 will be deployed and used Discussed and defined a plan to make GridIce information available via CEMon Some interest from VDT

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 3 CREAM Several changes/improvements, in particular: –Staging of files from client machine to CREAM via gridftp  For the time being gave up the staging via SwA - DIME –Gridftp server on the CE node  Contacted LCG CERN: they don’t see problem with that  They are already running a gridftp server for other reasons –Submit splitted in two methods (more or less as for WMProxy)  Register, which returns CREAM jobid + directory for ISB staging  Then staging of ISB files from client must be done  Start, which triggers the actual submission  Also a “special” Register with “automatic” Start, if no ISB files from client node have to be staged from the client machine –JobStatus  Job states redefined (see:  Redefined list of attributes to be returned  Possible to define a “filter” to have the status only for jobs being in one of a specified list of possible states

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 4 CREAM Journal manager – Journal file which persistently keeps track of the requests –To have a crash proof service Supported job types –“Normal” (sequential batch jobs) –Almost done also for Mpich jobs Interactive access to the WN –Some prototype code already done –Then how about forcing the execution of a restricted shell ?  Even if you are allowed just to run gdb, from gdb you can launch programs …  Even with the current interactive job support of WMS you can open an interactive shell on the WN and do whatever you want –Not integrated yet with CREAM

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 5 CREAM “Job status changes notifier” –We need some mechanisms to be told when a job changes its status, instead of doing job polling  Needed for CEMon notifications about jobs  Needed for integration with WMS  Needed also to improve performance of synchronous jobstatus In particular if the operation has to be done on multiple jobs –Some code for LSF that does that (parsing the LSF log files) already in place –Some overlaps with BLAH that should be discussed CREAM Clients –C++ CLI  This also needed changes, still to completed  Trying to stay synchronized with WMProxy CLI for CREAM CLI whenever possible  CREAM CLI is in CVS (HEAD) but not linked to the build system Circular dependency because cream-cli depends on org.glite.wms.jdl which depends on other modules of wms –Java client (examples)  Requested by GridCC (see below)

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 6 CREAM: integration with security sw Delegation –Blocking problems (It was possible to reuse a proxy delegated by another user, the delegated proxy was not good, etc) fixed  Often we also provided the fix –But there are still problems  Bug #9325: “Too many open files” when delegating many proxies These open files are the proxy files Credential mapping –Preliminary su-exec code implemented –Started analyzing it and its integration with CREAM Integration of G-PBOX –To be done –What are the plans to have this software in the EGEE CVS ?

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 7 CREAM: some rough numbers Configuration –Client in Padova & CREAM in LNL Submission of 50/200 jobs (without ISB staging) using the Java client took: –Delegating each time a proxy, invoking the “special” register+start method: 52.6/212 s. –Delegating each time a proxy, with two distinct register and start calls: 58.3/282 s. –Using a previously delegated proxy, invoking the “special” register+start method: 24.9/133 s. –Using a previously delegated proxy, with two distinct register and start calls: 33.0/161 s. Submission of 50/200 jobs (without ISB staging) using the C++ CLI took: –Delegating each time a proxy, with two distinct register and start calls: 67/240 s. –Using a previously delegated proxy, with two distinct register and start calls: 38.0/146 s. Submit returns when job has been inserted in the journal Disclaimers: –the two set of tests were performed in different configurations –Results not analyzed yet

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 8 CREAM: documentation Available: –Documented WSDL  Produced via wsdldoc –CREAM JDL attribute specifications –CREAM CLI user’s guide  Not completed yet –Installation guides CREAM CREAM CLI Quick doc for java client: will appear soon Just started a “high level” document describing CREAM architecture and functionality All available in

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 9 CREAM GRIDCC expressed its interest in CREAM –They are evaluating to integrate it in their web service oriented architecture –They are evaluating to use it in their demo (EU review?) in September Installed CREAM in a small LSF farm in Legnaro, which has been opened to some GRIDCC users Provided examples of Java clients, as they requested We are now interact directly with them –Elettra Trieste –Imperial College We are focused on this now

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 10 CREAM: some next steps Finalizing software under implementation Stress, performance, scalability, throughput tests –Trying to address the problems, of course Proxy renewal –As we discussed last time Support of other job types –Parametric jobs –Job collections Integration with WMS

Enabling Grids for E-sciencE INFSO-RI Massimo Sgaravatto - INFN Padova 11 Other Support for MPI jobs –Implemented (with Eli) the changes needed to support MPI jobs in CEs with a non shared file system –Deployed and tested in the CEs of the prototypes –Instructions to properly configure CE and WNs (e.g. ssh configuration) sent to ITeam JC/LM problems –A couple of bugs fixed –Still a couple to be addressed Prototype and testing activities  Maintenance of the WMS and CEs of the prototype  Support to testing team  Testing of candidate releases  Testing of new stuff