EGEE-II INFSO-RI Enabling Grids for E-sciencE Status of INFN middleware in gLite Claudio Grandi INFNGrid EB CNAF, 29 August 2006

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August Main focus for JRA1 developers –Support on the production infrastructure (GGUS, 2 nd line support) –Bug-fixing –Task Forces together with applications and site experts  The above items are supposed to take ~ 50% of the developers time! Furthermore: –Improve robustness and usability (efficiency, error reporting,...) –Support for SL(C)4 and for x86-64 and IA64 –Addressing requests for functionality improvements from users, site administrators, etc... (through the TCG) –Preview testbed –IPv6 compliance  Not much time left for anything else...

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August Main achievements Delivery of gLite 3.0 –Convergence of LCG and gLite 1.5 –New WMS/LB/UI/CE –Tuning and optimization –“CMS-WMS” exercise  Instance of the WMS attached to the PS where patches flow directly from the developers but in a “controlled” way (by SA3 and SA1)  To speed up bug-fixing and certification  Migration to the new version of Condor (6.7.19)  Tests by IT-PSS (former LCG-EIS) team (ATLAS & CMS) Porting to VDT (including GT4 pre-WS) –84% of the code builds –Mandatory step to support Scientific Linux 4 and 64-bit Preview testbed –ICE-CREAM and Job Provenance deployed –Other items will follow (G-PBox, glexec).

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August Preview test-bed The SA3 integration and certification teams are focused on providing code for the production infrastructure –Strong control over what is accepted, but slow process for the certification of the new components and of the improvements JRA1 requested a test-bed to expose to users those components not yet considered for certification –To get feedback from users and site managers –TCG and PEB acknowledged that this is needed, but no resources were foreseen for this activity in the EGEE-II proposal The JRA1 partners which have also strong commitments in SA1 have been requested to provide resources (machines and manpower) for this activity without compromising their commitment in SA1  At present, only INFN and CESNET have committed resources We need more sites!!!

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August VOMS/VOMSAdmin Now everyting under CNAF responsibility New Java API library being rewritten “pure java” –existing version from HIP was incomplete –version by CNAF using C++ libraries via JNI not acceptable by java developers –will be ready in October We may have a “political” problem with the interface of VOMS-Admin –The TCG may ask to stop the development of a VOMSAdmin interface in favour of VOMRS (used by the VOs) –Note that the VOMS-Admin service is not in discussion  it is also used by VOMRS! –The new version of the VOMSAdmin interface is foreseen to be ready in a couple of months and should have a better management of persistency, an improved usability and maintainability by the site managers and a better framework for implementing the registration procedures

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August WMS Thorough testing of the WMS in the CMS-WMS exercise –Followed closely by the EMT. Developers have been on holiday for 2 weeks  process slowed down  Many thanks to those who worked also during their holidays!!! –Most problems understood, some still under investigation. Fixes should be available soon. Build of the 3.1 branch –Would eventually merge the production and development branches into the same code-base. –Initially supposed to work with VDT had to be switched back to VDT because of missing dependencies (in the DM domain) –Also with VDT there are problems  Now builds only at CNAF, not at CERN, requires a recipe from Eli  When installed it crashes (report from Matteo, Roberto, Daniele) –Show-stopper for the preview testbed and also for the other tests foreseen in the CMS-WMS exercise

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August WMS, CE gLite Main enhancements in WMS –Added support for VOViews –Bulk match-making almost there –Support of bulk jobs without DAGMan  What about complex workflows? An instance of the gLite CE with Condor installed. Will be tested in the CMS-WMS exercise –Univ. of Wisconsin now giving direct support but Francesco is still the main reference person, also because of BLAH –Support of LRMSs is an issue (also for CREAM). Should improve the documentation. Working for supporting Condor pools Main enhancements in gLiteCE/BLAH –New log file for DGAS/APEL is there –Most of the enhancements in the future will deal with the use of information pass-through in BLAH Need to improve the developers testbed for WMS & CE!!!

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August CREAM, CEMon CREAM being installed on the preview testbed –Activity also affected by the holidays –Blocked by the WMS problem –Test CREAM alone and together with the gLiteCE: Site with two interfaces  may help in the acceptance from the political point of view CEMon is on the production infrastructure but not used –BDII solution still preferred –Will be mandatory for CREAM –Important also for VOs like LHCb that want to access the CE directly and exploit some form of pull model –The possibility to use an authorization layer in front of it makes it interesting in the long term for dealing with privacy issues BLAH CE ICE Submitter Job Status Handler CREAMCEMonCondor-C

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August DGAS, GPBOX DGAS –Installed at INFN, will be on the preview –Agreements for its certification at the LCG-MB –Waiting for the OK by the developers for asking SA3 to start GPBOX –Developing the components that run at the site –Plug-ins for LCAS/LCMAPS ready –Will be tested on the preview after CREAM, but would need more sites to make the test meaningful –MANY political problems here. The common convincement is that there are much easier ways to achieve the same result (e.g. with capabilities attached to the certificate by VOMS).

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August Strategies for the future 1/2 1.Make clients happy Porting to SL4, 64-bits, includes VDT Improve error reporting and logging 2.Consolidate the infrastructure middleware Improve the support for local batch systems produce plug-ins and improve documentation on how to write them exploit the information pass-through  collaboration with sites!!! Deliver CREAM make it work with the existing Condor-C based CE fully support CREAM on the WMS, i.e. bulk jobs managed independently of DAGMan improve collaboration in the OGF-BES group to define the interface! Fully exploit CEMon requires collaboration with the UK people to understand how to replace BDII with CEMon + Service Dsicovery with R-GMA backend Certify DGAS and make it work better than APEL verify performance of sensors on big sites

Enabling Grids for E-sciencE EGEE-II INFSO-RI Claudio Grandi - INFNGrid EB - CNAF- 29 August Strategies for the future 2/2 3.Security Consolidate the use of VOMS improve the take from the OGF extend functionalities and usability Improve VOMS-Admin improve the interface and provide a customization for HEP 4.Job support Improve WMS performances switch asap to the 3.1 branch bulk match-making, develop high availability systems in future: complex workflow support independent of DAGMan Job priorities at the site level, exploiting the LRMS via policy management systems (GPBox) do we need a revision of the architecture? can we make it simpler? we have learned many things... there’s nothing wrong re-discussing with others about how to do it in the best way!