DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 1 DataGrid Workshop INFN Testbed status report L. Gaido Oxford July,

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 2 Dedicated resources (available now) Q Quantum Grid: 3-4 PCs in 15 sites: Bari, Bologna, Cagliari, Catania, Cnaf, Ferrara, Lecce, Legnaro, Milano, Napoli, Padova, Parma, Pisa, Roma, Torino Total: ~ 50 nodes Q 1 SUN cluster (5 nodes) at CNAF Q Some more hosts dedicated to specific activities

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 3 Non dedicated resources The LHC experiments have their own farms (~ 100 nodes in 8 sites), dedicated to production activities, in some INFN sites. Since the experiments play a fundamental role in Testbed and some of the test steps will require experiments’ production environment these resources will also be used for Testbed activities (but it is up to the experiments decide when, where and and how).

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 4 Further dedicated resources (available soon) In order to have enough resources available for the Testbed activities 6 new farms have been funded by InfnGrid project. So, in the next months, farms made up of - 1 disk server (~ 300 GB) - 5 dual PIII 1 GHz will be bought and set up in: Catania, Cnaf, Milano, Padova, Pisa and Torino

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 5 How have they been used? The currently available resources have been used for: -Globus Toolkit evaluation activities (a report has been produced) -InfnGrid toolkit production (Globus toolkit + GDMP + support for various CAs + configuration scripts, etc.) -Implementation of the GIS INFN infrastructure (an index server, GIIS, in each INFN unit) -MDS-2 tests (presentation by A. Cavalli in the MDS-2 session, Wednesday 9:00-11:00) -GridFTP tests - “toys” for the site managers

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 6 How have they been used? -LHC experiments’ tests: - ALICE Catania, Cagliari, Torino, Lyon ATLAS Milano, Roma, CERN - CMS Bologna, Padova, Pisa, Roma.... Testbed resources and experiments’ resources have been used for these tests.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 7 How have they been used? Presentations about these activities have been given at the InfnGrid workshop held in Catania (April, 9-11) and in the DataGrid WP8 meeting (May, 31)

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 8 How will they be used? The INFN Testbed resources will be primarily used for InfnGrid Testbed activities and tests. Some of the InfnGrid Testbed activities are also DataGrid activities, some are INFN specific. Some words about them in a few moments… Some guidelines (and tools) to request and allocate the Testbed resources for test purposes have been produced for INFN.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 9 How will they be used? The INFN Testbed resources can also be used for DataGrid Testbed activities. But we need to: - define clearly which resources each DataGrid partner reserves for DataGrid Testbed - define how to manage the allocation and usage of the testbed resources at the DataGrid level (Europe + some USA sites)

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 10 InfnGrid guidelines -The requester (Test Coordinator) submits a form describing the goals and all the relevant information (users involved, needed resources, grid services, etc.) about the test to a technical group, the Planning Group. -The PG evaluates the request and allocates the resources (via the resource database) to the test users, gets the appropriate grid-mapfile from the Certificates database manager and sends it to the site managers.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 11 InfnGrid guidelines -The site managers have the responsibility to install the farms, the needed software and have to provide the first basic support. -At the end of the test activity the Test Coordinator produces a report -The InfnGrid WP6 coordinators supervise the whole activity

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 12 Grid resource database and tools (by L. Fonti)

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 13 Grid Resource Database §A database has been created for the resources of the Grid pool. §The database backend is an XML file. §The database management tool is a java servlet. §All database management is done on the web, through web pages produced by servlet.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 14 XML database backend §XML has been choosen as database backend, because its element nesting capability is particularly suitable to reflect the hierarchical tree-based structure of Grid resource layout. §Moreover, XML offers an open, freely expandable database schema.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 15 XML file structure § § giis.ba.infn.it § § Giorgio Maggi § § gris1.ba.infn.it § fork § § Pentium III> § Linux RedHat 6.2 § 500 § 256 § 12 § gris1.ba.infn.it § gris2.ba.infn.it §

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 16 Database Access §Database access is granted via web with HTTPS protocol and it is reserved to users owning a valid certificate. §Database and management tools are based on a multi-organizational architecture, i.e. all the resources of DataGrid testbed could be housed in a single database. §However, the choice between centralized and distributed approach has not yet been taken: this question is still subject of discussion.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 17 Database management tools §The main management tool is a java servlet, which uses a simple, non validating parser to access the XML file, and to convert it to an object tree. §It also generates a set of web pages which allow users to examine, add, remove and modify unit entries.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 18 Database management tools §Additional tools have been developed to search nodes with given characteristics, and to allocate them to a testing group. §All management tools have been implemented as java servlets, and tool access is restricted to users owning a valid certificate.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 19

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 20

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 21 Search Tool §A tool has been developed to search the database for nodes matching a given set of requirements. §In the current version, allowed search parameters are: Organization, CPU type and speed, memory size and disk space.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 22

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 23

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 24 Resource Allocation Tool §Another tool has been developed to allow marking a node as reserved to a given group for testing or working purposes. §The tool shows on a web page a list of available resources, similar to the list produced by the main management tool, but with an allocation button shown near each node.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 25 Resource Allocation Tool §The label on the button is “Allocate” if the node is currently free, or “Release” if it is currently reserved. §Clicking the button will swap the node status.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 26

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 27 What about DataGrid?

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 28 What about DataGrid? May the INFN resource database and the tools be used for the whole DataGrid Testbed? May the InfnGrid guidelines be generalized for DataGrid? - a proposal (where the Planning Group is replaced by a Testbed Coordination Group, composed by 1-2 people for each national Testbed) has been submitted to the Integration Convergence Team (ICT) - comments by Federico Carminati - resent by FCA to WP8 TWG ** Need for discussion

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 29 InfnGrid Testbed activities

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 30 InfnGrid Testbed activities for 2001 Software environment, packaging - Integration of various software components - Installation kit production - Experiments’ software environment installation some of the above activities are performed with WP4 and WP8 Grid services - Information Service MDS-2 tests (presentation in the MDS-2 session) - Security, Certification and Authorization a draft document (“Authorization Service for Grid”) will be presented in the themed Security workshop

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 31 Services and Working Groups The GRID basic services Working Groups are: - Information Service Participants: A. Cavalli, A. Bassi, L. Fonti, F. Semeria, G. VitaFinzi - Security, Certification and Authorization Participants: R. Cecchini, R. Alfieri, L. Dell'Agnello, A. Gianoli, C. Rocca, F. Spataro - INFNGrid installation kit and integration Participants: F. Donno, S. Barale, A. Cavalli, A. Chierici, A. Italiano, D. Mura, M. Serra

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 32 InfnGrid Testbed activities for 2001 Resource access policies and allocation for the tests - guidelines for the resource allocation - resource database and tools for search/allocation - tools for the automatic creation of grid-mapfiles Monitoring - development of a simple monitoring tool for whole InfnGrid Testbed starting from the tools developed by the italian components of the LHC experiments (since WP3 tools will not be available in the short term).

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 33 InfnGrid Testbed activities for 2001 Support for Testbed activities: - by the site managers - by the grid services WGs - by the Installation kit group Testbed resources validation Goal: check if the testbed resources are running properly Operational tests of: - grid software installation - batch systems - grid core services - network Is this activity useful for DataGrid, too? Some preliminary discussions with S. Nicoud and E. Fedè, need for a decision.

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 34 Validation Tools Proposal (A. Cavalli) (cont.)  The tool could be done extending the features of the available MDS perl web browser.  It will retrieve the list of resources from the MDS.  Of each GRIS it will show: - Cpu model, speed, cpucount & Globus version; - Time needed to gather information; - Published cpu load attributes; - RTT (just this for now from network); - Defined scheduler. (*) * dynamical values: the 1st depends on the others

DataGrid Workshop Oxford, July 2-5 INFN Testbed status report Luciano Gaido 35 Validation Tools Proposal (2)  Join the tool developed in Torino (C++ program + some simple scripts) with this one to test the defined scheduler and show the results (open issue: a test user on all resources?).  Write some simple script to test GSI-ftp. ? Is it enough? Is there anything else that has to be checked? ? Is such a tool useful for DataGrid Testbed?