1. FP6−2004−Infrastructures−6-SSA-026409 www.eu-eela.org E-infrastructure shared between Europe and Latin America AMGA Server Installation Tony Calanducci INFN Catania First EELA Grid tutorial for users and system administrators Madrid, 20-24th February 2006

2. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 2 Introduction to Metadata and AMGA Requirements Installation References Outline

3. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 3 Metadata is data about data On the Grid: information about files –Describe files –Locate files based on their contents But also simplified DB access on the Grid –Many Grid applications need structured data –Many applications require only simple schemas  Can be modelled as metadata –Main advantage: better integration with the Grid environment  Metadata Service is a Grid component  Grid security  Hide DB heterogeneity Metadata on the GRID

4. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 4 ARDA developed an implementation of PTF interface –AMGA – ARDA Metadata Grid Application Began as prototype to evaluate the Metadata Interface –Evaluated by community since the beginning:  LHCb and Ganga were early testers (more on this later) –Matured quickly thanks to users feedback Now part of gLite middleware –Official Metadata Service for EGEE –First release with gLite 1.5 –Also available as standalone component Expanding user community –HEP, Biomed, UNOSAT… AMGA Implementation

5. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 5 Some basic concepts –Metadata - List of attributes and their values associated with entries –Attribute – key/value pair with type information  Type – The type (int, float, string,…)  Name/Key – The name of the attribute  Value - Value of an entry's attribute –Schema – A set of attributes –Collection – A set of entries associated with a schema –Think of schemas as tables, attributes as columns, entries as rows Metadata Concepts

6. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 6 Dynamic Schemas –Schemas can be modified at runtime by client  Create, delete schemas  Add, remove attributes Metadata organised as an hierarchy –Collections can contain sub-collections –Analogy to file system:  Collection  Directory; Entry  File Flexible Queries –SQL-like query language –Joins between schemas –Example selectattr /DLibrary:FileName /DLAudio:Author /DLAudio:Album '/DLibrary:FILE=/DLAudio:FILE and like(/DLibrary:FileName, “%.mp3")‘ AMGA Features

7. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 Security Unix style permissions ACLs – Per-collection or per-entry. Secure connections – SSL Client Authentication based on –Username/password –General X509 certificates –Grid-proxy certificates Access control via a Virtual Organization Management System (VOMS):

8. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 8 C++ multiprocess server –Runs on any Linux flavour Backends –Oracle, MySQL, PostgreSQL, SQLite Two frontends –TCP Streaming  High performance  Client API for C++, Java, Python, Perl, Ruby –SOAP  Interoperability Also implemented as standalone Python library –Data stored on filesystem AMGA Implementation

9. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 9 AMGA Server Installation

10. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 10 Start from a fresh install of SLC 3.0.x Verify that apt is installed and properly configured: rpm -qa | grep apt Install apt if necessary: rpm -ivh http://linuxsoft.cern.ch/cern/slc30X/i386/SL/RPMS/apt-0.5.15cnc6- 8.SL.cern.i386.rpmhttp://linuxsoft.cern.ch/cern/slc30X/i386/SL/RPMS/apt-0.5.15cnc6- 8.SL.cern.i386.rpm Java SDK is required to install the AMGA Server. Due to license issues, it is not found in the gLite repositories. So please download and install a release of Java SDK >= 1.4.2_08 from http://java.sun.com/j2se/1.4.2/download.html http://java.sun.com/j2se/1.4.2/download.html chmod +x j2sdk-1_4_2_10-linux-i586-rpm.bin./j2sdk-1_4_2_10-linux-i586-rpm.bin rpm -ivh j2sdk-1_4_2_10-linux-i586.rpm Preparing... ########################################### [100%] 1:j2sdk ########################################### [100%] Installation Pre-requisites

11. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 11 Request host certificates for your AMGA Server from your CA. For example, to request host certs from GILDA CA go to: – https://gilda.ct.infn.it/CA/mgt/restricted/srvreq.php https://gilda.ct.infn.it/CA/mgt/restricted/srvreq.php Install host certificates (hostcert.pem and hostkey.pem) in /etc/grid-security. – chmod 644 hostcert.pem – chmod 400 hostkey.pem If you plan to use certificates released by unsupported EGEE CA’s, be sure that their public key, signing policy and CRLs (usually distributed with an rpm) are installed. – For the VO GILDA, the RPM is available from https://gilda.ct.infn.it/RPMS/ca_GILDA-0.28.1.i386.rpm Security pre-requisites

12. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 12 DB requirements AMGA server to its operation needs a database backend. It can be based on one of the following database services: MySQL, Oracle, PostgreSQL, SQLlite. –As of version 1.0.X of the gLite AMGA server deployment module, only “mysql” database backend is supported. Steps to install and configure properly a MySQL backend: –All the necessary packages can be found on: http://glite.web.cern.ch/glite/packages/externals/bin/rhel30/RPM S/ –Download and install the following packages:  MySQL-server-4.1.11-0.i386.rpm  MySQL-client-4.1.11-0.i386.rpm

13. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 13 MySQL configuration Set a Password for the MySQL root user: –/usr/bin/mysqladmin -u root password ‘holamadrid‘ –/usr/bin/mysqladmin -u root -h gilXX.ciemat.es password ‘holamadrid' Create a DB to be used by the AMGA service. Using mysql client (start it by ‘mysql -p’ command): –mysql> create database ; (ex: metadata) Create a DB user and grant him all rights on the previously created db: –mysql> grant all privileges on metadata.* to 'amgamgr'@'gil31.ciemat.es' identified by 'holaAMGA';

14. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 14 Other requirements An ODBC driver for the chosen DB backend has to be installed. For that purpose we can use the free unixODBC drivers that comes along with Scientific Linux Cern distribution. –apt-get install unixODBC Additionally, download and install the MySQL ODBC driver. You can get it from –http://glite.web.cern.ch/glite/packages/externals/bin/rhel30/RPM S/mysql-connector-odbc-3.51.12-1_EGEE.i386.rpmhttp://glite.web.cern.ch/glite/packages/externals/bin/rhel30/RPM S/mysql-connector-odbc-3.51.12-1_EGEE.i386.rpm Boost free portable C++ libraries. You can get it from: –http://glite.web.cern.ch/glite/packages/externals/bin/rhel30/RPM S/boost-1.32.0-1_EGEE.i386.rpmhttp://glite.web.cern.ch/glite/packages/externals/bin/rhel30/RPM S/boost-1.32.0-1_EGEE.i386.rpm –Use rpm -ivh --force for that

15. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 15 1.Verify if apt is present in your installation: – rpm -qa | grep apt – Install apt if necessary:  rpm -ivh http://linuxsoft.cern.ch/cern/slc30X/i386/SL/RPMS/apt- 0.5.15cnc6-8.SL.cern.i386.rpmhttp://linuxsoft.cern.ch/cern/slc30X/i386/SL/RPMS/apt- 0.5.15cnc6-8.SL.cern.i386.rpm 2.Add the gLite apt repository : –Put this line in a new file (e.g. glite.list) inside the /etc/apt/sources.list.d directory (R 1.5): rpm http://glitesoft.cern.ch/EGEE/gLite/APT/R1.5/ rhel30 externals Release1.5 updates – apt-get update – apt-get dist-upgrade 3.Install the AMGA Server: – apt-get install glite-amga-server-config More info at http://glite.web.cern.ch/glite/packages/APT.asp Installing AMGA Server via apt

16. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 16 This will install the following deployment modules: – AMGA server – AMGA client – R-GMA servicetool – Security tool The AMGA R-GMA server configuration script is installed in – $GLITE_LOCATION/etc/config/scripts/glite-amga-server- config.py Templates configurations files are located in – $GLITE_LOCATION/etc/config/templates Installing AMGA Server via apt (cont.) By default GLITE_LOCATION=/opt/glite

17. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 17 Copy the configuration template files from –$GLITE_LOCATION/etc/config/templates To –$GLITE_LOCATION/etc/config P.S: You can delete the glite-rgma-servicetool- serviceName.cfg.xml template file because it will not be used. Now we will start to customize the just copied configuration files by replacing the “changeme” value in all user-defined parameters with the proper values. AMGA Server configuration

18. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 18 List of XML files to customize: glite-global.cfg.xml glite-amga-server.cfg.xml glite-amga-client.cfg.xml glite-security-utils.cfg.xml glite-rgma-common.cfg.xml AMGA Server Configuration (cont.)

19. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 19 glite-global.cfg.xml Check the correct path of your JAVA JRE/JDK.

20. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 20 glite-amga-server.cfg.xml <amga.server.DBUser scope="user" description="Database user name" value=“amgamgr"/> <amga.server.DBPass scope="user" description="Database user password" value=“holaAMGA"/> <amga.server.DBName scope="user" description="Database name" value=“metadata"/> <amga.server.DBHost scope="user" description="Database host" value=“gil08.ciemat.es"/>

21. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 21 glite-rgma-common.cfg.xml <rgma.server.hostname description="Host name of the R-GMA server. [Example: lxb1420.cern.ch] [Type: 'string']" value=“gilXX.ciemat.es"/> <rgma.schema.hostname description="Host name of the R-GMA schema service. (See also configuration parameter 'rgma.server.run_schema_service' in the R-GMA server configuration file in case you install a server). [Example: lxb1420.cern.ch] [Type: 'string']" value=" gilXX.ciemat.es "/> <rgma.registry.hostname description="Host name of the R-GMA registry service. You must specify at least one hostname and you can specify several if you want to use several registries. (See also configuration parameter 'rgma.server.run_registry_service' in the R-GMA server configuration file in case you install a server). [Example: lxb2029.cern.ch] [Type: 'string']"> gilXX.ciemat.es

22. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 22 <amga.client.Host scope="user" description="Name of the AMGA server host. If empty 'localhost' is used" value="gil31.ciemat.es"/> <amga.client.Login scope="user" description="The login name of the user on the AMGA server." value="guest"/> glite-amga-client.cfg.xml

23. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 23 < rgma.servicetool.siteId description="Unique Id of the site your service is running in. This parameter must have the same value as the rgma.site-publisher.siteId parameter in the R-GMA Server configuration. It has to be a DNS entry owned by the site and does not have to be shared with another site (i.e it uniquely identifies the site). It normally defaults to the DNS name of the R-GMA Server running the Site Publisher service. [Example: lxb1420.cern.ch] [Type: 'string']" value=" gil04.ciemat.es "/> <rgma.servicetool.vo description="List of VOs that this service is considered part of.> gilda glite-rgma-servicetool.cfg.xml

24. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 24 glite-security-utils.cfg.xml <cron.mailto description="E-mail address for sending cron job notifications" value="root@localhost"/>

25. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 25 Change to the script directory: – cd /opt/glite/etc/config/scripts Configure the AMGA server by executing the R-GMA Server configuration script: –./glite-amga-server-config.py --configure If no error message is reported by the previous step, start the AMGA server by running –./glite-amga-server-config.py --start Configure and start the AMGA Server

26. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 26 Check that AMGA server is running properly by typing: –./glite-rgma-server-config.py --status Copy the AMGA Client configuration file from /etc/mdclient.config to $PWD/.mdclient.config. Start the AMGA client with: –mdclient If you want to stop the AMGA Server, you can do it by issuing –./glite-amga-server-config.py --stop Verify the installation

27. E-infrastructure shared between Europe and Latin America First EELA Grid tutorial for users and system administrators, 20-24.02.2006 27 References gLite Installation Guide Release 1.5 –http://glite.web.cern.ch/glite/packages/R1.5/R20051130/doc/inst allation_guide.htmlhttp://glite.web.cern.ch/glite/packages/R1.5/R20051130/doc/inst allation_guide.html gLite AMGA Administration & User Guide –http://project-arda-dev.web.cern.ch/project-arda- dev/metadata/downloads/amga-manual_1.1.0.pdf