Overview of software tools for gLite installation & configuration Bouchra RAHIM (rahim@cnrst.ma) Joint EPIKH/EUMEDGRID-Support Event in Rabat Morocco, 30.05.2011

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Agenda Virtual machines PuTTY + WinScp Screen Command Yum command Local repository (mrepo command) Yaim command Passwords & certificates Event links Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 2

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Virtual Machine Practice on virtual machine till next Wednesday during the site Admin tutorial. Each participant has access to 2 virtual machines that are hosted on his PC (one 64 bit and one 32 bit Linux). All grid nodes will be installed on 64 bit Linux except the WMS. All virtual machines have host certificates delivered by Magrid CA. The following UI is available for testing: ui01.magrid.ma With users gridXX and password gridXX where (XX=01 to 30) . ssh gridxx@ui01.magrid.ma Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 3

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 PuTTY + WinScp Root password: grid2011 Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 4

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Screen Command Interface PC Remote PC Ssh connection Unreliable network connection with the remote PC. You can start some long command and go for break without worrying about losing the connection. Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 5

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Screen Command usage To download and install the screen package: # yum install screen To create a screen: # screen –p screen_name To list the current screens: # screen –ls To open an existed screen: # screen –x <name of screen listed by the previous command> Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 6

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 YUM package manager Package manager is an automatic updater and package installer/remover for rpm systems It automatically computes dependencies and figures out what things should occur to install packages Files in /etc/yum.repos.d/package_name.repo will point to the repository path where the package rpms are located. Yum install gLite_service will invoke the download and the installation in one step Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 7

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Local repository Why? gLite includes a large number of tools and software to download Download from remote servers can require too much time When? Many server to install Large server farms Training events Slow connection How? gLite repository are managed by mrepo tool Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 8

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Local repository The mrepo service retrieves rpms from mirror sites and creates a repository with YUM meta-data Manages the local YUM repository for OS and middleware Requires configuration Publish the packages on the web Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 9

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Local repository installation Prepare a Scientific Linux 5.4 machine The minimal Hard Disk storage is 160 GB Starting from a Linux machine, retrieve the mrepo and createrepo tools, and ensure that httpd server is running (if you see a file /etc/yum.repos.d/dag.repo you can install mrepo more simply with yum --enablerepo=dag install mrepo.) Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 10

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Local repository installation (cont.) Edit the /etc/bin/mrepo script as follows: *** /usr/bin/mrepo 2010-03-22 01:57:41.000000000 +0100 --- mrepo.modified 2010-07-04 20:58:02.000000000 +0200 *************** *** 845,851 **** repoopts = repoopts + ' --cachedir "%s"' % cachedir if os.path.isdir(os.path.join(self.wwwdir, '.olddata')): remove(os.path.join(self.wwwdir, '.olddata')) ! groupfile = os.path.join(cf.srcdir, self.dist.nick, self.name + '-comps.xml') if os.path.isfile(groupfile): symlink(groupfile, os.path.join(self.wwwdir, 'comps.xml')) repoopts = repoopts + ' --groupfile "%s"' % groupfilename --- 845,853 ---- ! # groupfile = os.path.join(cf.srcdir, self.dist.nick, self.name + '-comps.xml') ! groupfile = os.path.join(cf.srcdir, self.dist.nick, self.name + '/comps.xml') ! info(2, 'Groupfile: %s' % (groupfile)) you will need to modify the following in the mrepo script, since the path to the comps.xml file is not composed properly. Or at the end createrepo -g /var/mrepo/glite_sl5-x86_64/ui-release/repodata/comps.xml /var/www/mrepo/glite_sl5-x86_64/RPMS.ui-release/ createrepo -g /var/mrepo/glite_sl5-x86_64/wn-release/repodata/comps.xml /var/www/mrepo/glite_sl5-x86_64/RPMS.wn-release/ Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 11 11

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Local repository installation (cont.) After installing the package, edit /etc/mrepo.conf file to read more or less as follows: [main] srcdir = /data2/mrepo # Make wwwdir to point to a directory served by the webserver wwwdir = /var/www/mrepo confdir = /etc/mrepo.conf.d arch = x86_64 metadata=yum repomd # option -P=4 will be passed to lftp, to allow parallel downloads #-P=4 option sometimes gives troubles...# lftp-mirror-options = -c -P=4 -X "*/Fermi/*" -X "*/example/*" lftp-mirror-options = -c -X "*/Fermi/*" -X "*/example/*" # Option -d is used to make 'yum groupinstall' work: it can be used ONLY: from createrepo 0.4.11 onwards. # If you have an older version of createrepo, comment out the following line createrepo-options = -d -p Note that all packages will be downloaded to srcdir On the other hand, the path pointed to by wwwdir will contain mainly compressed xml files, plus symbolic links to files under srcdir.   Configuration files for individual repositories to be mirrored shall be put under confdir. Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 12 12

Local repository installation (cont.) Go to /etc/mrepo.conf.d directory and put here a Distribution file for each repository you want to mirror #mrepo –ugfv : run mrepo with generate and update actions, to generate the local repository structure and update it from the remote repository [glite_sl5] name = gLite Middleware ($release - $arch) release = R3.2 arch = x86_64 # Generic Repositories generic-externals = http://linuxsoft.cern.ch/EGEE/gLite/$release/glite- GENERIC/sl5/$arch/RPMS.externals/ generic-release = http://linuxsoft.cern.ch/EGEE/gLite/$release/glite-GENERIC/sl5/$arch/RPMS.release/ generic-updates = http://linuxsoft.cern.ch/EGEE/gLite/$release/glite-GENERIC/sl5/$arch/RPMS.updates/ Cairo, Joint EPiKH/EUMEDGRID-Support in Egypt, 17.10.2010 13 13

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Using the local repository The repo files must be configured to point to the local repository. Then running yum install normally. # # Magrid repositories # This is the official YUM repository string for the glite 3.2 User Interface # Fetched from: http:///grid-deployment.web.cern.ch/grid-deployment/yaim/repos/glite-UI.repo # Place it to /etc/yum.repos.d/ and run 'yum update‘ [glite-UI] name=gLite 3.2 User Interface baseurl = http://repo.magrid.ma/mrepo/glite_sl5-$basearch/RPMS.ui-release/ http://glitesoft.cern.ch/EGEE/gLite/R3.2/glite-UI/sl5/$basearch/RPMS.release/ gpgkey=http://glite.web.cern.ch/glite/glite_key_gd.asc gpgcheck=0 enabled=1 Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 14

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Yaim Command YAIM (Yet Another Installation Method) is a software to configure grid services. You need to create a structure similar to the example found under /opt/glite/yaim/examples/siteinfo: mkdir /root/siteinfo rsync -av /opt/glite/yaim/examples/siteinfo/ /root/siteinfo/ Edit file /root/siteinfo/mysite-info.def Setting ALL the needed variables in the configuration file is essential for a successful configuration. Note that the SAME identical /root/siteinfo/ can be used to configure the whole site so you can either substitute /root/siteinfo with an NFS-shared path, or synchronize /root/siteinfo between servers: rsync -e ssh -av <server1>:/root/siteinfo /root/ A full documentation for the configuration variables is listed here: https://twiki.cern.ch/twiki/bin/view/LCG/Site-info_configuration_variables Running the yum configuration script should be from the following path: /opt/glite/yaim/bin/yaim which invokes a set of bash_script with a modular structure, depending on the type of site, Grid service to be configured Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 15

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Passwords & certificates Virtual machine password: grid2011 Host pair Xcertificates are located on /root/ on the virtual machine : pcxxcert.pem  hostcert.pem pcxxkey.pem  hostkey.pem For the UI user certificates are located on /root/user_cert and must be copied to the .globus directory in the user’s home directory Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 16

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 Event links Site Administration : http://agenda.ct.infn.it/conferenceOtherViews.py?view=standard&c onfId=524 Application porting: http://agenda.ct.infn.it/conferenceOtherViews.py?view=standard&c onfId=525 Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 17

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 References glite documentation: http://glite.web.cern.ch/glite/documentation/ https://twiki.cern.ch/twiki/bin/view/LCG/GenericInstallGuide320 YUM package manager and YAIM tool: http://yum.baseurl.org/ https://twiki.cern.ch/twiki/bin/view/LCG/YaimGuide400#Basics Local repository installation: http://wiki.eumedgrid.eu/twiki/bin/view/InfrastructureStatus/Eume dSiteInstallation Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 18

Rabat, Joint EPIKH/EUMEDGRID Support Site Admin 30.05.2011 19