The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) WMS LB BDII Installation and Configuration Salma Saber ERI-Cairo, Egypt Joint EPIKH/EUMEDGRID Support event in Cairo Africa 4 – Site Admin Tutorial + Remote site inst. Egypt,

o The Workload Management System (WMS) is the gLite service that allows users to submit jobs, and performs all tasks required to execute them, without exposing the user to the complexity of the Grid. o The workload manager (WM) is the principal component in the WMS. o Purpose of the Workload Manager (WM) is to accept and satisfy requests for job management coming from its clients o The WM hands over the job to an appropriate Computing Element (CE) for execution taking into account requirements and the preferences expressed in the job description. 2 Workload Management System (WMS) Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o The Logging and Bookkeeping service (LB) tracks jobs in terms of event (important points of job life, e.g. submission, finding a matching CE, starting execution etc.) gathered from various WMS components as well as CEs. 3 Logging and Bookkeeping service (LB) Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

4 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

5 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Acts as the portal for accepting all external requests to the workload manager. Job submission, cancellation expressed via a JDL

6 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Keeps submission requests Requests are kept for a while for a while if no resources are immediately available

7 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Finds an appropriate CE for each submission request, taking into account job requests and preferences, Grid status, utilization policies on resources

8 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial A cache where overall resource information about the grid is stored for querying by the matchmaker. Updating these info via the information updater.

9 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Updating of information is carried out by reading information from the top- level BDII server of the virtual organization.

10 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial The LB is responsible to: - Stores events generated by the various components of the WMS - Querying the LB user can retrieve information about the status of the job

o The instructions provided in this presentation assume that you are going to use a machine with Scientific Linux 4.8 installed.  Virtual Machine image (GILDA VM Base_SLC4.8_i386) available at: Preliminaries Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

To login to the VM you can use the winscp then open a terminal and write the password Password: cairoschool 12 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Specify the mrepo host: export MREPO= o Configure the repository as follows: REPOS="dag ig glite-generic lcg-ca jpackage glite-bdii glite-wms" o Get repositories with: for name in $REPOS; do wget $MREPO/$name.repo -O /etc/yum.repos.d/$name.repo; done o Copy the gilda utils (Note: only for Gilda VO): wget -O /etc/yum.repos.d/gilda.repo 13 Repository settings Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Let’s check if date’s machine is correct with: date o if ntp date isn’t correct: /etc/init.d/ntpd status ntpdate ntp-1.infn.it o if not let’s configure file and make service start on boot: /etc/init.d/ntpd start chkconfig ntpd on 14 Network Time Protocol Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Please check: –If you have a valid hostname  To verify, type: hostname -f  To edit the hostname: vi /etc/hosts vi /etc/hostname vi /etc/sysconfig/network hostname 15 System requirements Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Modify the ig.repo as following: (vi /etc/yum.repos.d/ig.repo) # # INFNGRID repositories # [ig_sl4_i386] name = INFNGRID 3.1 i386 baseurl = enabled = 1 protect = 0 [ig_sl4_i386_externals] name = INFNGRID 3.1 i386 (externals) baseurl = enabled = 0 protect = 0 16 Some Modifications Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Modify the glite-generic.repo as following: (vi /etc/yum.repos.d/glite-generic.repo) # # gLite generic repositories # [glite-generic_sl4_i386_release] name = gLite generic 3.1 i386 (release) baseurl = enabled = 1 protect = 0 [glite-generic_sl4_i386_updates] name = gLite generic 3.1 i386 (updates) baseurl = enabled = 0 protect = 0 17 Some Modifications Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Then update your host: yum clean all o Install CAs on ALL profiles: yum install lcg-CA o Ig: yum install ig-yaim o Gilda utils: yum install gilda_utils 18 YUM install Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o WMS install: yum install ig_WMS 19 WMS installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

Transaction Check Error 20 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Transaction Check Error: file /opt/glite/lib/libvomsapi_nog.a conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api-noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api- noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so.0 conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api- noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security- voms-api-noglobus slc4 Transaction Check Error: file /opt/glite/lib/libvomsapi_nog.a conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api-noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api- noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so.0 conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security-voms-api- noglobus slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp slc4 and glite-security- voms-api-noglobus slc4 If continuing having this error try yum remove glite-security-voms-cpp

o This error will appear during the installation of ig_WMS: rpm -ivh rpm -ivh 1.i386.rpm yum install ig_WMS 21 Yum Install Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Error: Missing Dependency: google-perftools >= is needed by package ig_WMS Error: Missing Dependency: google-perftools-devel >= is needed by package ig_WMS Error: Missing Dependency: google-perftools >= is needed by package ig_WMS Error: Missing Dependency: google-perftools-devel >= is needed by package ig_WMS

cd /etc/yum.repos.d/ wget yum clean all yum install ig_LB 22 LB installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o BDII install: yum install ig_BDII 23 Top BDII installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Move the host cert to the right path mv /root/SRVXX.eun.eg/SRVXX.eun.eg-cert.pem /etc/grid-security/hostcert.pem mv /root/SRVXX.eun.eg/SRVXX.eun.eg-key.pem /etc/grid-security/hostkey.pem o Give them the right permissions chmod 400 /etc/grid-security/hostkey.pem chmod 644 /etc/grid-security/hostcert.pem ~]# ll /etc/grid-security/ total 32 drwxr-xr-x 2 root root Jun 27 12:08 certificates -rw-r--r-- 1 root root 887 Jun 16 10:02 hostcert.pem -r root root 887 Jun 16 10:02 hostkey.pem drwxr-xr-x 2 root root 4096 Jun 27 11:21 vomsdir 24 Host cetificates Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o generate configurations file from YAIM template cp /opt/glite/yaim/examples/siteinfo/ig-site- info.def /opt/glite/yaim/examples/siteinfo/mysite- info.def o Open your own site-info.def and customize it: vi /opt/glite/yaim/examples/siteinfo/mysite-info.def 25 site-info.def Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

26 WMS-LB Configuration (1/2) WMS Configuration INSTALL_ROOT=/opt MYSQL_PASSWORD=secret PX_HOST=myproxy.ct.infn.it WMS_HOST=SRVXX.eun.eg SITE_ = VOS="gilda"  Add gilda here ######### # gilda # ######### VO_GILDA_VOMS_CA_DN="/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it" VO_GILDA_SW_DIR=$VO_SW_DIR/gilda VO_GILDA_DEFAULT_SE=$DPM_HOST VO_GILDA_STORAGE_DIR=$CLASSIC_STORAGE_DIR/gilda VO_GILDA_VOMS_SERVERS="voms://voms.ct.infn.it:8443/voms/gilda?/gilda" VO_GILDA_VOMSES="gilda voms.ct.infn.it /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it gilda" Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

o Open glite-wms file and edit: vi /opt/glite/yaim/examples/siteinfo/services/glite-wms LB_HOST= “SRVXX.eun.eg:9000" GLITE_WMS_LCGMON_FILE="/var/log/wms_lcgmon.log" o Copy the gilda profiles inside ig ones cat /opt/glite/yaim/etc/gilda/gilda_ig-groups.conf >> /opt/glite/yaim/examples/ig-groups.conf cat /opt/glite/yaim/etc/gilda/gilda_ig-users.conf >> /opt/glite/yaim/examples/ig-users.conf o CONFIGURE /opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/mysite-info.def -n ig_WMS -n ig_LB 27 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial WMS-LB Configuration (2/2)

28 /opt/glite/etc/init.d/glite-wms-wmproxy: line 37: 6431 Segmentation fault $httpdpath -k -f $config ABORT: Service glite-wms-wmproxy failed to start! ERROR: Error during the execution of function: config_glite_wms ERROR: Error during the configuration.Exiting. [FAILED] /opt/glite/etc/init.d/glite-wms-wmproxy: line 37: 6431 Segmentation fault $httpdpath -k -f $config ABORT: Service glite-wms-wmproxy failed to start! ERROR: Error during the execution of function: config_glite_wms ERROR: Error during the configuration.Exiting. [FAILED] # rpm -qa |grep gridsite gridsite-apache gridsite-shared # rpm -e --nodeps gridsite-apache gridsite-shared # wget repository.cern.ch/repository/download/registered/org.glite/org.gridsite.core/1.1.20/slc4_ia32_ gcc346/gridsite-apache i386.rpm # wget repository.cern.ch/repository/download/registered/org.glite/org.gridsite.core/1.1.20/slc4_ia32_ gcc346/gridsite-shared i386.rpm # rpm -ivh gridsite-shared i386.rpm # rpm -ivh gridsite-apache i386.rpm THEN CONFIGURE AGAIN Configuration Error Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

29 Testing WMS-LB From a UI enter: ~]$ voms-proxy-init –voms gilda ~]$ export GLITE_WMS_WMPROXY_ENDPOINT= ms_wmproxy_server o Create a.jdl file: ~]$ vi hostname.jdl Type = "Job"; JobType = "Normal"; Executable = "/bin/hostname"; StdOutput = "hostname.out"; StdError = "hostname.err"; OutputSandbox = {"hostname.err","hostname.out"}; Arguments = "-f"; ShallowRetryCount = 3; Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

30 Testing WMS-LB ~]$ glite-wms-job-list-match -a hostname.jdl Connecting to the service ========================================================================== COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* - grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_long - grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_short - ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_long - ce1-egee.srce.hr:2119/jobmanager-sge-prod - gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - ce-edu.grid.acad.bg:2119/jobmanager-pbs-gilda - ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_short - sirius-ce.ct.infn.it:2119/jobmanager-lcgpbs-gilda - grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-gilda ========================================================================== Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

examples]# service gLite status *** globus-gridftp: globus-gridftp-server (pid 13095) is running... *** glite-wms-wmproxy: WMProxy httpd listening on port 7443 httpd (pid ) is running.... === WMProxy Server running instances: UID PID PPID C STIME TTY TIME CMD *** glite-wms-wm: /opt/glite/bin/glite-wms-workload_manager (pid 12466) is running... *** glite-wms-lm: Logmonitor running... *** glite-wms-jc: JobController running in pid: CondorG master running in pid: CondorG schedd running in pid: *** glite-wms-ice: /opt/glite/bin/glite-wms-ice-safe (pid 12636) is running... *** glite-proxy-renewald: glite-proxy-renewd running *** glite-lb-proxy: glite-lb-proxy running as *** glite-lb-locallogger: glite-lb-logd running glite-lb-interlogd not running *** glite-lb-bkserverd: glite-lb-notif-interlogd running glite-lb-bkserverd running as Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial Testing WMS-LB

32 BDII Configuration BDII_HOST=SRVXX.eun.eg SITE_NAME=GILDA_EGYPT BDII_HTTP_URL= " ite-bdii-update.conf " NTP_HOSTS_IP=" ntp-1.infn.it" VOS= "gilda" Then configure: /opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/mysite-info.def -n ig_BDII_top BDII Configuration Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

33 BDII Post Configuration Set the CA autoupdate on /opt/glite/yaim/etc/gilda/set_CAs_autoupdate.sh vi /opt/bdii/etc/bdii.conf Then Edit: BDII_AUTO_UPDATE=yes BDII_MODIFY_DN=yes BDII_UPDATE_URL= lite-bdii-update.conf Restart BDII /etc/init.d/bdii restart Test bdii ldapsearch -x -h localhost -p b mds-vo- name=local,o=grid Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

In order to customize the Site BDII list ( install the htppd service yum install -y httpd (if necessary) Publish your customized glite-bdii-update.conf file (i.e. /var/www/html/bdii) Set the BDII_UPDATE_URL accordingly in the file opt/bdii/etc/bdii.conf file Restart the BDII service /etc/init.d/bdii restart 34 Customize your BDII Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial

35 References  INFNGrid installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial