1. Procedures on how to enter the GRID Christos Papachristos Site Manager of the HG-05-FORTH and GR-04-FORTH-ICS nodes Distributed Computing Systems Lab
Crete, 19th Jun

2. Answers provided to… Who can apply for registration?
What is the registration procedure?
Who should be contacted?
What are the Virtual Organizations?
What tools do I need and where can I find them?
How do I submit jobs to the Grid?
What is the procedure of submitting jobs?
Tutorials on job submission.
Failed jobs. Who should be contacted?
Where can I see the status of the Grid?
What does the infrastructure look like?
Is there any user support?
..and some other Grid related issues.
Crete, 19th Jun

3. Who can apply for registration?
The whole Greek research community
Universities
Technical Education Centers
Research Centers
The Eligible (global) Grid User Community is defined to be people from: 
the academic research and education community;
public funded bodies, institutes and research laboratories working in areas related to research and education;
other bodies and organisations, such as commercial companies, who are collaborating on academic research and education with the above institutes.
EGEE Access Users Policy (AUP)
Crete, 19th Jun

4. Registration procedure
Register yourself as a HellasGRID user
Get an account to a User Interface node (UI)
Acquire a digital certificate from your accredited National Certification Authority (CA)
Join an existing Virtual Organization (VO)
Crete, 19th Jun

5. HellasGRID registration. Step 1
The first step in order to enroll is to fill the User Registration form at the HellasGrid website
This will send a notification to the User Support Team which will get in touch with you in order to clarify your needs and help you with your first steps in joining the EGEE Grid infrastructure.
Crete, 19th Jun

6. Get a UI account. Step 2
In order to become a user of the EGEE Grid Infrastructure you should:
Gain access to a User Interface (UI) machine which hosts the required client software for developing and running applications in the EGEE Grid
A UI is a Linux box having the appropriate client software that will help you send and execute your applications in the Grid
The choices on how to obtain a UI account are:
Set up your own UI node
Obtain access to a local UI, already installed in your organization
hellasgrid-ui.ics.forth.gr. Send an at
{cpapachr,
Request for a UI account in the GRNET node
Crete, 19th Jun

7. Get a UI account. Step 2.1
In practice everyone can install and setup a UI with the required EGEE tools following the instructions in the LCG-2 Installation Guide.
Also the Italian SA1 EGEE Activity has made available a package called UIPnP which can be installed in any Linux machine and turn it into a User Interface (you can even install it as a regular non-root user). Note though that it is pre-configured for accessing the Italian EGEE sites and some manual configuration steps are required at the moment in order to be useful for EGEE-SEE users (for instructions see How to customize UIPnP for SEE-VO).
Crete, 19th Jun

8. Acquire a Digital Certificate. Step 3
In order to access the Grid you need a Digital Certificate.
First of all a fax has to be sent to the National Certification Authority (fax: ) with names and IDs.
Then:
Login to your UI account and execute the following
$grid-cert-request
The command will produce a .globus directory with 3 files: .globus/usercert.pem size=0 .globus/usercert_request.pem size<>0 .globus/userkey.pem size<>0
The user-cert-request.pem file has to be ed to the HellasGrid-CA
Crete, 19th Jun

9. Acquire a Digital Certificate. Step 3.1
After you receive your signed certificate
Overwrite the usercert.pem file with the signed certificate (e.g #cp /home/cpapachr/.globus/usercert.pem
Import your certificate to your and web browsers
Transform the certificate to pkcs12 format by executing the following command in the UI: #openssl pkcs12 -in usercert.pem -inkey userkey.pem -export -out certkey.p12
Follow the instructions to import the certificate to the browser
Return a signed to the HellasGRID CA accepting the certificate.
Crete, 19th Jun

10. Join an existing VO. Step 4
VO=Virtual Organization. A group of scientists with common research interests.
Most popular VOs can be found here,
Most VOs are those of CERN dealing with experiments based on data that will be produced from the LHC (Large Hadron Collider) which will be fully operational in 2007.
In our region we have the SEE VO which is supported by 27 sites.
Crete, 19th Jun

11. Our SEE VO
In order to make life easier for the regional users, speed up and simplify the process of new application induction, SEE has established its own VO called SEE-VO.
This VO will be the most adequate for SEE users that do not fit in any of the existing VOs.
In order to request acceptance to the SEE-VO as a user you have to fill the following request form:
Please note that this page has to be visited using the browser on which you have already loaded your digital certificate otherwise the process cannot be completed.
Crete, 19th Jun

12. Brief summary of the procedure…
login
%HOME/.globus/usercert-request.pem
%HOME/.globus/userkey.pem
%HOME/.globus/usercert.pem
Start using the GRID CA UI
Acceptance
Fax your ID
Usercert.pem
usercert-request.pem
Import pkcs12 file into mailer/browser
Generate pkcs12 file
grid-cert-request
Public key is signed and
the Certificate is issued
Crete, 19th Jun

13. Ready to use the infrastructure!
Crete, 19th Jun

14. A simplified overview of a job workflowUI
JDL
Input “sandbox”
grid-proxy-init
Storage
Element
Compute
Output “sandbox”
Resource
Broker
Output “sandbox”
Input “sandbox” + Broker Info
Author.
&Authen.
Storage
Element
Compute
Storage
Element
Compute
Storage
Element
Compute
Crete, 19th Jun

15. How do I submit jobs to the Grid?
Create a proxy certificate
Write your own .jdl file (JDL:Job Description Language file)
Submit your job
Retrieve the status of the job
Get the output of the job
Crete, 19th Jun

16. Step 1- Creating a proxy certificate
cpapachr]$ grid-proxy-init -debug -verify
User Cert File: /home/cpapachr/.globus/usercert.pem
User Key File: /home/cpapachr/.globus/userkey.pem
Trusted CA Cert Dir: /etc/grid-security/certificates
Output File: /tmp/x509up_u0
Your identity: /C=GR/O=HellasGrid/OU=ics.forth.gr/CN=Christos Papachristos
Enter GRID pass phrase for this identity:
Creating proxy
Done
Proxy Verify OK
Your proxy is valid until: Wed Jun 14 02:49:
cpapachr]$
Crete, 19th Jun

17. Step 2 – Creating the JDL file
test.jdl
test.sh
Executable = "test.sh";
StdOutput = "test.out";
StdError = "test.err";
InputSandbox = {"./test.sh"};
OutputSandbox = {"test.out","test.err"};
Requirements = other.GlueCEUniqueID=="grid001.ics.forth.gr:2119/jobmanager-lcgpbs-dteam";
RetryCount = 0;
#!/bin/bash
echo Hostname: `hostname`
echo Working Directory: `pwd`
echo Running As: `id`
echo Executed in: `date`
echo Enviroment Variables: `env`
Crete, 19th Jun

18. Step 3 – Submitting the job
cpapachr]$ edg-job-submit --vo dteam test.jdl
Selected Virtual Organisation name (from --vo option): dteam
Connecting to host rb.isabella.grnet.gr, port 7772
Logging to host rb.isabella.grnet.gr, port 9002
*********************************************************************************************
JOB SUBMIT OUTCOME
The job has been successfully submitted to the Network Server.
Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: -
Keep this URL for further reference to the job already submitted
cpapachr]$
Crete, 19th Jun

19. Step 4 – Querying the job status
cpapachr]$ edg-job-status
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job :
Current Status: Scheduled
Status Reason: Job successfully submitted to Globus
Destination: grid001.ics.forth.gr:2119/jobmanager-lcgpbs-dteam
reached on: Tue Jun 13 15:54:
Finally after a couple of minutes…
Current Status: Done (Success)
Exit code:
Status Reason: Job terminated successfully
reached on: Tue Jun 13 16:00:
Crete, 19th Jun

20. Step 5 – Retrieving the results
cpapachr]$ edg-job-get-output
Retrieving files from host: rb.isabella.grnet.gr ( for )
*********************************************************************************
JOB GET OUTPUT OUTCOME
Output sandbox files for the job: -
have been successfully retrieved and stored in the directory:
/tmp/jobOutput/cpapachr_GgfNDFsgB0L3btK_2CRfKQ
Crete, 19th Jun

21. Tutorials on job submission
The LCG-2 User Guide describing almost everything
A small tutorial with information on JDL files and job submission
There is also MPI support (MPICH v1.2.5). A test example can be found here
Crete, 19th Jun

22. Failed job submissions
Who do I contact if my jobs fail?
GGUS system is the main user support framework.
Contact via as well:
(<MyVO>=atlas, biomed, ...)
Crete, 19th Jun

23. Monitoring the Grid Where can I see the status of the Grid?
The Grid Operations Centre (GOC) maintains a map with the overall state of the grid and links to various monitoring pages.
Our ROC also keeps track of the regional Grid resources of the SEE
Crete, 19th Jun

24. Monitoring services to check
GridICE monitoring server
General status of the Grid
Network latency monitoring
Crete, 19th Jun

25. A closer look at the infrastructure
The HG-05-FORTH site
Crete, 19th Jun

26. HG-05-FORTH Hardware Software 60 nodes with dual Intel xeon processors
3.4GHz each
SATA disk 80GB
4 nodes with dual Intel xeon processors
2 SCSI disks RAID (80GB each)
4.2TB of storage (300GBx12disks)
6 CISCO 3750 switches
Software
1 CE
1 SE (SE_dpm_mysql)
1 MON BOX
1 Quattor
60 WNs
GPFS on SE and quattor
(storage)
All running:
LCG middleware v. 2_7_0
Operating System: SL3.05
Crete, 19th Jun

27. User Support Team
Operated by NCSR “Demokritos” - Institute of Nuclear Physics (
Responsibilities – To guide new users in their first steps in the Grid and provide continuous support in their activities.
Wiki (
Mailing list – Includes all registered grid users
Newsletter – Published every ~2 months in Greek. Provides local and EGEE-wide news update (
Crete, 19th Jun

28. For urgent matters {cpapachr, hargikas}@ics,forth.gr
The site managers can be contacted.
{cpapachr,
Crete, 19th Jun

29. MPI example: MPItest.c #include "mpi.h" #include <stdio.h>
int main(int argc, char *argv[]) {
int numprocs; /* Number of processors */
int procnum; /* Processor number */
/* Initialize MPI */
MPI_Init(&argc, &argv);
/* Find this processor number */
MPI_Comm_rank(MPI_COMM_WORLD, &procnum);
/* Find the number of processors */
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
printf ("Hello world! from processor %d out of %d\n", procnum, numprocs);
/* Shut down MPI */
MPI_Finalize(); return 0; }
Crete, 19th Jun

30. MPI example: MPItest.jdl
Type = "Job";
JobType = "MPICH";
NodeNumber = 10;
Executable = "MPItest.sh";
Arguments = "MPItest";
StdOutput = "test.out";
StdError = "test.err";
InputSandbox = {"MPItest.sh","MPItest.c"};
OutputSandbox = {"test.err","test.out","mpiexec.out"};
Requirements = (other.GlueCEInfoLRMSType == "PBS") && RegExp("grid.*.lal.in2p3.fr.*",other.GlueCEUniqueID);
Crete, 19th Jun

31. MPI example: MPItest.sh
#!/bin/sh -x
# the binary to execute
EXE=$1
echo "**********************************
echo "Running on: $HOSTNAME"
echo "As: " `whoami`
echo "Compiling binary: $EXE"
echo mpicc -o ${EXE} ${EXE}.c
mpicc -o ${EXE} ${EXE}.c
if [ "x$PBS_NODEFILE" != "x" ] ; then
echo "PBS Nodefile: $PBS_NODEFILE"
HOST_NODEFILE=$PBS_NODEFILE fi
if [ "x$LSB_HOSTS" != "x" ] ; then
echo "LSF Hosts: $LSB_HOSTS"
HOST_NODEFILE=`pwd`/lsf_nodefile.$$
for host in ${LSB_HOSTS} do
echo $host >> ${HOST_NODEFILE}
done
if [ "x$HOST_NODEFILE" = "x" ]; then
echo "No hosts file defined. Exiting..."
exit
echo "************************************
CPU_NEEDED=`cat $HOST_NODEFILE | wc -l`
echo "Node count: $CPU_NEEDED"
echo "Nodes in $HOST_NODEFILE: "
cat $HOST_NODEFILE
echo "Checking ssh for each node:"
NODES=`cat $HOST_NODEFILE`
for host in ${NODES} do
echo "Checking $host..."
ssh $host hostname
done
echo "Executing $EXE with mpiexec"
chmod 755 $EXE
mpiexec `pwd`/$EXE > mpiexec.out 2>&1
echo "Executing $EXE with mpirun"
mpirun -np $CPU_NEEDED -machinefile $HOST_NODEFILE `pwd`/$EXE
Crete, 19th Jun