1. Grid Services
Ouafa Bentaleb
CERIST, Algeria
Africa 6, Rabat ­ Joint CHAIN/EUMEDGRID-Support/EPIKH School Application Porting

2. Introduction
31/07/2018
The Grid relies on advanced software, called middleware, which interfaces between resources and applications
Deployed by the Enabling Grids for E-sciencE projects (EGEE)
presently is supporting Scientific Linux 4 and 5, and also Debian 4
gLite 3.2 is the newest version based on SL5
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

3. Introduction
31/07/2018
combines components developed in various related projects (condor, Globus,..)
Provides the user with high level services for:
Scheduling and running computational jobs
Accessing and moving data
Obtaining information on the Grid infrastructure as well as Grid applications
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

4. What gLite does? What does not What does
31/07/2018
What does not
Somehow my application just run faster
Users can access to any resource
Users can rely of a huge amount of software, libraries
What does
Provides sharing of resources (CPUs, Storage, Sensors …)
Allows the creation of virtual organizations (People, Groups)
Applications will run faster only if properly developed and best if thought for Grid environments (Trivial parallelization, MPIs)
Provides access to computational/storage/other resources accordingly to defined: Policies and Access rights
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

5. gLite Grid Scenario
Users of different organizations geographycally distant, are grouped in Virtual Organizations that share computational, storage and other resources among other VOs resulting in COLLABORATION!
Members of a Grid can be part
of different VOs
at the same time
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

6. gLite Grid services
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

7. gLite Grid components services ½
The User Interface (UI), is the user access point to the Grid site.
The Workload Management System (WMS), its a set of services which is responsible of finding Computing and Storage resources available on a Grid to manage job transparently.
WMProxy, is component of gLite (WMS) that is responsible for accepting incoming requests from the User Interface
The Logging and bookkeeping service (LB), It keeps fresh information about the status of jobs processed by associated WMS instances allowing user for jobs status checks: Ready, Scheduled, Waiting, Running, Done
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

8. gLite Grid components services 2/2
Information System (IS), maintain data related to available grid resources and their health statuses, in order to optimize resource exploitation
The Computing element (CE), A batch queue on a site’s computers where the user’s job is executed
The Worker Nodes are the machines where jobs are really executed and managed by the CE’s queue management system
Storage Element (SE), allow to manage Grid files and offer a mechanism to locate them easily for users and jobs
Logical File Catalog (LFC), Location of grid files and grid file replicas
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

9. gLite - Grid services
gLite services can be grouped in 5 main high level set of services
Grid Access
Security
Information system & Monitoring
Job Workload Management System
Data Management
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

10. gLite - Grid access
All gLite services are accessible via APIs and CLIs
API - Application Programming Interface
job management ( Java, Python, C++ )
Data management ( Java, C++ )
Resource discovering
CLI – Command Line Interface
A set of command lines utilitities that are distribuited together with the gLite-UI packages
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

11. gLite - Security Services
Authentication based on X.509 PKI infrastructure
Certification Authorities (CA) issue (long lived) certificates identifying individuals (much like a passport)
Commonly used in web browsers to authenticate to sites
Trust between CAs and sites is established (offline)
In order to reduce vulnerability, on the Grid user identification is done by using (short lived) proxies of their certificates
Proxies can
Be delegated to a service such that it can act on the user’s behalf
Include additional attributes (like VO information via the VO Membership Service VOMS)
Be stored in an external proxy store (MyProxy)
Be renewed (in case they are about to expire)
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

12. gLite - Security Services
31/07/2018
Authentication
The user receives a certificate from a Certification Authorities (CA)
The user has to subscribe to a VO
He/She connects to the UI ( typically via SSH )
He/She creates the user proxy, all Grid services will use this proxy to identify the user - then Grid Security Infrastructure identifies user to other machines
Authorisation
The VO establish the user rights
In any Grid service it will be verified if the user belongs to the VO and VOMS assigns the proper access rights to the user
VOMS ( Virtual Organization Membership Service )
is a service that keeps track of the members belonging to a VO
supports MyProxy ( stored proxies )
grants users authorization to access the resources at VO Level
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

13. gLite Grid Security service : VOMS
31/07/2018
It extends the proxy certificates with info on VO membership, groups, roles (Administrator, normal user, student ), capabilities
Each VO has a database containing
group membership, roles and capabilities information for each user
User contacts VOMS server requesting
his authorization informations
If request is legitimate, VOMS server
sends signed authorization Info to the client
Client includes them in a proxy certificate
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

14. Workload Management
In order to submit a job to the infrastructure, users contact the Workload Management System (WMS)
This service that performs all tasks required to execute user’s jobs, without exposing the user to the complexity of the Grid
It is composed by different services
The Workload Manager (WM), accepts and satisfy requests for job management (Matchmaking) is the process of assigning the best available resource.
Logging & Bookeeping (LB) : keeps track of job execution in term of events: (Submitted, Running, Done,...)
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

15. Computing Element
Service that represents the computing resource localized at the site that is responsible to manage the queue of jobs to execute
Is generally composed by :
Grid Gate: which acts as a generic interface to the cluster
LocalResourceManagementSystem or batch-system: OpenPBS, Condor, Maui/Torque, LSF
Cluster: a collection of Worker Nodes, the nodes where the jobs are run
The CE may be used
by a Generic Client: when an end-user interacting directly with the Computing Element (Submission via CE), or
by the Workload Manager, which submits a given job to an appropriate CE found by the matchmaking process (Submission via WMS).
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

16. WMS :Computing Element
31/07/2018
Job management through the WMS provides many benefits compared to direct job submission to the CE
WMS can manage multiple CEs
is able to forward the job requests to
the one which better satisfies
a set of requirements
WMS provides a Global job tracking
facility using the LB service
WMS supports complex job types, which
can not be handled directly by the CEs
Dag, collections, parametric
WMS handles job failures
if a Job aborts, WMS automatically re-submit it to the matched CE
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

17. Information System and Monitoring
The Information System (IS) architecture used in gLite is based on the
Berkeley Database Information Index (BDII)
The information hierarchically stored via three modeling (The LDAP implementation of GLUE)
GRIS Stores information at resource level
BDII_Site Stores information at site level
BDII Stores information at VO level
VO Level
Site Level
Resource Level
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

18. Storage Element SE Services are at least:
Storage back-end (Disk Pool Manager, Mass Storage System, dCache)
Storage Resource Manager (SRM)
It is a Middleware interface application, that makes standard data management operations between SEs of different resource type, transparent to the end user.
These data management operations include:
File transfers
Space reservation
Renaming of files
File directory creation
Transfer services to transfer files in and out of a SE
Globus GridFTP protocol (gsiftp)
Globus-url-copy,
https, ftp protocols
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

19. LFC mantains mappings between LFN, GUID, SURL
LFC File Catalog
31/07/2018
LCG File catalog (LFC) simply keeps track of the location of a file in the Grid in a logical way so that a file will be accessible from anywhere
The identification of files on the storage element is done through the use of different identifiers:
LFN (Logical file name)
GUID (Grid unique identifier)
SimLinks
SURL (Site URL)
TURL (Transfer URL)
LFC mantains mappings between LFN, GUID, SURL
LFN Logical File Name: An alias created by the user to refer to some item
of data • e.g. “lfn:/grid/disk-pool/simulations/run01/outputdata.01”
 GUID Globally Unique Identifier: A non-human readable unique identifier for
an item data
• e.g. “guid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6
 SURL Storage URL or Physical Name: The location of an actual piece of
data on a storage system
• e.g. “srm://grid009.ct.infn.it/dpm/ct.infn.it/gilda/output10_1”
 TURL Transport URL: Describes access protocol for physical retrieval of
files
• e.g. “ rfio://lxshare0209.cern.ch//data/alice/ntuples.dat”
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

20. Grid Requirements Heterogeneous (OSes, Devs, Apps.)
31/07/2018
Grid Requirements
Heterogeneous (OSes, Devs, Apps.)
VO Resource Sharing (Management, Security and Accounting)
Resource Utilisation (Reservation, Metering, Monitoring and Logging)
Job Execution (VO access, QoS..)
Data Services (Integration, Provisioning, Cataloguing, Metadata)
Security (Authentication, Authorisation and Auditing)
Administrative Costs (Provisioning, Deployment and Configuration)
Scalability
Availability (Disaster Recovery, Fault Management)
QoS – quality of service
LCM – Local Credential Maping
WFM – Workflow management system
SLA – Service Layer Architecture
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

21. Grid Services and their interactions
Grid Access
User Iinterface
Info system
Security
MyProxy (Normal, Long term)
VOMS
Job submission
WMS
Computing Element
Worker Node
Data management
Catalogs
Storage elements
BDII
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting

22. Any questions …?
Ouafa Bentaleb, CERIST Rabat, Joint CHAIN/EUMEDGRID-Support/EPIKH ,Africa School Application Porting