1. Introduction to Grid Computing and the Trigrid VL infrastructure
Fabio Scibilia
INFN – Catania
, Messina

2. Fundamentals of Grid Computing
Messina, ,

3. The Grid and the Internet
It is considered the next step in the evolution of distributed computing
Adopts the Internet as communication infrastructure for exchanging information
It is considered as the equivalent of the Internet in terms of exchanging of computational power
Messina, ,

4. Computing Grid The Grid idea The user: The society:
Computing power is made available over the Internet
Some computing farms produce computing power to be shared
The society will provide for grid facilities allowing the user to access to its grid resources and providing for proper tools
One user wants to access to intensive computational power
Now the user accesses to grid facilities as a grid user
He/she comes to an agreement with some society that offers grid services
The user:
Does not need to know what stays beyond the user interface
Can access to a massive amounts of computational power through a simple terminal
The society:
Can extend grid facilities at any moment
Manages the architecture of the grid
Defines policies and rules for accessing to grid resources
Messina, ,

5. Why Computing Grids now?
Because the amount of computational power needed by many applications is getting very huge
Because the amount of data requires massive and complex distributed storage systems
To make easier the cooperation of people and resources belonging to different organizations
To access to particular instrumentation that is not easily reachable in a different way
Because it is the next step in the evolution of distribution of computation
Thousands of CPUs working at the same time on the same task
From hundreds of Gigabytes to Petabytes (1015) produced by the same application.
People of several organizations working together to achieve a common goal
Because it cannot be moved or replicated or its cost is too much expensive.
To create a marketplace of computational power and storage over the Internet
Messina, ,

6. Who is interested in Grids?
Research community, to carry out important results from experiments that involve many and many people and massive amounts of resources
Enterprises that can have huge computation without the need for extending their current informatic infrastructures
Businesses, which can provide for computational power and data storage against a contract or for rental
Messina, ,

7. Virtual Organizations
Virtual Organization (VO)
Is a collection of people and resources working together to achieve the same goal
It is cross-domain (people and resources)
One user
Identified by his/her personal X.509 certificate issued by trusted Certification Authorities (CA)
Can belong to more than one VO at the same time
Does not require detailed knowledge of grid technologies to access to the Grid VO
INFN
Medical
Institure
Messina, ,

8. Virtual Laboratory A new way of cooperating in experiments
A platform that allow scientists to work together in the same “Virtual” Laboratory
Strictly correlated to Grids and Virtual Organizations
Messina, ,

9. Properties of Grids Transparency Openness Heterogeneity Scalability
The complexity of the Grid architecture is hidden to the final user
The user has to be able to use a Grid as it was a unique virtual supercomputer
Resources must be accessible setting their location apart
Openness
Each subcomponent of the Grid is accessible independently of the other components
Heterogeneity
Grids are composed by several and different resources
Scalability
Resources can be added and removed from the Grid dynamically
Fault Tolerance
Grids must be able to work even if a component fails or a system crashes
Concurrency
Different processes on different nodes must be able to work at the same time accessing to the same resources
Messina, ,

10. Challenged Issues in Grids (i)
Security
Authentication and authorization of users
Confidentiality and not repudiation
Information Services
Discovery and monitoring of Grid resources
Checking for health-status of resources
To be the base for any decision making processes
File Management
Creation, modification and deletion of files
Replication of files to improve access performances
Ability to access to files without the need to move them locally to the code
Administration
Systems to administer Grid resource respecting local administration policies
Messina, ,

11. Challenged Issues in Grids (ii)
Resource Brokering
To schedule tasks across different resources
To make optimal or suboptimal decisions
To reserve (in the future) resources and network bandwidth
Naming services
To name resources in an unambiguous way in the Grid scope
Friendly User Interfaces
Because most of Grid users have nothing to do with computing science (physicians, chemistries, videogame players . . .)
Graphical User Interfaces (GUIs)
Grid Portals (very similar to classical Web Portals)
Command Line Interfaces (CLIs) for experts
Messina, ,

12. On the Infrastructure
Messina, ,

13. The Trigrid Infrastructure
Messina, ,

14. Infrastruttura di TriGrid
Grid site in Catania:
INAF, INFN, DMI, DIIT
About 100 machines dual CPU 32/64 bit with a computing power of about 300 kSpecInt2000.
30 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
About 20 workstations
Grid site in Messina:
Facoltà di Ingegneria and CECUM
About 80 machines dual CPU 32/64 bit with a computing power of about 240 kSpecInt2000.
20 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
Grid site in Palermo:
Dipartimento di Fisica e Tecnologie Relative (DIFTER)
About 30 machines dual CPU 32/64 bit with a computing power of about 00 kSpecInt2000.
10 TBs di storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
Messina, ,

15. Integration with INFN Production Grid
INFN Production Grid started in 1999
Working over the GARR (the Italian research communication network)
Managed by INFN
More than 20 sites, at the moment
Messina, ,

16. Integration with EGEE Enabling Grids for E-science (EGEE)
Funded by European Union
Working over the GEANT network (the network of the European research community)
70 partners of 27 countries
Was born for conducting physics applications. Now extended to other fields, such as Earth Observation (EO), Bioinformatics and so on.
EGEE is providing a production quality grid infrastructure spanning more than 30 countries with over 150 sites to a myriad of applications from various scientific domains, including Earth Sciences, High Energy Physics, Bioinformatics and Astrophysics."
Bob Jones (Project Director)
Messina, ,

17. Application Areas (1/2) Physicical Science Applications
GryPhiN,
Particle Physics DataGrid (PPDG),
GridPP,
AstroGrid,
Life Science Applications
Protein Data Bank (PDB),
Biomedical Informatics Research Network (BIRN),
Telemicroscopy,
myGrid,
Messina, ,

18. Application Areas (2/2) Engineering Oriented Applications
NASA Information Power Grid (IPG),
Grid Enabled Optimization and Design Search for Engineering (GEODISE),
Commercial Applications
Butterfly Grid,
Everquest,
E-Utility
ClimatePrediction experiment,
Messina, ,

19. The Architecture and the Middleware
Messina, ,

20. The Globus Alliance Globus Alliance The Globus Toolkit (GT)
It is a community of people of different organizations involved in the design and development of Grid Technologies
Università dell’Illinois, Argonne National Laboratory, University of Edinburgh, EPCC, etc…
The Globus Toolkit (GT)
It is a standard di fatto
Is is a set of services
Fourth release, at the moment
Involves Web Services Interfaces
The Global Grid Forum (GGF)
It is a forum for Grid researchers
Its goal is to define standards and protocols for Grids
It is organized in Work Groups (WGs)
Messina, ,

21. The Globus hourglass model
The hourglass model
Reference model used by Globus designers
Levels of services
Local Operating Systems: Management of several types of resources, local scheduling systems, monitoring and so on.
Core Globus Services: Services to create and manage the Grid infrastructure
Diverse Global Services: Grid based applications
Messina, ,

22. Layers Fabric Connectivity Resource Collective Application
Manages resources
Connectivity
Security Context (authentication, authorization)
Single Sign On (SSO)
Resource
Allocation, reservation and monitoring of resources
Data movement
Collection of information on resources
Collective
Discovery of resources
Data cataloguing and replication
Workflow management
Application
User applications
Tools and interfaces
Application
Collective
Resource
Connectivity
Fabric
Messina, ,

23. Globus Services
Messina, ,

24. Architecture Overview
Messina, ,

25. The Single Sign On (SSO)
It is a way to sign in to the Grid just one time and access to all services you need
It is done via proper command lines of the User Interface and providing a valid certiicate
A user submits his certificate to the VOMS server
VO Membership Service (VOMS)
Issues proxies to users with VOMS extensions
Assigns groups/roles/capabilities to the users
Verifies user credentials (ceritificate)
Messina, ,

26. Computing Element (CE)
Local Resource Manager (LRMS)
It is NOT a Grid component
Local administration
Dispatches job across Worker Nodes according to local policies
Accessed by local users
Computing Element
Service oriented
Exports computing power of the LRMS to grid users
Maps grid users to local users
Wraps a “grid job” into a “local job”
Gatekeeper
Access point of the CE
Manages one or more job queues
Transparent interface to the LRMS
Worker Nodes
More than one (farm)
To execute grid jobs, a specific environment must be set
Messina, ,

27. Storage Element (SE) Mass Storage System (MSS)
It is NOT a grid component
Manages the local storage device
Hard disks, digital tapes
Storage Resource Manager (SRM)
Is the grid compliant interface to access to the storage system
Maps grid users to local users
Access and control to the MSS
Grid FTP compliant daemons
Grid compliant daemons for uploading/downloading of grid files
On top of the local MSS
Protocols: gsiftp, rfio, dcap
Grid client
Any user/process with a valid proxy
Messina, ,

28. Data Management LHC File Catalogue (LFC) Logical File Name (LFN)
Hierarchical file catalogue
Each VO has its own catalogue
Logical File Name (LFN)
The name of a file within the LFC
To each LFN correspond one or more replicas stored on different SEs
lfn://grid/cometa/file.txt
Storage URL (SURL)
The name of a replica within a SE
To each SURL correspond more TURLs
srm://se1.cometa.it/some/path/file.txt
Transport URL (TURL)
Is the name of the physical file plus the grid FTP protocol to access to it
gsiftp://se1_disk2.cometa.it/local/path/file1.txt
Messina, ,

29. Information System Globus Meta Directory Server (MDS)
Hierarchical, based on LDAP servers
The user can query the system at every level
The higher the information is in the tree, the older it is
Grid Resource Information Service (GRIS)
Collects info on that resource
Static or dynamic info
Uses sensors
Grid Index Information Service (GIIS)
One for each site
Collects info from above GRISes
Caches info according to its validity time
Queries above GRISes or GIISes whether needed
Berkeley Database Information Index (BDII)
Same behaviour but more stable
Messina, ,

30. Workload Management System (1/4)
The grid user describes a job via a Job Description Language (JDL) document.
Some input files (Input Sandbox) can be attached to the JDL doc.
The grid user submits the JDL job using the command line interface and waits for reply.
The Resource Broker gets and stores the JDL document together with attached input files.
The just generated jobId is sent back to the user to refer to that job univocally in the future
Messina, ,

31. Workload Management System (2/4)
The Resources Broker executes a special algorithm (matchmaking) and chooses a Computing Element according to best-fit rules
The job is handed to the chosen CE together with the Input SandBox
The Computing Element accepts the job and queues it.
The job starts execution over the Local Resources Management System (LRMS)
Messina, ,

32. Workload Management System (3/4)
When the job terminates, the produced output is sent back to the Resource Broker
The Resource Broker gets the results and the Output Sandbox and stores them in the local repository
At the same time, the Computing Element notifies the Logging & Bookkeeping
Now the job output is available on the Resource Broker
Messina, ,

33. Workload Management System (4/4)
The User queries the Logging and Bookkeping to have a look on his/her jobs and realizes that the job has terminated.
The User downloads the Output Sandbox from the Resource Broker.
The Resource Broker clears all no more needed info from its repository to save space.
The job life cycle has terminated !!!
Messina, ,

34. References TriGrid Homepage Consorzio COMETA Homepage
Consorzio COMETA Homepage
INFN Production Grid
EGEE
Messina, ,