1. Introduction to Grid Computing and the Trigrid/Cometa VL infrastructure
Marcello IaconoManno
PRIMO GRID TUTORIAL PER
L’UNIVERSITA’ DI PALERMO
Palermo, 10 Dicembre 2007

2. Fundamentals about Grid Computing
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

3. The Grid and the Internet
It is considered the next step in the evolution of distributed computing
Adopts the Internet as its communication infrastructure for information exchange
It is considered equivalent to Internet as for exchanging of computational power and storage capacity
Grid: the future of the Internet
mobiles
SuperComputing, Clusters PC
Grid services
Data Banks, Experiments
workstation
Fast Networks
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

4. Computing Grid The Grid idea The user: The society:
Computing power is made available over the Internet
Some computing farms produce computing power to be shared
The society will provide for Grid facilities allowing the user to access to its Grid resources and providing for proper tools
A user wants to access intensive computational power
Now the user accesses to grid facilities as a grid user
He/she comes to an agreement with some society that offers grid services
The user:
Does not need to know what stays beyond the user interface
Can access to a massive amounts of computational power and storage capacity through a simple terminal
The society:
Can extend Grid facilities at any moment
Manages the architecture of the Grid
Defines policies and rules to access Grid resources
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

5. Why Computing Grids now?
Because the amount of computational power needed by many applications is getting very huge
Because the amount of data requires massive and complex distributed storage systems
To make easier the cooperation of people and resources belonging to different organizations
To access to particular instrumentation that is not easily reachable in a different way
Because it is the next step in the evolution of distribution of computation
Thousands of CPUs working at the same time on the same task
From hundreds of Gigabytes to Petabytes (1015) produced by the same application.
People of several organizations working together to achieve a common goal
Because it cannot be moved or replicated or its cost is too much expensive.
To create a marketplace of computational power and storage over the Internet
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

6. Who is interested in Grids?
Research community, to carry out important results from experiments that involve many and many people and massive amounts of resources
Enterprises that can have huge computation without the need for extending their current calculus and storage infrastructures
Businesses, which can provide for computational power and data storage against a contract or for rental
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

7. Virtual Organizations
Virtual Organization (VO)
Is a collection of people and resources working together to achieve the same goal
It is cross-domain (people and resources)
One user
Identified by his/her personal X.509 certificate issued by trusted Certification Authorities (CA)
Can belong to more than one VO at the same time
Does not require detailed knowledge of grid technologies to access to the Grid VO
INFN
Medical
Institute
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

8. Virtual Laboratory A new way of cooperating in experiments
A platform that allow scientists to work together in the same “Virtual” Laboratory
Strictly correlated to Grids and Virtual Organizations
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

9. Properties of Grids Transparency Openness Heterogeneity Scalability
The complexity of the Grid architecture is hidden to the final user
The user has to be able to use a Grid as it was a unique virtual supercomputer
Resources must be accessible setting their location apart
Openness
Each subcomponent of the Grid is accessible independently of the other components
Heterogeneity
Grids are composed by several and different resources
Scalability
Resources can be added and removed from the Grid dynamically
Fault Tolerance
Grids must be able to work even if a component fails or a system crashes
Concurrency
Different processes on different nodes must be able to work at the same time accessing to the same resources
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

10. Challenged Issues in Grids (i)
Security
Authentication and authorization of users
Confidentiality and not repudiation
Information Services
Discovery and monitoring of Grid resources
Checking for health-status of resources
To be the base for any decision making processes
File Management
Creation, modification and deletion of files
Replication of files to improve access performances
Ability to access to files without the need to move them locally to the code
Administration
Systems to administer Grid resource respecting local administration policies
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

11. Challenged Issues in Grids (ii)
Resource Brokering
To schedule tasks across different resources
To make optimal or suboptimal decisions
To reserve (in the future) resources and network bandwidth
Naming services
To name resources in an unambiguous way in the Grid scope
Friendly User Interfaces
Because most of Grid users have nothing to do with computing science (physicians, chemistries, common users….)
Graphical User Interfaces (GUIs)
Grid Portals (very similar to classical Web Portals)
Command Line Interfaces (CLIs) for experts
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

12. Integration with EGEE Enabling Grids for E-science (EGEE)
Funded by European Union
Working over the GEANT network (the network of the European research community)
70 partners of 27 countries
Was born for conducting physics applications. Now extended to other fields, such as Earth Observation (EO), Bioinformatics and so on.
EGEE is providing a production quality grid infrastructure spanning more than 30 countries with over 150 sites to a myriad of applications from various scientific domains, including Earth Sciences, High Energy Physics, Bioinformatics and Astrophysics."
Bob Jones (Project Director)
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

13. Integration with INFN Production Grid
INFN Production Grid started in 1999
Working over the GARR (the Italian research communication network)
Managed by INFN
More than 20 sites, at the moment
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

14. Objectives of the Grid Projects in Sicily
Create a “co-laboratory” in Sicily, both for scientific and industrial applications, built on top of a grid infrastructure
Connect the Sicilian e-Infrastructure to those already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the “competitiveness” of e-Science and e-Industry “made in Sicily”
Disseminate the “grid paradigm” through the organization of dedicated events and training courses
Trigger/foster the creation of spin-offs in the ICT area in order to reduce the “brain drain” of brilliant young people to other parts of Italy and beyond
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

15. About the Trigrid Infrastructure
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

16. The TriGrid Activities (Work Packages)
List of TriGrid VL Activities
WP1
Design, set-up, and management of the
Grid infrastructure
WP2
Collection of requirements, tests of grid
middleware and access to resources and to data
(Middleware developments)
WP3
Dissemination and Training
WP4
Applications’ porting and users’ support
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

17. The TriGrid Partners
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

18. The Trigrid Infrastructure
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

19. Infrastruttura di TriGrid
Grid site in Catania:
INAF, INFN, DMI, DIIT
About 100 machines dual CPU 32/64 bit with a computing power of about 300 kSpecInt2000.
30 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
About 20 workstations
Grid site in Messina:
Facoltà di Ingegneria and CECUM
About 80 machines dual CPU 32/64 bit with a computing power of about 240 kSpecInt2000.
20 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
Grid site in Palermo:
Dipartimento di Fisica e Tecnologie Relative (DIFTER)
About 30 machines dual CPU 32/64 bit with a computing power of about 00 kSpecInt2000.
10 TBs di storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

20. The PI2S2 Project (http://www.pi2s2.it)
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 20

21. The COMETA Consortium (http://www.consorzio-cometa.it)
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 21

22. ~300 FTEs (2/3 new hired people)
The PI2S2 e-Infrastructure
1500+ CPUs
250+ TB of storage
~11,5 M€ (01/01/ /12/2008)
~300 FTEs (2/3 new hired people)
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 22

23. Second tender under way
First tender for infrastructure completed!
(Running since Summer 2007)
INFN-CT
(expansion w.r.t. TriGrid)
Second tender under way
UNI-ME
UNI-PA
Padova, V Workshop INFN Grid,
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 23

24. Computing, Networking, and Storage (1/2)
8 IBM BladeCenter H enclosures
84 IBM LS21 “blades”
336 cores AMD Opteron 2218 rev. F
772 GB of RAM (2 GB/core)
0.55 MSpecInt2000
0.66 MSpecFP2000
More than 6 kSpec(Int/FP)Rate
~ 48.8 mW/SpecInt2000 at full load !
G-Ethernet service network
CISCO Topspin Infiniband-4X additional low-latency network for HPC applications
LSF 6.1 HPC included !
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

25. Computing, Networking, and Storage (2/2)
4 IBM DS4200 Storage Systems (sites 1, 2, 3, and 6)
FC-2-SATA technology
GB disks
68 TB of storage (raw) in total
Expandability up to 0.45 PB
GPFS distributed/parallel file sytem included !
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

26. Photo Gallery CECUM Messina (Trigrid) INFN Catania (Trigrid + Cometa)
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

27. The NEW COMETA WebSite (http://www.pi2s2.it/)
Recently updated
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

28. The COMETA AUP Usage policies in PI2S2
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

29. >150 users already registered !
The “Cometa” Virtual Organization (
Authorization for PI2S2
>150 users already registered !
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

30. The PI2S2 User Interface on VM (http://www.pi2s2.it/tc/download.php)
Already usable on the TriGrid VL e-Infrastructure
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

31. >50 questionnaires filled so far
The PI2S2 Application Questionnaire (
>50 questionnaires filled so far
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

32. Statistics of Application Questionnaires
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

33. Industrial Applications
Applications already proposed
Industrial Applications
IR&T engineering
Cultural heritage
Numidia srl
Fluid dynamic simulation in car engines
Chemical, thermo dynamic, electromagnetic simulations
SCIRE – FIAT auto
Simulation of aerodynamics problems
Fluid dynamics optimization
Italcompany e Microsol
Using the Fluent software tool in the grid
Seasoft
Document management and workflow
Inquadro srl
Software performance analysis
Grid for multimedia streaming
Hitec2000 srl
Simulation of critical working conditions
Act.
Areas of interest
4.5
Engineering
4.6
Physics and astrophysics
4.7
Earth science
4.8
Bio-Informatics
4.9
Chemistry
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 33

34. Relationships with Private Companies
Official agreement signed with ORACLE for the S-Sicilia Project aiming at set-up a Competence Centre in Sicily for accounting on the Grid in collaboration with the Consorzio COMETA and a University of Messina spin-off SME.
Agreement finalized with MICROSOFT:
for the porting of gLite on Windows including the integration of MS CCS 2003 as LRMS;
for the adoption of Conference-XP as a platform to be used for/during Grid training
Other agreements are foreseen in the next future with Ferrari, FIAT, and Technipetrol.
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

35. Application Areas (1/2) Physical Science Applications
GryPhiN,
Particle Physics DataGrid (PPDG),
GridPP,
AstroGrid,
Life Science Applications
Protein Data Bank (PDB),
Biomedical Informatics Research Network (BIRN),
Telemicroscopy,
myGrid,
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

36. Application Areas (2/2) Engineering Oriented Applications
NASA Information Power Grid (IPG),
Grid Enabled Optimization and Design Search for Engineering (GEODISE),
Commercial Applications
Butterfly Grid,
Everquest,
E-Utility
ClimatePrediction experiment,
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre

37. References TriGrid Homepage Consorzio COMETA Homepage
Consorzio COMETA Homepage
INFN Production Grid
EGEE
Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre