Introduction to Grid Computing and the Trigrid/Cometa VL infrastructure Marcello IaconoManno PRIMO GRID TUTORIAL PER L’UNIVERSITA’ DI PALERMO Palermo, 10 Dicembre 2007

Fundamentals about Grid Computing Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 2

The Grid and the Internet It is considered the next step in the evolution of distributed computing Adopts the Internet as its communication infrastructure for information exchange It is considered equivalent to Internet as for exchanging of computational power and storage capacity Grid: the future of the Internet mobiles SuperComputing, Clusters PC Grid services Data Banks, Experiments workstation Fast Networks Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 3

Computing Grid The Grid idea The user: The society: Computing power is made available over the Internet Some computing farms produce computing power to be shared The society will provide for Grid facilities allowing the user to access to its Grid resources and providing for proper tools A user wants to access intensive computational power Now the user accesses to grid facilities as a grid user He/she comes to an agreement with some society that offers grid services The user: Does not need to know what stays beyond the user interface Can access to a massive amounts of computational power and storage capacity through a simple terminal The society: Can extend Grid facilities at any moment Manages the architecture of the Grid Defines policies and rules to access Grid resources Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 4

Why Computing Grids now? Because the amount of computational power needed by many applications is getting very huge Because the amount of data requires massive and complex distributed storage systems To make easier the cooperation of people and resources belonging to different organizations To access to particular instrumentation that is not easily reachable in a different way Because it is the next step in the evolution of distribution of computation Thousands of CPUs working at the same time on the same task From hundreds of Gigabytes to Petabytes (1015) produced by the same application. People of several organizations working together to achieve a common goal Because it cannot be moved or replicated or its cost is too much expensive. To create a marketplace of computational power and storage over the Internet Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 5

Who is interested in Grids? Research community, to carry out important results from experiments that involve many and many people and massive amounts of resources Enterprises that can have huge computation without the need for extending their current calculus and storage infrastructures Businesses, which can provide for computational power and data storage against a contract or for rental Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 6

Virtual Organizations Virtual Organization (VO) Is a collection of people and resources working together to achieve the same goal It is cross-domain (people and resources) One user Identified by his/her personal X.509 certificate issued by trusted Certification Authorities (CA) Can belong to more than one VO at the same time Does not require detailed knowledge of grid technologies to access to the Grid VO INFN Medical Institute Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 7

Virtual Laboratory A new way of cooperating in experiments A platform that allow scientists to work together in the same “Virtual” Laboratory Strictly correlated to Grids and Virtual Organizations Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 8

Properties of Grids Transparency Openness Heterogeneity Scalability The complexity of the Grid architecture is hidden to the final user The user has to be able to use a Grid as it was a unique virtual supercomputer Resources must be accessible setting their location apart Openness Each subcomponent of the Grid is accessible independently of the other components Heterogeneity Grids are composed by several and different resources Scalability Resources can be added and removed from the Grid dynamically Fault Tolerance Grids must be able to work even if a component fails or a system crashes Concurrency Different processes on different nodes must be able to work at the same time accessing to the same resources Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 9

Challenged Issues in Grids (i) Security Authentication and authorization of users Confidentiality and not repudiation Information Services Discovery and monitoring of Grid resources Checking for health-status of resources To be the base for any decision making processes File Management Creation, modification and deletion of files Replication of files to improve access performances Ability to access to files without the need to move them locally to the code Administration Systems to administer Grid resource respecting local administration policies Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 10

Challenged Issues in Grids (ii) Resource Brokering To schedule tasks across different resources To make optimal or suboptimal decisions To reserve (in the future) resources and network bandwidth Naming services To name resources in an unambiguous way in the Grid scope Friendly User Interfaces Because most of Grid users have nothing to do with computing science (physicians, chemistries, common users….) Graphical User Interfaces (GUIs) Grid Portals (very similar to classical Web Portals) Command Line Interfaces (CLIs) for experts Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 11

Integration with EGEE Enabling Grids for E-science (EGEE) Funded by European Union Working over the GEANT network (the network of the European research community) 70 partners of 27 countries Was born for conducting physics applications. Now extended to other fields, such as Earth Observation (EO), Bioinformatics and so on. http://www.eu-egee.org/ EGEE is providing a production quality grid infrastructure spanning more than 30 countries with over 150 sites to a myriad of applications from various scientific domains, including Earth Sciences, High Energy Physics, Bioinformatics and Astrophysics." Bob Jones (Project Director) Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 12

Integration with INFN Production Grid INFN Production Grid started in 1999 Working over the GARR (the Italian research communication network) Managed by INFN More than 20 sites, at the moment http://grid-it.cnaf.infn.it/ Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 13

Objectives of the Grid Projects in Sicily Create a “co-laboratory” in Sicily, both for scientific and industrial applications, built on top of a grid infrastructure Connect the Sicilian e-Infrastructure to those already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the “competitiveness” of e-Science and e-Industry “made in Sicily” Disseminate the “grid paradigm” through the organization of dedicated events and training courses Trigger/foster the creation of spin-offs in the ICT area in order to reduce the “brain drain” of brilliant young people to other parts of Italy and beyond Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 14

About the Trigrid Infrastructure Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 15

The TriGrid Activities (Work Packages) List of TriGrid VL Activities WP1 Design, set-up, and management of the Grid infrastructure WP2 Collection of requirements, tests of grid middleware and access to resources and to data (Middleware developments) WP3 Dissemination and Training WP4 Applications’ porting and users’ support Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 16

The TriGrid Partners Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 17

The Trigrid Infrastructure Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 18

Infrastruttura di TriGrid Grid site in Catania: INAF, INFN, DMI, DIIT About 100 machines dual CPU 32/64 bit with a computing power of about 300 kSpecInt2000. 30 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5 About 20 workstations Grid site in Messina: Facoltà di Ingegneria and CECUM About 80 machines dual CPU 32/64 bit with a computing power of about 240 kSpecInt2000. 20 TBs of storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5 Grid site in Palermo: Dipartimento di Fisica e Tecnologie Relative (DIFTER) About 30 machines dual CPU 32/64 bit with a computing power of about 00 kSpecInt2000. 10 TBs di storage EIDE-to-SCSI or FC-to-SATA in RAID 0,1,5 Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 19

The PI2S2 Project (http://www.pi2s2.it) Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 20 20

The COMETA Consortium (http://www.consorzio-cometa.it) Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 21 21

~300 FTEs (2/3 new hired people) The PI2S2 e-Infrastructure 1500+ CPUs 250+ TB of storage ~11,5 M€ (01/01/2006-31/12/2008) ~300 FTEs (2/3 new hired people) Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 22 22

Second tender under way First tender for infrastructure completed! (Running since Summer 2007) INFN-CT (expansion w.r.t. TriGrid) Second tender under way UNI-ME UNI-PA Padova, V Workshop INFN Grid, 18.12.2006 Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 23 23

Computing, Networking, and Storage (1/2) 8 IBM BladeCenter H enclosures 84 IBM LS21 “blades” 336 cores AMD Opteron 2218 rev. F 772 GB of RAM (2 GB/core) 0.55 MSpecInt2000 0.66 MSpecFP2000 More than 6 kSpec(Int/FP)Rate ~ 48.8 mW/SpecInt2000 at full load ! G-Ethernet service network CISCO Topspin Infiniband-4X additional low-latency network for HPC applications LSF 6.1 HPC included ! Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 24

Computing, Networking, and Storage (2/2) 4 IBM DS4200 Storage Systems (sites 1, 2, 3, and 6) FC-2-SATA technology 136 500-GB disks 68 TB of storage (raw) in total Expandability up to 0.45 PB GPFS distributed/parallel file sytem included ! Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 25

Photo Gallery CECUM Messina (Trigrid) INFN Catania (Trigrid + Cometa) Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 26

The NEW COMETA WebSite (http://www.pi2s2.it/) Recently updated Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 27

The COMETA AUP Usage policies in PI2S2 Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 28

>150 users already registered ! The “Cometa” Virtual Organization (https://voms.ct.infn.it:8443/voms/cometa/) Authorization for PI2S2 >150 users already registered ! Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 29

The PI2S2 User Interface on VM (http://www.pi2s2.it/tc/download.php) Already usable on the TriGrid VL e-Infrastructure Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 30

>50 questionnaires filled so far The PI2S2 Application Questionnaire (http://www.consorzio-cometa.it/questionario/quest.php) http://www.pi2s2.it/tc/show_quest_2006.php >50 questionnaires filled so far Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 31

Statistics of Application Questionnaires Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 32

Industrial Applications Applications already proposed Industrial Applications IR&T engineering Cultural heritage Numidia srl Fluid dynamic simulation in car engines Chemical, thermo dynamic, electromagnetic simulations SCIRE – FIAT auto Simulation of aerodynamics problems Fluid dynamics optimization Italcompany e Microsol Using the Fluent software tool in the grid Seasoft Document management and workflow Inquadro srl Software performance analysis Grid for multimedia streaming Hitec2000 srl Simulation of critical working conditions Act. Areas of interest 4.5 Engineering 4.6 Physics and astrophysics 4.7 Earth science 4.8 Bio-Informatics 4.9 Chemistry Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 33 33

Relationships with Private Companies Official agreement signed with ORACLE for the S-Sicilia Project aiming at set-up a Competence Centre in Sicily for accounting on the Grid in collaboration with the Consorzio COMETA and a University of Messina spin-off SME. Agreement finalized with MICROSOFT: for the porting of gLite on Windows including the integration of MS CCS 2003 as LRMS; for the adoption of Conference-XP as a platform to be used for/during Grid training Other agreements are foreseen in the next future with Ferrari, FIAT, and Technipetrol. Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 34

Application Areas (1/2) Physical Science Applications GryPhiN, http://www.gryphin.org/ Particle Physics DataGrid (PPDG), http://grid.fnal.gov/ppdg/ GridPP, http://www.gridpp.ac.uk/ AstroGrid, http://www.astrogrid.org/ Life Science Applications Protein Data Bank (PDB), http://www.rcsb.org/pdb/Welcome.do Biomedical Informatics Research Network (BIRN), http://www.nbirn.net/ Telemicroscopy, http://ncmir.ucsd.edu/ myGrid, http://www.mygrid.org.uk/ Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 35

Application Areas (2/2) Engineering Oriented Applications NASA Information Power Grid (IPG), http://www.ipg.nasa.gov/ Grid Enabled Optimization and Design Search for Engineering (GEODISE), http://www.geodise.org/ Commercial Applications Butterfly Grid, http://www.butterfly.net/ Everquest, http://www.everquest.com/ E-Utility ClimatePrediction experiment, http://www.climateprediction.net/ Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 36

References TriGrid Homepage Consorzio COMETA Homepage http://www.trigrid.it Consorzio COMETA Homepage http://www.consorzio-cometa.it INFN Production Grid http://grid-it.cnaf.infn.it/ EGEE http://www.eu-egee.org/ Palermo, Primo Grid Tutorial per l’Università di Palermo, 10 Dicembre 2007 37