1. www.epikh.eu The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Riccardo Bruno (riccardo.bruno@ct.infn.it)riccardo.bruno@ct.infn.it COMETA Consortium Joint CHAIN/EUMEDGRID-Support/EPIKH School for Application Porting to Science Gateways Amman, 28.11.2011 Introduction to Grid distributed environments

2. Today’ research 2

3. The Grid A GRID is a distributed computing and storage infrastructure – spanning several administrative domains - allowing sharing of resources in a coordinated manner by a set of homogeneous users organized within Virtual Organizations A GRID provides single sign on to access a large variety of resources and an added value with respect to the bare sum of its components GRIDS are the key enabler of e-Science

4. e-Science – e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it e-Infrastructure = Networks + Grids.. + Operations, Support, Training… + Data centres, archives, instruments… Networks interconnect resources Grids enable flexible usage of interconnected resources: Distributed computing across different administrative domains

5. Overview Location, Meeting title, dd.mm.yyyy 5 Virtual organisations e-Science 5 Applications Data Instruments e-InfrastructureNetwork Computational and Storage resources

6. e-Infrastructure Location, Meeting title, dd.mm.yyyy 6 National Research and Education Networks National Grid Initiatives International Grid Initiatives

7. Location, Meeting title, dd.mm.yyyy 7 The Global Grid EELA OSG TeraGrid NAREGI EUMedGrid BalticGrid SEE-Grid EUIndiaGrid EUAsiaGrid EUChinaGrid DEISA EGI EU fundend Non EU

8. A GRID Metaphore Location, Meeting title, dd.mm.yyyy 8 Using a PC or a work station – Login using username & password (“Authentication”) – Owning some rights (“Authorisation”) – Run programs or jobs – manage files: create, read, list Components are interconnected by a bus You are using the operating system There is only one administrative domain Using the GRID – Login using digital credentials (“Authentication”) – Owning some rights (“Authorisation”) – Run programs or jobs – manage files: create, read, list Components are interconnected by internet You are using the GRID middleware There are many administrative domain

9. GRID Requirements 9 Heterogeneous (OSes, Devs, Apps.) VO Resource Sharing (Management, Security and Accounting) Resource Utilisation (Reservation, Metering, Monitoring and Logging) Job Execution (VO access, QoS, LCM, WFM, SLA) Data Services (Integration, Provisioning, Cataloguing, Metadata) Security (Authentication, Authorisation and Auditing) Administrative Costs (Provisioning, Deployment and Configuration) Scalability Availability (Disaster Recovery, Fault Management) Specific requirements (EGI: HEP, BioMed,…)

10. Grid components Location, Meeting title, dd.mm.yyyy 10 Authorization and Authentication Users/Host/Robot certificates CA/RA concepts Proxy certificates IGTF Authorization providers (VOMS)

11. AuthN/AuthZ Location, Meeting title, dd.mm.yyyy 11 – Resources are generally owned by VOs that allow access to them based on the “role” of the user and/or its belonging to a specific “group” – Every user, server or service is identified by means of a digital certificate (X509) certifying its identity (Authentication) – Access to resources takes place in a safe way (integrity, confidentiality), using a granularity which can go at the single user level – Each VOs associate resource access rights accordingly to the user “group” and “role” (Authorization) – Authorization granularity can go at the single user level

12. Certificate issuing User certificates 1.The user will be identified by a Registration Authority (RA) 2.The RA releases a PIN 3.The user asks to the CA for a personal certificate using the PIN 4.The request acknowledged by mail exchanges 5.The user receives the certificate Host certificates –They are linked to the ‘hostname’ of the server Robot certificates –Certificates securely stored into HW devices protected by PIN –Mostly used by GRID service providers –Not all CAs are supporting yet Robot certificates Location, Meeting title, dd.mm.yyyy 12

13. Certificate Proxies Personal certificates are not directly exposed Most of Grids use Temporary certificates (proxies) Normal lifetime 12h The Original Certificate will be not exposed Proxies are certificates digitally signed by the original certificate or another proxy (delegation) GRID Services may operate on the user behaf (SSO) Proxies may be securely stored (i.e. Globus and gLite: MyProxy) Stored proxies may be used to renew other proxies 13 … CA Self signed Digitally Signed by CA Digitally Signed by User Cert Digitally Signed by Prev. Cert

14. IGTF Most of GRID infrastructure accept only certificates released by accredited Certification Authorities The International Grid Trust Federation collects all accredited Cas Generation of CAs encouraged while developing NGIs Location, Meeting title, dd.mm.yyyy 14

15. Authorization providers VOs own phisical resources GRID Authotization services guarantee the correct user access rights mapping users to configured ‘pool accounts Most of Grid Infrastructures use VOMS Virtual Organization Membership Service –Allow the creation of Groups of users –Allow the creation of different roles among existing groups Before to access VOs resources users must request the membership and agree the AUP A GRID site may support one or more Vos VOMS extends Proxy certificate with further information related to –User Group –User Role –VO resource access expiration Location, Meeting title, dd.mm.yyyy 15

16. Grid components Location, Meeting title, dd.mm.yyyy 16 The User Interface Grid Portals APIs Science Gateways

17. User Interface Most of GRID infrastructure provides CLI Unix/Windows/Mac machine whith client applications installed User account created after subscribing a VO User interfaces could be –Centralized servers (many users) –Virtualized machines (single/low number of users) –Software packages (single user) High level user interfaces (GUI) –Applications offering graphic front-end to existing UI client applications Location, Meeting title, dd.mm.yyyy 17

18. Grid portals Web front end to GRID capabilities Offering a generic interface to GRID resources Need user certificate configured into the web browser Location, Meeting title, dd.mm.yyyy 18 Genius Web Portal P-GRADE

19. Science Gateways Community-developed set of tools, applications, and data that is integrated via a portal or a suite of applications –No general purpose GRID interaction –No longer requestet to deal with digital certificates –Just need to belong to a Community though an Identity Federation Location, Meeting title, dd.mm.yyyy 19

20. Grid components Location, Meeting title, dd.mm.yyyy 20 GRID Information system (GLUE) Berkley Database Info. Index (BDII)

21. GLUE schema Most of GRID infrastructures uses the GLUE schema to represent resource information GLUE Schema is an abstract modeling for Grid resources developed by the Open Grid Forum (OGF) There are many implementation of the GLUE schema –LDAP, RDBMS, XML, … The most famous implementation of the GLUE schema is the BDII Location, Meeting title, dd.mm.yyyy 21 UML representation

22. 22 To change: View -> Header and Footer Berkeley Database Information Index (BDII) (The LDAP implementation of GLUE) The information hierarchically stored via tree modeling GRISStores information at resource level Site BDII/GIIS (deprecated) Stores information at site level BDIIStores information at VO level VO Level Site Level Resource Level Information System and Monitoring

23. Grid components Location, Meeting title, dd.mm.yyyy 23 GRID Job Workflow Resource Manager Computing Element

24. Overview of a GRID job Location, Meeting title, dd.mm.yyyy 24 … Job output SUBMITTE D WAIT READY SCHEDULE D RUNNING DONE (OK) DONE (Failed) CLEARED CANCELLE D ABORTED

25. 25 Set of middleware components responsible of distribution and management of jobs across Grid resources. Two main components Workload Manager Accepts and satisfy requests for job management. (Matchmaking) is the process of assigning the best available resource. Logging & Bookeeping keeps track of job execution in term of events (Submitted, Running, Done, Abort) Resource Manager

26. 26 Service that represents the computing resource that is responsible of to manage the queue of jobs to execute The CE may be used by a Generic Client: an end-user interacting directly with the Computing Element, or by the Resource Manager, which submits a given job to an appropriate CE found by the matchmaking process. Two job submission models : PUSH (Eager Scheduling) (jobs pushed to CE), PULL (Lazy Scheduling) (jobs received when CE has free slots) Computing Element

27. Grid components Location, Meeting title, dd.mm.yyyy 27 Storage Elements File Catalog

28. 28 Storage Element services Storage back-end (Drivers and Hardware) Abstraction Layer (SRM) ( Interface to manage the specific storage solution : dpm, rfio, …) Transfer service ( Protocols: GridFTP ( gsiftp ), glubus-url-copy, …) Native POSIX like file I/O API (GFAL) Auxiliary Accounting and Logging services Data are stored on Disk Pool Servers or Mass Storage Systems File replicas Reliability, Geographic coverage, Fault tollerance, Network latences Storage Element

29. 29 Maps SE files with a human readable ‘filename’ LFN (Logical file name) GUID (Grid unique identifier) SimLinks SURL (Site URL) TURL (Transfer URL) File Catalog

30. 30 Questions …