1 The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) GRID Science Gateway Riccardo Rotondo Consortium GARR Tutorial for Grid Application Porting on Grid Science Gateway Beijing, Beijing, Asia 4, Riccardo Rotondo 1

2 Outline Grid Usage Science Gateway The e-Collaboration environment –The old paradigm –Science Gateway architecture –The new “brick” approach Use Cases: –The DECIDE Science Gateway –The INDICATE e-Culture Science Gateway –The CHAIN Science Gateway –The Gilda t-Infrastructure Portal –The RICeVI Portal Conclusions Beijing, Asia 4, Riccardo Rotondo 2

3 GRID at world scale Beijing, Asia 4, Riccardo Rotondo >340 sites in 57 countries ~337,000 CPU cores >220 PB of storage ~1 million jobs/day ~20,000 users in ~280 VOs 3

4 But…using Grids is not straightforward  Beijing, Asia 4, Riccardo Rotondo Type = "Job"; JobType = "MPICH"; MPIType = "MVAPICH2_PGI706"; CpuNumber = 16; MPIGranularity = 4; Executable = "flash2"; StdOutput = "mpi.out"; StdError = "mpi.err"; InputSandbox = {"watchdog.sh","mpi.pre.sh","mpi.post.sh","flash.par","summers_den_ 1e0.rates","flash2"}; OutputSandbox = {"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"}; Requirements = (other.GlueCEUniqueId == "unipa-ce- 01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc"); RetryCount = 0; $ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy/CN=proxy issuer : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy identity : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy type : proxy strength : 1024 bits path : /tmp/x509up_u512 timeleft : 500:38:44 === VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft : 23:59:33 uri : voms.ct.infn.it:15001 $ glite-wms-job-status XTAIGAg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : XTAIGAg Current Status: Done (Success) Logged Reason(s): - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda Submitted: Tue Jun 29 15:34: CEST ************************************************************* CLI Scripts JDL echo Staging Input Data \(Courtesy of European Space Agency\); #edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1; lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1; echo Staging Application; gunzip beam20.tar.gz; tar xvf beam20.tar; cd beam-2.0/bin; echo Starting Application;./pds2jpg-ASAR-run.sh $1; mv $1-b*.jpg../.. cd../.. rm -fr beam-2.0; rm -fr $PWD/$1.N1; rm -fr $PWD/beam20.tar; echo Input ENVISAT Product courtesy of European Space Agency touch ENVISAT_Product_courtesy_of_European_Space_Agency echo No Output Packaging; echo Done!; GSI Users have to cope with complex security procedures, execution scripts, job description languages, command line based interfaces and lack of standards. This makes the learning curve very steep and keeps non IT-experts away. 4

5 Grid Interface Evolution The way users access Grid resources has continuously evolved towards simplicity and transparency: Command Line – Globus and gLite CLI – Used by the enthusiastic and early adopter scientists GUI applications – gEclipse, Grid2Win – Good to expand the communities but difficult to maintain Web Interface – GENIUS, P-GRADE – Easier for new users but monolithic Science Gateways Beijing, Asia 4, Riccardo Rotondo “robot” certificate s on “e-tokens” 5

6 Science Gateways A framework of tools that allows scientists to run applications with little concern for where the computation actually takes place. This is similar to cloud computing in which applications run as Web services on remote resources in a manner that is not visible to the end user. However, a science gateway is usually more than a collection of applications. Gateways often let users store, manage, catalogue, and share large data collections or rapidly evolving novel applications they cannot find anywhere else. Training and education are also a significant part of some Science Gateways Beijing, Asia 4, Riccardo Rotondo 6

7 Reference Model Beijing, Asia 4, Riccardo Rotondo Science Gateway Science Gateway Appl 1 Appl 2 Appl N Grid Services Embedded Applications Administrator Power User Basic User Users from different organisations having different roles and privileges Other Middleware 7

8 Federated Grid User Riccardo Rotondo 8 Science Gateway Beijing, Asia 4,

9 Identity provided federated Riccardo Rotondo 9 { idp1, idp2, … idpN } { idp1, idp2, … idPN } { idp1, idp2, … idpN } Beijing, Asia 4,

10 Federated Grid User Riccardo Rotondo 10 Science Gateway Beijing, Asia 4,

11 AuthN & AuthZ Schema Beijing, Asia 4, Riccardo Rotondo AuthorisationAuthorisation Science Gateway GrIDP (“catch-all”) GrIDP (“catch-all”) IDPCT (“catch- all”) IDPCT (“catch- all”) IDP_y LDAP Register to a Service 2. Sign in Authentication Social Networks’ Bridge IdP 11

12 AuthN & AuthZ Schema Beijing, Asia 4, Riccardo Rotondo 12 Science Gateway

13 Identity Provider Federated Beijing, Asia 4, Riccardo Rotondo 13 { idp1, idp2, … idpN } { idp1, idp2, … idPN } { idp1, idp2, … idpN }

14 eduGAIN ( Beijing, Asia 4, Riccardo Rotondo All the Science Gateways developed at Catania are Service Providers of the eduGAIN inter-federation! 14

15 Federation Supported 7 Federations supported: GrIDP (“catch-all”, Italy); GARR-IDEM (Italy); CARSI (Chinese); GRNET-AAI (Greece); RCTSaai (Portugal); SIR (Spain); eduGAIN (Europe). Beijing, Asia 4, Riccardo Rotondo 15

16 Federation supported by DECIDE Science Gateway Beijing, Asia 4, Riccardo Rotondo 16

17 Other federation supported (1/2) Beijing, Asia 4, Riccardo Rotondo 17

18 Other federation supported (2/2) Beijing, Asia 4, Riccardo Rotondo 18

19 The GrIDP Identity Provider (1/2) 4 Identity Providers are available in GrIDP: A “catch-all” IdP created at Catania; The maat-G (enterprise) IdP; INFN-AAI IdP (all INFN researchers and associates); An idp that enables Social Networks credentials. Beijing, Asia 4, Riccardo Rotondo 19

20 The GrIDP Identity Provider (2/2) Beijing, Asia 4, Riccardo Rotondo 20

21 Liferay ( Highly-configurable, scalable, open source portal framework; Compatible with JSR 168/286 standards and based on modern web 2.0 technologies; Liferay services planned to be used: –Portal; –CMS & WCM; –Collaboration and “social” software Beijing, Asia 4, Riccardo Rotondo 21

22 Grid Access enable: Portlets as bricks Portlets can interact with the Grid e- Infrastructure Different approaches are available: –Execute the Command Line behind the portal; –Using API where available: Must be in Java or other languages supported by Liferay; –Call REST services from Javascript code in the browser; Additional layers between Liferay and the Grid can be necessary for some services; Each portlet can follow its own communication method. Beijing, Asia 4, Riccardo Rotondo 22

23 Usage Workflow Beijing, Asia 4, Riccardo Rotondo 1. Sign in eTokenServer User Track- ing DB 3. Proxy request 4. Proxy transfer 5. Grid Interactions 5. Tracking Grid Request 6. Getting Results 23

24 Functionalities Beijing, Asia 4, Riccardo Rotondo 24

25 Science Gateway Developed DECIDE Science Gateway EUMDEDGRID SUPPORT Science Gateway INDICATE Science Gateway VIRALGRID Science Gateway GISELA Science Gateway Beijing, Asia 4, Riccardo Rotondo CHAIN Science Gateway The Gilda t-Infrastructure Portal The RICeVI Portal 25

26 Liferay AT INFN-CT Beijing, Asia 4, Riccardo Rotondo 26

27 GRID Science Gateway Beijing, Asia 4, Riccardo Rotondo GRID SCIENCE GATEWAY 27

28 The DECIDE Science Gateway (1/5) Beijing, Asia 4, Riccardo Rotondo 28

29 The DECIDE Science Gateway (2/5) Beijing, Asia 4, Riccardo Rotondo 29

30 The DECIDE Science Gateway (3/5) Beijing, Asia 4, Riccardo Rotondo 30

31 The EUMEDGRID-Support Science Gateway (1/2) Beijing, Asia 4, Riccardo Rotondo 31

32 The EUMEDGRID-Support Science Gateway (2/2) Beijing, Asia 4, Riccardo Rotondo 32

33 The INDICATE e-Culture Science Gateway (5/6) Beijing, Asia 4, Riccardo Rotondo 33

34 The INDICATE e-Culture Science Gateway (6/6) Beijing, Asia 4, Riccardo Rotondo 34

35 CMS Science Gateway Beijing, Asia 4, Riccardo Rotondo CMS SCIENCE GATEWAY 35

36 The CHAIN Science Gateway Beijing, Asia 4, Riccardo Rotondo 36

37 The CHAIN Science Gateway Beijing, Asia 4, Riccardo Rotondo 37

38 The CHAIN Science Gateway Beijing, Asia 4, Riccardo Rotondo 38

39 The Gilda t-Infrastructure Portal Beijing, Asia 4, Riccardo Rotondo 39

40 E-learning Science Gateway Beijing, Asia 4, Riccardo Rotondo E-learning SCIENCE GATEWAY 40

41 RICeVI Portal Beijing, Asia 4, Riccardo Rotondo Collaboration Sharing Dissemination Training 41

42 Current Status 16 Liferay-based Science Gateways (hosted in 2 servers) are currently powered by Shibboleth at INFN Catania; 7 Federations supported; 4 instances are registered as official IDEM Service Provider; 4 Identity Providers are available in GrIDP. Beijing, Asia 4, Riccardo Rotondo 42

43 Credits & Acknowledgments Valeria Ardizzone (GARR); Roberto Barbera (UNICT & INFN) Riccardo Bruno (COMETA); Antonio Calanducci (COMETA); Marco Fargetta (COMETA) Elisa Ingrà (GARR); Giuseppe La Rocca (INFN) Salvatore Monforte (INFN); Fabrizio Pistagna (INFN); Rita Ricceri (INFN); Diego Scardaci (INFN); Beijing, Asia 4, Riccardo Rotondo Credits Acknowledgments Vincenzo Ciaschini (INFN); Enrico Fasanelli (INFN); Maria Laura Mantovani (GARR); Barbara Monticini (GARR); Simona Venuti (GARR) 43

44 References Science Gateway Url: – – – – – – Beijing, Asia 4, Riccardo Rotondo 44

45 Questions? Beijing, Asia 4, Riccardo Rotondo Thank you for your attention 45