1. Enabling Grids for E-sciencE Conjugating Science Gateways and Grid Portals into e-Collaboration environments: the Liferay and GENIUS/EnginFrame use case Riccardo Rotondo (riccardo.rotondo@ct.infn.it) INFN Catania - Italy TeraGrid ‘10 Pittsburgh, 2-5 August 2010

2. Enabling Grids for E-sciencE Outline Who’s INFN ? Who’s NICE ? The e-Science paradigm and its adoption “hurdles”; The current GENIUS portal: new functionalities and current limitations; The approach towards a new e-collaboration environment: –The new GENIUS architecture; –Liferay; –EnginFrame 2010; Demo; Future work; Summary and conclusions. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 2

3. Enabling Grids for E-sciencE The INFN (www.infn.it) The INFN - the Italian National Institute of Nuclear Physics – is an organization dedicated to the study of the fundamental constituents of matter, and conducts theoretical and experimental research in the fields of sub-nuclear, nuclear, and astro-particle physics. Fundamental research in these areas requires the use of cutting-edge technologies and instrumentation, which the INFN develops both in its own laboratories and in collaboration with the world of industry. These activities are conducted by INFN in close collaboration with the academic world. PARTICLE PHYSICS ASTROPARTICLE PHYSICS NUCLEAR PHYSICS THEORETICAL PHYSICS TECHNOLOGY RESEARCH INFN is organized in 19 Divisions, located at university physics departments, and 4 National Laboratories located in Catania, Legnaro, Rome and under the Gran Sasso mountain; The INFN workforce includes about 2000 of its own employees, almost 2000 university employees involved in research conducted by the Institute, and 1300 young researchers, including undergraduate and graduate students and research fellows. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 3

4. Enabling Grids for E-sciencE The INFN Grid Project (grid.infn.it) The INFN Grid project is an internal special initiative used by INFN to develop and deploy the Grid middleware services which allow its various user communities, while taking part in worldwide experiments and projects, to transparently and securely share the computing and storage resources together with the applications and technical facilities available in the different administrative domains of the various Institutions and geographical sites. –More information at http://grid.infn.it/modules/IG_history/. INFN operates since 2001 a country-wide production quality Grid Infrastructure made, as of today, of almost 40 sites located all over Italy; The INFN Grid Infrastructure counts more than 16,000 CPU cores and several Petabytes of storage, both on disks and tapes; The INFN Grid Infrastructure provide computing and storage services not only to the High Energy Physics community but also to several other sciences such as Astrophysics, Bioinformatics, Biomedicine, Chemistry, Earth Science, etc. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 4

5. Enabling Grids for E-sciencE INFN in the “Grid World” EU Projects tagged with are those where INFN manages coordination tasks. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 5

6. Enabling Grids for E-sciencE 3 About NICE Founded 1996, profitable, self-funded NICE as ISV (Independent Software Vendor) – Mission: facilitate access to Technical and Cloud Computing applications, data and resources – EnginFrame product line for Technical Computing Portal (web and web-services) – CloudFrame solution for Cloud Platform-as-a-Service – Global partnerships in HPC and Visualization NICE as solution provider – 14 years experience with Enterprise Technical Computing throughout all industries – Vertical and horizontal solutions – Cloud Computing solutions

7. Enabling Grids for E-sciencE ~100 Customers... and growing Automotive & Industrial Equipment Audi, ARRK, Bridgestone, Bosch, Corus Automotive, Delphi, Elasis/CRF, Ferrari, Brawn GP, Jaguar-LandRover, Magneti Marelli, McLaren, P+Z, PSA, RedBull Engineering, Swagelok, Suzuki, Toyota, TRW, Volkswagen Life Sciences Bayer, LitBio project, DEISA project, Biolab, Swiss Institute for Bioinformatics, Partners Healthcare, Pharsight, M.D. Anderson Cancer Center Energy & Utilities Addax Petroleum, AECL, BG, Chevron, ConocoPhillips, DSC-Libya, ENI, ExxonMobil, Hess, Marathon Oil, Nexen, Rosneft, Schlumberger, Senergy, Sibneft, Sinopec, Slavneft, Sonatrach, Statoil, Talisman Energy, TNK-BP, TNNC, TOTAL, TyumenNIIGaz, VNIIGaz, WesternGeco, Research & Education CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN2P3, ENEA, Georgia State Uni, ICI,, INFN, ITEP, Harvard Business School, RMSC, SSC-Russia, SDSC, Ferrara Uni, ITU, T.U.Dresden, Trinity College Dublin, Huazhong Normal Uni, Yale University High Tech STMicroelectronics, Accent, Samsung SDI, SensorDynamics, Motorola Aerospace & Manufacturing AIRBUS, Air Products and Chemicals, Procter&Gamble, Galileo Avionica, Hamilton Sunstrand, Kimberly Clark, Magellan Aerospace, MTU, Pratt & Whitney, Raytheon, Simpson Strong-Tie Financial Services UniCredit, Sal Oppenheim

8. Enabling Grids for E-sciencE Computationally intensive research Teragrid ‘10 Pittsburgh PA – August 5 2010 - 8

9. Enabling Grids for E-sciencE Computationally intensive research Teragrid ‘10 Pittsburgh PA – August 5 2010 - 9

10. Enabling Grids for E-sciencE e-Science Applications Data Instruments e-Infrastructure Virtual Organisations Teragrid ‘10 Pittsburgh PA – August 5 2010 - 10

11. Enabling Grids for E-sciencE Tier-1 Centers: TRIUMF (Canada); GridKA(Germany); IN2P3 (France); CNAF (Italy); SARA/NIKHEF (NL); Nordic Data Grid Facility (NDGF); ASCC (Taipei); RAL (UK); BNL (US); FNAL (US); PIC (Spain) e-Infrastructure at a “world” scale! Teragrid ‘10 Pittsburgh PA – August 5 2010 - 11

12. Enabling Grids for E-sciencE Type = "Job"; JobType = "MPICH"; MPIType = "MVAPICH2_PGI706"; CpuNumber = 16; MPIGranularity = 4; Executable = "flash2"; StdOutput = "mpi.out"; StdError = "mpi.err"; InputSandbox = {"watchdog.sh","mpi.pre.sh","mpi.post.sh","flash.par","summers_den_ 1e0.rates","flash2"}; OutputSandbox = {"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"}; Requirements = (other.GlueCEUniqueId == "unipa-ce- 01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc"); RetryCount = 0; $ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy/CN=proxy issuer : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy identity : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy type : proxy strength : 1024 bits path : /tmp/x509up_u512 timeleft : 500:38:44 === VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft : 23:59:33 uri : voms.ct.infn.it:15001 $ glite-wms-job-status https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN- XTAIGAg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN- XTAIGAg Current Status: Done (Success) Logged Reason(s): - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda Submitted: Tue Jun 29 15:34:40 2010 CEST ************************************************************* But…using Grids is not straightforward  CLI Scripts JDL echo Staging Input Data \(Courtesy of European Space Agency\); #edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1; lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1; echo Staging Application; gunzip beam20.tar.gz; tar xvf beam20.tar; cd beam-2.0/bin; echo Starting Application;./pds2jpg-ASAR-run.sh $1; mv $1-b*.jpg../.. cd../.. rm -fr beam-2.0; rm -fr $PWD/$1.N1; rm -fr $PWD/beam20.tar; echo Input ENVISAT Product courtesy of European Space Agency touch ENVISAT_Product_courtesy_of_European_Space_Agency echo No Output Packaging; echo Done!; GSI

13. Enabling Grids for E-sciencE A viable solution ? The GENIUS Grid Portal (INFN-NICE collaboration – https://genius.ct.infn.it) “robot” certificates on “e-tokens”

14. Enabling Grids for E-sciencE Robot certificates in a nutshell Robot certificates have been introduced by several Certification Authorities to allow users, who are not familiar with personal certificates and do not belong to any Vos, to access and use Grids. – They are extremely useful, for instance, to automatize grid service monitoring, data processing production, distributed data collection systems, etc.; – Basically, these certificates can be used to identify a person responsible for an “unattended” service or process acting as client and/or server. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 14

15. Enabling Grids for E-sciencE Robot certificates & eTokens In order to strong reduce the risks to have the portal certificate compromised, the INFN CA has decided to issue these special certificates on board of Aladdin eToken PRO 32K smart cards. Each smart card can support several (up to 32) robot certificates: one for each application available on the grid portal. GENIUS fully supports robot certificates ! Teragrid ‘10 Pittsburgh PA – August 5 2010 - 15

16. Enabling Grids for E-sciencE GENIUS and the robot certificates – The User Tracking tool 1. ask for a service 2. create a proxy with the robot certificate User 5. get the results 3. execute action 2’,3’. track user 6/7. query L&B for accounting data L&B 4. get output Admin 1. ask for a service The Grid Teragrid ‘10 Pittsburgh PA – August 5 2010 - 16

17. Enabling Grids for E-sciencE User Tracking System (1/2) Teragrid ‘10 Pittsburgh PA – August 5 2010 - 17

18. Enabling Grids for E-sciencE User Tracking System (2/2) Teragrid ‘10 Pittsburgh PA – August 5 2010 - 18

19. Enabling Grids for E-sciencE Current GENIUS pros and cons Pros: –Hides completely the complexity of the underlying middleware; –Shows the same GUI to the end users as the middleware changes/evolves; –Through the support to robot certificates, allows everybody to access and use the Grid greatly reducing the slope of the learning curve for non expert users; –Can be easily customized to expose services specific to a particular VO; Cons: –Does not include Web 2.0 functionalities; –It is quite monolithic and it is very difficult to integrate it inside thematic portals (e.g., science gateways). Teragrid ‘10 Pittsburgh PA – August 5 2010 - 19

20. Enabling Grids for E-sciencE The new GENIUS architecture 2010 Single Sign-On Teragrid ‘10 Pittsburgh PA – August 5 2010 - 20

21. Enabling Grids for E-sciencE Liferay (www.liferay.com) Highly-configurable, scalable, open source portal framework; Compatible with JSR 168/268 standards and based on modern web 2.0 technologies; Liferay services planned to be used: –Portal; –CMS & WCM; –Collaboration and “social” software. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 21

22. Enabling Grids for E-sciencE EnginFrame 2010 architecture Distributed and Heterogeneous Computational Resources Distributed and Heterogeneous Data Resources Distributed Resource Manager (LSF, PBS, MOAB, Torque/MAUI, SGE, WinHPC, iRODS, gLite) Distributed Data Management Applications Security Monitoring & Management Services Gateway Portal Data Remote2D/3D Viz. (tightVNC) Virtual/Physical Provisioning Teragrid ‘10 Pittsburgh PA – August 5 2010 - 22

23. Enabling Grids for E-sciencE EnginFrame 2010 new GUI (1/3) (based on the latest Web 2.0 and AJAX technologies) Modern, elegant and visually appealing Very unobtrusive and easy to use Designed to help organi- zations and people to work more efficiently and to get more done in less time Teragrid ‘10 Pittsburgh PA – August 5 2010 - 23

24. Enabling Grids for E-sciencE EnginFrame 2010 new GUI (2/3) (based on the latest Web 2.0 and AJAX technologies) Sorting Filtering Searching Pagination Charts Commands Teragrid ‘10 Pittsburgh PA – August 5 2010 - 24

25. Enabling Grids for E-sciencE EnginFrame 2010 new GUI (3/3) (based on the latest Web 2.0 and AJAX technologies) The RFB (Remote File Browser) dialog has been completely renewed to make server-side (and Grid) input data selection even easier The new file manager component allows to seamless navigate and access server-side (and Grid) files from the web browser

26. Enabling Grids for E-sciencE What has been done so far …and what’s missing Done: –Single Sign-On across Liferay, EnginFrame and the Grid with robot certificate and VOMS proxy extensions; –Co-existence of Liferay and EnginFrame on the same physical machine but with two different Tomcat instances; –EnginFrame services rendered as Liferay portlets; To do: –Evaluate the need to make Liferay and EnginFrame co-exist within the same Tomcat instance; –Improvement of the look&feel (so, do not expect beautiful web pages during the demo ). Teragrid ‘10 Pittsburgh PA – August 5 2010 - 26

27. Enabling Grids for E-sciencE WNs Demo: Large scale phylogenetic analyses with Mr. Bayes Teragrid ‘10 Pittsburgh PA – August 5 2010 - 27 http://www.youtube.com/watch?v=oo5F0N9H9EQ SSO

28. Enabling Grids for E-sciencE DEMO Teragrid ‘10 Pittsburgh PA – August 5 2010 - 28

29. Enabling Grids for E-sciencE Future work 2010 1.Hooks to e- collaboration and e-learning tools 2.EnginFrame ser- vices fully exposed as Liferay portlets 3.Creation of gene- ric APIs to call different middle- ware 1. 2. 3. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 29

30. Enabling Grids for E-sciencE Summary and conclusions After several years of experience with GENIUS and EnginFrame, the work presented here will break with the past; We won’t provide any more top-down solutions; we will instead work with NICE to create “bricks” that could be used to quickly build thematic science gateways; EnginFrame is a commercial product fully based on open standards and open platforms; not-for-profit projects with educational and research purposes will be granted with free licenses; first line support could be provided by INFN or, if requested, by NICE; We will test this new approach in the framework of two Euro- pean Grid projects: DECIDE (life sciences) and INDICATE (cultural heritage) that are going to start on the 1 st of September 2010; If there is any interest in the TeraGrid community, we are of course, open to discuss synergies and possible collabo- rations, especially on the extension of the Grid Layer to US middleware. Teragrid ‘10 Pittsburgh PA – August 5 2010 - 30

31. Enabling Grids for E-sciencE Acknowledgments Roberto Barbera 1,2 (roberto.barbera@ct.infn.it) Giuseppe La Rocca 2 (giuseppe.larocca@ct.infn.it) Alberto Falzone 3 (alberto.falzone@nice-software.com) Paolo Maggi 3 (paolo.maggi@nice-software.com) Nicola Venuti 3 (nicola.venuti@nice-software.com) Luca Carrogu 3 (luca.carrogu@nice-software.com) Enrico Usai 3 (enrico.usai@nice-software.com) 1. Department of Physics and Astronomy of the University of Catania - Italy 2. Italian National Institute of Nuclear Physics, Division of Catania - Italy 3. NICE srl - Italy Teragrid ‘10 Pittsburgh PA – August 5 2010 - 31

32. Enabling Grids for E-sciencE Teragrid ‘10 Pittsburgh PA – August 5 2010 - 32 http://agenda.ct.infn.it/event/iwsg2010 Important announcement Submission deadline: August 31, 2010 Workshop dates: September 20-22, 2010