Conjugating Science Gateways and Grid Portals into e-Collaboration environments: the Liferay and GENIUS/EnginFrame use case Riccardo Rotondo (riccardo.rotondo@ct.infn.it) INFN Catania – Italy International Workshop on Science Gateways 2010 Catania, 20-21 September 2010

Outline Who’s INFN ? Who’s NICE ? The e-Science paradigm and its adoption “hurdles”; The current GENIUS portal: new functionalities and current limitations; The approach towards a new e-collaboration environment: The e-Collaboration environment architecture; Liferay; EnginFrame 2010; Demo; Future work; Summary and conclusions. IWSG ‘10 Catania – September 21 2010 - 2

ASTROPARTICLE PHYSICS The INFN (www.infn.it) The INFN - the Italian National Institute of Nuclear Physics – is an organization dedicated to the study of the fundamental constituents of matter, and conducts theoretical and experimental research in the fields of sub-nuclear, nuclear, and astro-particle physics. Fundamental research in these areas requires the use of cutting-edge technologies and instrumentation, which the INFN develops both in its own laboratories and in collaboration with the world of industry. These activities are conducted by INFN in close collaboration with the academic world. PARTICLE PHYSICS  ASTROPARTICLE PHYSICS  NUCLEAR PHYSICS  THEORETICAL PHYSICS  TECHNOLOGY RESEARCH  INFN is organized in 19 Divisions, located at university physics departments, and 4 National Laboratories located in Catania, Legnaro, Rome and under the Gran Sasso mountain; The INFN workforce includes about 2000 of its own employees, almost 2000 university employees involved in research conducted by the Institute, and 1300 young researchers, including undergraduate and graduate students and research fellows. IWSG ‘10 Catania – September 21 2010 - 3

The INFN Grid Project (grid.infn.it) The INFN Grid project is an internal special initiative used by INFN to develop and deploy the Grid middleware services which allow its various user communities, while taking part in worldwide experiments and projects, to transparently and securely share the computing and storage resources together with the applications and technical facilities available in the different administrative domains of the various Institutions and geographical sites. More information at http://grid.infn.it/modules/IG_history/. INFN operates since 2001 a country-wide production quality Grid Infrastructure made, as of today, of almost 40 sites located all over Italy; The INFN Grid Infrastructure counts more than 16,000 CPU cores and several Petabytes of storage, both on disks and tapes; The INFN Grid Infrastructure provide computing and storage services not only to the High Energy Physics community but also to several other sciences such as Astrophysics, Bioinformatics, Biomedicine, Chemistry, Earth Science, etc. IWSG ‘10 Catania – September 21 2010 - 4

INFN in the “Grid World” EU Projects tagged with are those where INFN manages coordination tasks. IWSG ‘10 Catania – September 21 2010 - 5

Computationally intensive research IWSG ‘10 Catania – September 21 2010 - 8

Computationally intensive research IWSG ‘10 Catania – September 21 2010 - 9

e-Science Virtual Organisations e-Infrastructure Applications Data Instruments IWSG ‘10 Catania – September 21 2010 - 10

e-Infrastructure at a “world” scale! Tier-1 Centers: TRIUMF (Canada); GridKA(Germany); IN2P3 (France); CNAF (Italy); SARA/NIKHEF (NL); Nordic Data Grid Facility (NDGF); ASCC (Taipei); RAL (UK); BNL (US); FNAL (US); PIC (Spain) IWSG ‘10 Catania – September 21 2010 - 11

But…using Grids is not straightforward  echo Staging Input Data \(Courtesy of European Space Agency\); #edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1; lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1; echo Staging Application; gunzip beam20.tar.gz; tar xvf beam20.tar; cd beam-2.0/bin; echo Starting Application; ./pds2jpg-ASAR-run.sh $1; mv $1-b*.jpg ../.. cd ../.. rm -fr beam-2.0; rm -fr $PWD/$1.N1; rm -fr $PWD/beam20.tar; echo Input ENVISAT Product courtesy of European Space Agency touch ENVISAT_Product_courtesy_of_European_Space_Agency echo No Output Packaging; echo Done!; Scripts $ voms-proxy-info --all subject   : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy/CN=proxy issuer    : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy identity  : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes/CN=proxy type      : proxy strength  : 1024 bits path      : /tmp/x509up_u512 timeleft  : 500:38:44 === VO gilda extension information === VO        : gilda subject   : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes issuer    : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft  : 23:59:33 uri       : voms.ct.infn.it:15001 $ glite-wms-job-status https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rN-XTAIGAg Current Status:     Done (Success) Logged Reason(s):    - Job terminated successfully Exit code:          0 Status Reason:      Job terminated successfully Destination:        grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda Submitted:          Tue Jun 29 15:34:40 2010 CEST ************************************************************* Type = "Job"; JobType = "MPICH"; MPIType = "MVAPICH2_PGI706"; CpuNumber = 16; MPIGranularity = 4; Executable = "flash2"; StdOutput = "mpi.out"; StdError = "mpi.err"; InputSandbox = {"watchdog.sh","mpi.pre.sh","mpi.post.sh","flash.par","summers_den_1e0.rates","flash2"}; OutputSandbox = {"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"}; Requirements = (other.GlueCEUniqueId == "unipa-ce-01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc"); RetryCount = 0; JDL CLI GSI

A viable solution ? The GENIUS Grid Portal  (INFN-NICE collaboration – https://genius.ct.infn.it) “robot” certificates on “e-tokens”

Current GENIUS pros and cons Hides completely the complexity of the underlying middleware; Shows the same GUI to the end users as the middleware changes/evolves; Through the support to robot certificates, allows everybody to access and use the Grid greatly reducing the slope of the learning curve for non expert users; Can be easily customized to expose services specific to a particular VO; Cons: Does not include Web 2.0 functionalities; It is quite monolithic and it is very difficult to integrate it inside thematic portals (e.g., science gateways). IWSG ‘10 Catania – September 21 2010 - 17

The new architecture 2010 Single Sign-On IWSG ‘10 Catania – September 21 2010 - 18

Liferay (www.liferay.com) Highly-configurable, scalable, open source portal framework; Compatible with JSR 168/286 standards and based on modern web 2.0 technologies; Liferay services planned to be used: Portal; CMS & WCM; Collaboration and “social” software. IWSG ‘10 Catania – September 21 2010 - 19 19

EnginFrame 2010 architecture Services Gateway Portal Monitoring & Applications Data Security Management Distributed Resource Manager (LSF, PBS, MOAB, Torque/MAUI, SGE, WinHPC, iRODS, gLite) Remote2D/3D Viz. (tightVNC) Virtual/Physical Provisioning Distributed Data Management Distributed and Heterogeneous Distributed and Heterogeneous Computational Resources Data Resources IWSG ‘10 Catania – September 21 2010 - 20

EnginFrame 2010 new GUI (1/3) (based on the latest Web 2 EnginFrame 2010 new GUI (1/3) (based on the latest Web 2.0 and AJAX technologies) Modern, elegant and visually appealing Very unobtrusive and easy to use Designed to help organi-zations and people to work more efficiently and to get more done in less time IWSG ‘10 Catania – September 21 2010 - 21

EnginFrame 2010 new GUI (2/3) (based on the latest Web 2 EnginFrame 2010 new GUI (2/3) (based on the latest Web 2.0 and AJAX technologies) Sorting Filtering Searching Pagination Charts Commands IWSG ‘10 Catania – September 21 2010 - 22

EnginFrame 2010 new GUI (3/3) (based on the latest Web 2 EnginFrame 2010 new GUI (3/3) (based on the latest Web 2.0 and AJAX technologies) The RFB (Remote File Browser) dialog has been completely renewed to make server-side (and Grid) input data selection even easier The new file manager component allows to seamless navigate and access server-side (and Grid) files from the web browser

What has been done so far …and what’s missing Single Sign-On across Liferay, EnginFrame and the Grid with robot certificate and VOMS proxy extensions; Co-existence of Liferay and EnginFrame on the same physical machine but with two different Tomcat instances; EnginFrame services rendered as Liferay portlets; To do: Evaluate the need to make Liferay and EnginFrame co-exist within the same Tomcat instance; Improvement of the look&feel (so, do not expect beautiful web pages during the demo ). IWSG ‘10 Catania – September 21 2010 - 24

Demo: Large scale phylogenetic analyses with Mr. Bayes WNs SSO http://www.youtube.com/watch?v=oo5F0N9H9EQ IWSG ‘10 Catania – September 21 2010 - 25

DEMO IWSG ‘10 Catania – September 21 2010 - 26

Future Implementations For future implementations we are thinking how to cover the distances between enginframe and grid services. Currently we still use a gLite user interface to access middleware service. In other to avoid its usage… IWSG ‘10 Catania – September 21 2010 - 27

Future Implementations One of the possible solution could be a grid layer. A collection of standard api designed to go through a specific architecture. It will act as an abastract layer that have to hid complex grid command to access grid services. We are not designing it to be attached only to enginframe. If we design a standard interface in future users could use it to tie … IWSG ‘10 Catania – September 21 2010 - 28

Future Implementations … a specific application, in the same way, we can put everything we want, it could be something totally different from a grid portal. On the other side, the grid side, if we design a standard interface, service oriented, we can not support glite middleware only, but other middleware too so that user can use grid general services without being worried about the middleware behind the grid layer. IWSG ‘10 Catania – September 21 2010 - 29

Future work 1. 2. 3. Hooks to e- collaboration and e-learning tools EnginFrame ser- vices fully exposed as Liferay portlets Creation of gene- ric APIs to call different middle- ware 1. 2. 3. 2010 IWSG ‘10 Catania – September 21 2010 - 30

Summary and conclusions After several years of experience with GENIUS and EnginFrame, the work presented here will break with the past; We won’t provide any more top-down solutions; we will instead work with NICE to create “bricks” that could be used to quickly build thematic science gateways; One of this bricks will support remote visualization for those application who needs to render three dimensional images. EnginFrame is a commercial product fully based on open standards and open platforms; not-for-profit projects with educational and research purposes will be granted with free licenses; first line support could be provided by INFN or, if requested, by NICE; We will test this new approach in the framework of two European Grid projects: DECIDE (life sciences) and INDICATE (cultural heritage) that are going to start on the 1st of September 2010. If there is any interest in your community, we are of course, open to discuss synergies and possible collaborations, especially on the extension of the Grid Layer to others middleware used. IWSG ‘10 Catania – September 21 2010 - 31

Acknowledgments Roberto Barbera1,2 (roberto.barbera@ct.infn.it) Giuseppe La Rocca2 (giuseppe.larocca@ct.infn.it) Alberto Falzone3 (alberto.falzone@nice-software.com) Paolo Maggi3 (paolo.maggi@nice-software.com) Nicola Venuti3 (nicola.venuti@nice-software.com) Luca Carrogu3 (luca.carrogu@nice-software.com) Enrico Usai3 (enrico.usai@nice-software.com) 1. Department of Physics and Astronomy of the University of Catania - Italy 2. Italian National Institute of Nuclear Physics, Division of Catania - Italy 3. NICE srl - Italy IWSG ‘10 Catania – September 21 2010 - 32