1. The GENIUS Grid Portal Giuseppe LA ROCCA INFN Catania giuseppe.larocca@ct.infn.it ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia

2. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions Outline

3. Grid technology allows users to share a wide pletora of distributed computational resources regardless of their geographical location. Virtual services are exposed to the users through rather complex Command Line Interfaces or API languages. Grid security is indeed based on the Public Key Infrastructure (PKI) of X.509 certificates and the procedure to get and manage those certificates is unfortunately not straightforward; Up to now, the high security policy requested to access distributed computing resources has been a rather big limiting factor when trying to broaden the usage of Grids into a wide community of users; + +

4. User has to adhere to a Virtual Organization (VO) User needs an account on one of the trusted User Interface (UI) + + = Grid portals provide an added value to make Grids more appealing for non-expert users.

5. 5 A grid portal: why and how It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, cell phone). It can keep the same user interface to several back-ends. It must be redundantly “secure” at all levels: –1) secure for web transactions, –2) secure for user credentials, –3) secure for user authentication, –4) secure at VO/VOMS level. All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly.

6. A Grid Portal improves usability of Grids –Lowering end-user requirements for accessing the Grid –Hiding the complexity of data and job services management in the Grid A Grid Portal improves utilization of Grids –Making the Grid (r)evolution transparent to the end-user –Providing an appealing user-friendly Web interface –Enforcing Grid utilization policies Grid Portal Benefits

7. Interactive Applications Intranet Clients Win LX UXMac Grid / Compute Farm Internal Users Batch Applications Storage and Data Grid Portal / Gateway Project Managers Client Apps Standard protocols Licenses Home Users The Grid Portal / Gateway

8. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

9. What is EnginFrame ? It is a web-based technology able to expose Grid services running on Grid infrastructures It allows organizations to provide application- oriented computing and data services to both users (via Web browsers) and applications (via SOAP/WSDL and/or RSS) ‏ It’s a Grid gateway It greatly simplifies the development of Web Portals exposing computing services that can run on a broad range of different computational Grid systems

10. Spoolers HTML page Custom plugin Script Browser SDF XML EnginFrame Server HTML XSLT Grid Compute Farm Grid Compute Farm MetaFrame + NFuse MetaFrame + NFuse Application Server Application Server EnginFrame Agent Execute Service Req XML output Service Req User Authorize Groups, ACLs XML Layout XSL Service Submission EnginFrame Working Environment

11. gzip sample maximum medium none EF_SPOOLER_NAME="gzip $file” export EF_SPOOLER_NAME ${EF_ROOT}/plugins/lsf/bin/bsub -o output.txt gzip -$level \"$FILE\” Service example

12. EnginFrame snapshots Services are XML description defining –Input parameters –The action to accomplish (Unix/Windows script, Java, …)

13. User friendly, Application-oriented Job submission Flexible and efficient Input file management Hide complexity of underlying scheduler EF Customizable Job Submission

14. EF Monitoring & control Global Job monitoring Cluster & host monitoring Job details & control

15. Output management Data lifecycle managemnet Comprehensive output File manipulation (view, edit, delete, zip, …) Follow-up actions support RESUBMIT jobs – Rapidly edit input files and re-submit with same parameters/settings‏

16. 16 A growing number of customers… Automotive & Industrial Equipment Audi, ARRK, Bridgestone, Bosch, Corus Automotive, Delphi, Elasis/CRF, Ferrari, Brawn GP, Jaguar-LandRover, Lear, Magneti Marelli, McLaren, P+Z, PSA, RedBull Engineering, Swagelok, Suzuki, Toyota, TRW, Volkswagen Life Sciences LitBio project, DEISA project, Biolab, Swiss Institute for Bioinformatics, Partners Healthcare, M.D. Anderson Cancer Center Energy & Utilities Addax Petroleum, AECL, Amerada Hess, British Gas, CC of Water Resources, Chevron, Conoco-Phillips, DSC-Libya, ENI/Agip, GazPromNeft, Marathon Oil, Nexen, Rosneft, Schlumberger, Sibneft, Sinopec, Slavneft, Sonatrach, Statoil, Talisman Energy, Telecom Italia, TNK-BP, TNNC, TOTAL, TyumenNIIGaz, VNIIGaz, Xinjiang Oil Research & Education ASSC, CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN2P3, ENEA, FzU, ICI, IFAE, INFN, ITEP, Harvard Business School, SSC- Russia, SDSC, Ferrara Uni, ITU, T.U.Dresden, Trinity College Dublin, Huazhong Normal Uni, Yale University, UPM High Tech STMicroelectronics, Accent, Samsung SDI, SensorDynamics, Motorola Aerospace & Manufacturing AIRBUS, Air Products and Chemicals, Procter&Gamble, Galileo Avionica, Hamilton Sunstrand, Kimberly Clark, Magellan Aerospace, MTU, Northrop Grumman, P&W, Raytheon, Simpson Strong-Tie

17. GENIUS is a powerful Grid Portal that allows scientists to exploit Grid resources only using a conventional Web browser It has been built on top of the EnginFrame framework It’s a gateway to European EGEE Project middle-ware It allows to expose gLite-enabled applications via Web- browser as well as Web Services What is GENIUS ? www.enginframe.com www.nice-italy.com www.infn.it

19. GENIUS: Grid Preferences

20. 20 GENIUS: Job Submission

21. 21 GENIUS: Job Submission

22. Code for Job Queue management rewritten using GridML tags GENIUS: Job(s) Queue

23. New Confirmation Message! GENIUS: Job Retrieving

24. GENIUS: Data Spooler

25. Tight VNC GENIUS: Interactive Services

26. 26 Local Browse on laptop Remote Browse on UI (GENIUS Server) Extended Remote File Browse on LFC Catalog GENIUS: Data Management

27. 27 Extended Multiple Remote File Browsing on Catalog!

28. 28 GENIUS: Workflow

29. 29

30. 30

31. 31

32. 32

33. 33

34. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

35. VOMS Proxy Init Service A CAPTCHA Code is required to start the VOMS Proxy Applet for the proxy initialization The Java plugin 1.6.0 or higher is mandatory required.

36. Jointly developed by NICE and INFN Catania

41. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

42. Robot certificates have been introduced to permit users, who are not familiar with deal personal certificates and don’t belong to any VOs, to experience the Grid paradigm for research activity and reduce the initial barriers. –They are extremely useful for instance to automate grid service monitoring, data processing production, distributed data collection systems. –Basically these certificates can be used to identify a person responsible for an unattended service or process acting as client and/or server. Robot certificates in a nutshell

43. In order to strong reduce the risks to have the portal certificate compromised the INFN CA decided to issue this new certificate on board of the Aladdin eToken PRO 32K smart card. Each smart card can support several robot certificates: one for each application user wants to share with the other. –An user’s PIN is prompted every time user try to read the certificate stored on the smart card to generate a proxy. –A first prototype of Grid Portal using robot certificate to generate an user’s proxy has been successfully designed. Robot certificates in a nutshell

44. 1. ask for a service 2. create a proxy with the robot certificate 5. get the results 3. execute action 4. get output 2’,3’. track user User Admin The GENIUS Portal & Robot Certificates

45. The Users Tracking System (UTS) ACL-based services that enable easier access control customization for users not belonging to any group!

46. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

47. Porting the „MrBayes” application to Grid Case study from CNR - ITB

48. General Introduction MrBayes is a program for the Bayesian estimation of phylogeny. Bayesian inference of phylogeny is based on the posterior probability distribution of trees, which is the probability of a tree conditioned on the observations. –To approximate the posterior probability distribution of trees MrBayes uses a simulation technique called Markov Chain Monte Carlo (or MCMC). The program takes as input a character matrix in a NEXUS file format. The output is several files with the parameters that were sampled by the MCMC algorithm. The application is CPU demanding, especially if the MPI version of the software is used.

49. WMS LFC Catalog SE Phylogenetic analysis on large scale Robot Certificate UI + GENIUS Portal Job Submission Tool GRID

50. Job Submission Tool: is driven by the concept of “Task” as the applications are – Each task could be independent or could be described as depended from another “Task” – Each task is described by a “status” – The task is executed by a wrapper that takes care of monitoring the task: If the task is correctly executed the wrapper can change the status of the task from “Free” to “Done” If a single step on the job execution fails, the whole task is considered failed and automatically rescheduled JST tool takes care of submitting jobs, retrieving the output and monitoring the status of each task It is able to deal with accidental failure of grid services It is possible to change at run time the priority of each task/application JST characteristics

51. EF Monitoring with JST https://glite-tutor1.ct.infn.it

52. Case study from Porting the „ASTRA” application to Grid

53. The ASTRA project in a nutshell The ASTRA (Ancient instrument Sound/Timbre Reconstruction Application) projects aims to reconstruct the sound or timbre of ancient instruments using archaeological data as fragments from excavations, written descriptions, pictures... The technique used is the physical modeling synthesis, a complex digital audio rendering technique which allows to recreate a model of the musical instrument and produce the sound by simulating its behavior as a mechanical system.

54. Computer model The Grid Network Reconstructed sounds Archaeological findings Load the sounds on a piano keyboard and play 30 sec. of audio sound => 90min. on Pentium @ 3.73Ghz, 2 GB RAM Modeling and computation on the Grid

55. How does ASTRA reconstruct the sound of the instruments ? The modeling process is known as Physical Modeling Synthesis Physical modeling creates a virtual model of the instrument and reproduces its sound by simulating its behaviour as a mechanical system. »This approach is also referred to as “synthesis by rule”. The higher is the quality of the audio files, the longer is the time required To have an idea of the needed time for simulation, on a Pentium IV 3.73 Ghz, 2GB RAM Personal Computer to correctly reproduce a sound lasting for 30 seconds it could be required more than 90 min. (image unisa.it)

56. ASTRA project is involved on..

57. GRID UI ASTRA software VOMS Server WMS User

58. In June 2009, the concert "Musica @ Fisica", was organized by the Catania Division of the Italian National Institute of Nuclear Physics."Musica @ Fisica" This world premiere showcased the sounds of the Epigonion, an instrument of the past, reconstructed via computer-intensive modelling, being performed alongside real instruments such as baroque cello and percussions Purcell’sPurcell’s “The Sparrow and the Gentle Dove”, (a reconstructed Epigonion played live with percussion and baroque cello) http://www.youtube.com/watch?v=mnEbtIQkJ-A Musica @ Fisica

59. Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

60. GENIUS offers the following advantages: it is a complete production-ready environment which combines the concepts of “user portal” and “science portal”; absolutely no client software needs to be installed on the user’s workstation apart from the web browser with its usual plug-ins like Java (at least JRE 1.6.0 or higher); it provides a new unique tool to authorize users, in a very strong secure way, into the grid environment with or without VOMS support as well, easy to use; it includes support for both single and composite jobs (including DAG’s); interactive analysis and web access to personal spooling areas are possible; environment and settings customizable for the users; security for data management and sessions.

61. References NICE web-site http://www.nice-italy.comhttp://www.nice-italy.com EnginFrame Framework http://www.enginframe.com http://www.enginframe.com GENIUS Portal https://genius.ct.infn.ithttps://genius.ct.infn.it GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it GENIUS based on gLite at https://glite- tutor.ct.infn.ithttps://glite- tutor.ct.infn.it GENIUS Installation GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it Write an email message to alberto.falzone@nice- italy.com or nicola.venuti@nice-italy.com for an account request to download the GENIUS packagealberto.falzone@nice- italy.comnicola.venuti@nice-italy.com

62. Hands-on practicals Open your browser and connect to https://glite-tutor.ct.infn.it login=kualalumpurXX PassWord=GridKUAXX where XX = 001,..,050 Connect to the gLite User Interface