1. www.consorzio-cometa.it FESR Consorzio COMETA - Progetto PI2S2 Il portale grid: GENIUS Alberto Falzone NICE srl Consorzio COMETA – Progetto PI2S2 Tutorial per l’Università di Catania Catania, 14-15.03.2007

2. Technical Training Catania, PI2S2, 15.03.2007 2 Contents Grid Portals technology GENIUS Portal Some Integrated Applications Future perspectives

3. Technical Training Catania, PI2S2, 15.03.2007 3 What is a Grid Portal? A Grid Portal is a system to expose grid services that operate on grid resources Services are: –generic as well as application-specific grid services Resources are (stateful) instances of: –jobs, datasets, computing/storage nodes, queues, SW licenses Services are exposed through: –(X)HTML, Portlets, WebServices

4. Technical Training Catania, PI2S2, 15.03.2007 4 Grid Portal Benefits A Grid Portal improves usability of Grids –Verticalizing the service offering –Lowering end-user requirements for accessing the Grid –Hiding the complexity of data management in the Grid A Grid Portal improves utilization of Grids –Making the Grid (r)evolution transparent to the end-user –Providing an appealing user-friendly Web interface –Enforcing Grid utilization policies

5. Technical Training Catania, PI2S2, 15.03.2007 5 The Web-centric vision Licenses & Data Licenses & Data Grid / Compute Farm Grid / Compute Farm Interactive Apps Interactive Apps Inter/Intranet Server(s) Clients Standard Web Browser

6. Technical Training Catania, PI2S2, 15.03.2007 6 The Grid Portal / Gateway Interactive Applications Intranet Clients Win LX UXMac Grid / Compute Farm Internal Users Batch Applications Storage and Data Grid Portal / Gateway Project Managers Client Apps Standard protocols Licenses Home Users

7. Technical Training Catania, PI2S2, 15.03.2007 7 Contents Grid Portals technology GENIUS Portal Some Integrated Applications Future perspectives

8. Technical Training Catania, PI2S2, 15.03.2007 8 What is GENIUS? GENIUS is a powerful Grid Portal that allows scientists to exploit Grid resources only using a conventional Web browser It’s a gateway to European EGEE Project’s middle- ware It has been built on top of the EnginFrame framework It allows to expose gLite-enabled applications via Web-browser as well as Web Services

9. Technical Training Catania, PI2S2, 15.03.2007 9 The GENIUS hourglass model OS & Net services Basic Services High level GRID middleware ALICEATLASCMSLHCb Applications’ specific layer Other apps GLOBUS toolkit EGEE (LCG/gLite) architecture GENIUS web portal GENIUS Grid Enabled web eNvironment for site Independent User job Submission INFN / NICE Collaboration

10. Technical Training Catania, PI2S2, 15.03.2007 10 GENIUS Architecture Apache + Tomcat EnginFrame GENIUS https+java/xml+rfb WEB Browser UI Local WS the Grid M/W+GSI 3-tier model

11. Technical Training Catania, PI2S2, 15.03.2007 11 Globus middleware Compute resources EGEE middleware LCG-2 / gLite Local Data Distributed Data WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Contaiers Classic GENIUS Authentication – ACL management General XML Application Kits Data Management & Virtualization VO n - XML Application Kit VO 1 - XML Application Kit Monitoring & Accounting VNC remote Desktop over SSL MyProxy auth. w/ VOMS extensions GENIUS as Grid Gateway End users Presentation engine Portlet GW WS GW RSS GW

12. Technical Training Catania, PI2S2, 15.03.2007 12 GENIUS Terminology Service – anything that can be submitted through GENIUS Portal SDF – Service Definition File: XML file which contains the definitions of services Authorization & Authentication – a system aimed to allow/deny user access to resources based on policies Plug-in – a piece of code that enhances the GENIUS GRID Portal Spooler – data container created by GENIUS to host the submitted commands

13. Technical Training Catania, PI2S2, 15.03.2007 13 Spoolers HTML page GENIUS Working Environment Custom plugin Script Browser SDF XML EnginFrame Server HTML XSLT Grid Compute Farm Grid Compute Farm MetaFrame + NFuse MetaFrame + NFuse Application Server Application Server EnginFrame Agent Execute Service Req XML output Service Req User Authorize Groups, ACLs XML Layout XSL Admin Service Submission

14. Technical Training Catania, PI2S2, 15.03.2007 14 GENIUS Grid Portal Reference Web Site: https://genius.ct.infn.it

15. Technical Training Catania, PI2S2, 15.03.2007 15 Layout structure: –Left navigation tree with the list of services –Central "content" area with forms and output from services –Top Menu area Customizing the layout - Structure

16. Technical Training Catania, PI2S2, 15.03.2007 16 Creating a SDF – example <ef:agent id="myagentname" authority="${EF_DEFAULT_AUTHORITY}“ xmlns:ef="http://www.enginframe.com/2000/EnginFrame"> EnginFrame Grid Portal Welcome to EnginFrame <ef:location host="${EF_AGENT_HOST}" port="${EF_AGENT_PORT}"/> …

17. Technical Training Catania, PI2S2, 15.03.2007 17 Services Example gzip sample maximum medium none EF_SPOOLER_NAME="gzip $file” export EF_SPOOLER_NAME ${EF_ROOT}/plugins/lsf/bin/bsub -o output.txt gzip -$level \"$FILE\”

18. Technical Training Catania, PI2S2, 15.03.2007 18 GENIUS: security infrastructure All web transactions are executed under the Secure Socket Layer (SSL) via HTTPS The user must have an account on the User Interface When the user wants to interact with the file-system of the UI, he gets prompted for the username and password of the account on that machine Grid authentication in GENIUS is currently based on the concept Proxy

19. Technical Training Catania, PI2S2, 15.03.2007 19 GENIUS: files management OS authority Both authorities os and genius are distinct: we have services with differen authority!

20. Technical Training Catania, PI2S2, 15.03.2007 20 GENIUS: files management Remote File Browser On the User Interface

21. Technical Training Catania, PI2S2, 15.03.2007 21 GENIUS: files management

22. Technical Training Catania, PI2S2, 15.03.2007 22 GENIUS: files management Multiple Action in a Single Service

23. Technical Training Catania, PI2S2, 15.03.2007 23 GENIUS: Grid Authentication with Myproxy myproxy-get-delegation

24. Technical Training Catania, PI2S2, 15.03.2007 24 Grid authentication with MyProxy UI Local WS MyProxy Server GENIUS Server (UI) myproxy-init any grid service myproxy-get-delegation output the Grid execution WEB Browser Now, VOMS Extensions needed to run jobs on the GRID ( --voms )

25. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 25 Grid authentication with MyProxy UI Loc al WS MyProxy Server GENIUS Server (UI) myproxy-init (proxy+myproxy passphrase) any grid service myproxy-get-delegation output the Grid execution WEB Browser

26. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 26 A proxy from MyProxy subject : /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/CN=Valeria Ardizzone/Email=valeria.ardizzone@ct.infn.it/CN=proxy/CN=proxy/CN=proxy issuer : /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/CN=Valeria Ardizzone/Email=valeria.ardizzone@ct.infn.it/CN=proxy/CN=proxy identity : /C=IT/O=GILDA/OU=Personal Certificate/L=INFN Catania/CN=Valeria Ardizzone/Email=valeria.ardizzone@ct.infn.it/CN=proxy/CN=proxy type : unknown strength : 512 bits path : /tmp/x509up_u515 timeleft : 11:59:20 >myproxy-init ( Enter GRID pass phrase for this identity:…Enter MyProxy pass phrase:…Verifying password-Enter MyProxy pass phrase:… ) >myproxy-get-delegation (Enter MyProxy pass phrase:……) USER from command line interface: GENIUS service:

27. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 27 Coming soon.... 1st Solution GENIUS Server (UI) MyProxy Server Loc al WS WEB Browser VOMS Server the Grid execution APPLET User Private key and User Certificate MyProxyInit Java Applet Create a proxy with AC (12h) (chain =1) Store a proxy with AC (12h) (chain = 2) myproxy-get- delegation with AC(12h) (chain = 3) GSI Pure VOMS JAVA API https+java/xml+rfb

28. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 28 Coming soon..... 2nd Solution GENIUS Server (UI) Loc al WS WEB Browser VOMS Server the Grid execution APPLET User Private key and User Certificate VOMSProxyInit Java Applet Create a proxy (chain =1) GSI Proxy Create a new proxy with AC (chain =2) voms-proxy-init –voms voms –cert …. in waiting pure VOMS JAVA API Server Daemon 2nd Solution

29. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 29 Next Developments GENIUS Server (UI) Loc al WS WEB Browser VOMS Server the Grid execution APPLET User Private key and User Certificate VOMSProxyInit Java Applet Create a proxy with AC (12h) (chain =1) GSI Proxy Server Daemon GSI

30. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 30 GENIUS VOMS Proxy Init service

31. To change: View -> Header and Footer Catania, PI2S2, 15.03.2007 31 GENIUS VOMS Proxy Init service

32. Technical Training Catania, PI2S2, 15.03.2007 32 GENIUS: security Transparent VOMS Support

33. Technical Training Catania, PI2S2, 15.03.2007 33 GENIUS: preferences

34. Technical Training Catania, PI2S2, 15.03.2007 34 GENIUS: Job Submission Preferences are on the horinzontal bar after successful login to the Grid (under genius authority)

35. Technical Training Catania, PI2S2, 15.03.2007 35 GENIUS: Job Editor for jdl files

36. Technical Training Catania, PI2S2, 15.03.2007 36 GENIUS: Job Queue Code for Job Queue management rewritten using GridML tags

37. Technical Training Catania, PI2S2, 15.03.2007 37 GENIUS: Get Output New Confirmation Message!

38. Technical Training Catania, PI2S2, 15.03.2007 38 GENIUS: Job Queue - Spooler

39. Technical Training Catania, PI2S2, 15.03.2007 39 GENIUS: Interactive Tight VNC

40. Technical Training Catania, PI2S2, 15.03.2007 40 GENIUS: Data & Grid Catalogs Local Browse on laptop Remote Browse on UI (GENIUS Server) Extended Remote File Browse on LFC Catalog

41. Technical Training Catania, PI2S2, 15.03.2007 41 GENIUS: Data & Grid Catalogs Extended Multiple Remote File Browsing on Catalog!

42. Technical Training Catania, PI2S2, 15.03.2007 42 GENIUS: Data & Grid Catalogs

43. Technical Training Catania, PI2S2, 15.03.2007 43 Contents Grid Portals technology GENIUS Portal Some Integrated Applications Future perspectives

44. Technical Training Catania, PI2S2, 15.03.2007 44 B L A S T This is a work supported in part by the LIBI and the BIOINFOGRID project.LIBI BIOINFOGRID

45. Technical Training Catania, PI2S2, 15.03.2007 45 What BLAST is ? BLAST (Basic Local Alignment Search Tool) provides a method for rapid searching of nucleotide and protein databases. The program compares nucleotide or protein sequences to sequence databases and calculates the statistical significance of matches.

46. Technical Training Catania, PI2S2, 15.03.2007 46 The home - https://glite-demo.ct.infn.it/ The Services

47. Technical Training Catania, PI2S2, 15.03.2007 47 Bioinformatics databases This table is dynamically created take into account the databases published in the LFC Catalog.

48. Technical Training Catania, PI2S2, 15.03.2007 48 MULTI BLAST : input form /1 Upload the file with the FASTA sequences

49. Technical Training Catania, PI2S2, 15.03.2007 49 MULTI BLAST : submit Multi FASTA successfully submitted to WMProxy Inspect the status of the Collection

50. Technical Training Catania, PI2S2, 15.03.2007 50 MULTI BLAST : queue Retrieve the output of the Collection

51. Technical Training Catania, PI2S2, 15.03.2007 51 MULTI BLAST : Data Spooler View the output of each subjobs

52. Technical Training Catania, PI2S2, 15.03.2007 52 Considerations We have proposed a Grid infrastructure, based on a user-friendly interface, that ease the interact with the last release of gLite middleware and allow biologists to submit, in a transparent way regardless the complexity of the underlying infrastructure, a multi BLAST search composed by several FASTA sequences. At the moment the Grid does not give any advantage respect to the Blast Web portal for simple queries: the time needed to start a grid job is considerably longer then the time to execute the query on the web portal. The advantage comes if a user is interested in a bunch of queries: in this case the different queries can be subdivide between several jobs, and the answer to a multiple query can be obtained in the same time as a single query.

53. Technical Training Catania, PI2S2, 15.03.2007 53 R O S E T T A

54. Technical Training Catania, PI2S2, 15.03.2007 54 What ROSETTA does Proteins existing on Earth are only an infinitesimal fraction of the possible sequences – How the “few” extant proteins were produced and/or selected during prebiotic molecular evolution? – Do extant proteins have particular features which make them eligible for selection (e.g., thermodynamic, kinetic or other physical properties)? ROSETTA is a protein structure prediction method Prediction of the three-dimensional structure of never born proteins using the ROSETTA software

55. Technical Training Catania, PI2S2, 15.03.2007 55 The home – https://glite-tutor.ct.infn.it The Services Application available under “VO Services” for the VO GILDA.

56. Technical Training Catania, PI2S2, 15.03.2007 56 Create the dynamic ClassAD /1

57. Technical Training Catania, PI2S2, 15.03.2007 57 Submit ROSETTA to the Grid /1

58. Technical Training Catania, PI2S2, 15.03.2007 58 Submit ROSETTA to the Grid /2 Inspect the status of the Parametric Job

59. Technical Training Catania, PI2S2, 15.03.2007 59 Inspect the status /2

60. Technical Training Catania, PI2S2, 15.03.2007 60 Data Spooler /1

61. Technical Training Catania, PI2S2, 15.03.2007 61 UNOSAT - PROJECT

62. Technical Training Catania, PI2S2, 15.03.2007 62 What is UNOSAT The goal of UNOSAT is to make satellite images and geographic information easily accessible to the humanitarian community and to experts worldwide working to reduce disasters and plan sustainable development. To do this we acquire satellite images from all commercial providers. UNOSAT is a unique cooperation initiative between the UN, science and satellite industry that ensures low-cost and high quality solutions. UNOSAT provides services in the following areas: –Satellite imagery selection and procurement assistance –Image processing –Map production –Methodological guidance –Technical assistance –Training

63. Technical Training Catania, PI2S2, 15.03.2007 63 The UnoSat Integration

64. Technical Training Catania, PI2S2, 15.03.2007 64 UnoSat Submission

65. Technical Training Catania, PI2S2, 15.03.2007 65 Monitoring

66. Technical Training Catania, PI2S2, 15.03.2007 66 GetOutPut

67. Technical Training Catania, PI2S2, 15.03.2007 67 Contents Grid Portals technology GENIUS Portal Some Integrated Applications Future perspectives

68. Technical Training Catania, PI2S2, 15.03.2007 68 Future perspectives in Genius Portal Genius will integrate a workflow engine. We are integrating an authentication-applet to execute voms-proxy-init.. Genius Portlet will be standard JSR 168 compliant ASAP a new version of Genius…

69. Technical Training Catania, PI2S2, 15.03.2007 69 References NICE web-site http://www.nice-italy.comhttp://www.nice-italy.com EnginFrame Framework http://www.enginframe.comhttp://www.enginframe.com GENIUS Portal https://genius.ct.infn.it (Reference main site)https://genius.ct.infn.it GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it GENIUS based on gLite at https://glite-tutor.ct.infn.it ************************************************************************* GENIUS Installation GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it email: –alberto.falzone@nice-italy.comalberto.falzone@nice-italy.com –nicola.venuti@nice-italy.comnicola.venuti@nice-italy.com

70. Technical Training Catania, PI2S2, 15.03.2007 70 Questions???