1. FP6−2004−Infrastructures−6-SSA-026409 www.eu-eela.org E-infrastructure shared between Europe and Latin America The GENIUS Grid Portal Roberto Barbera Univ. of Catania and INFN Third EELA Tutorial Rio de Janeiro, 26-30.06.2006

2. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 2 The Grid metaphor

3. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 3 UI JDL Logging & Book-keeping ResourceBroker Job Submission ServiceStorageElementComputeElement InformationService Job Status ReplicaCatalogue DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish voms-proxy-init Expanded JDL SE & CE info A typical job workflow

4. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 4 The transparent grid access local disk StorageElement StorageElementStorageElement ReplicaCatalogue UI dis k ResourceBroker ComputeElementComputeElement VOcolleague VOcolleague

5. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 5 A grid portal: why and how It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, cell phone). It can keep the same user interface to several back- ends. It must be redundantly “secure” at all levels: –1) secure for web transactions, –2) secure for user credentials, –3) secure for user authentication, –4) secure at VO/VOMS level. –5) secure in data management All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly.

6. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 6 The GENIUS hourglass model OS & Net services Basic Services High level GRID middleware ALICEATLASCMSLHCb Applications’ specific layer Other apps GLOBUS toolkit EGEE (LCG/gLite) architecture GENIUS web portal GENIUS ® (Grid Enabled web eNvironment for site Independent User job Submission) https://genius.ct.infn.it INFN/NICE collaboration

7. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 Globus middleware Compute resources EGEE middleware LCG-2 / gLite Local Data Distributed Data WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Contaiers Classic GENIUS Authentication – ACL management General XML Application Kits Data Management & Virtualization VO n - XML Application Kit VO 1 - XML Application Kit Monitoring & Accounting VNC remote Desktop over SSL MyProxy auth. w/ VOMS extensions GENIUS architecture End users Presentation engine Portlet GW WS GW RSS GW

8. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 8 GENIUS inside Powered by EnginFrame 4.1 !! The Stack on the User Interface: –Apache + Mod_JK2 ; –Mod_SSL for secure encrypted connections ; –Tomcat as servlets container ; –EnginFrame 4.1 as core of 3-tier model Thanks to New EF 4.1 features, NOW GENIUS Portal can export its services in JSR168 as portlets! New Flexibility of EF 4.1 improves the ways to write the code for grid integrations: plugins as independent modules Improved virtualization of Remote File Browsing A power GridML has been created for Job details

9. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 9 GENIUS: how it works Apache + Tomcat EnginFrame GENIUS https+java/xml+rfb WEB Browser gLite UI Local WS the Grid M/W+GSI 3-tier model

10. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 10 GENIUS Portal Reference Web Site: https://genius.ct.infn.it

11. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 11 GENIUS: file management SSL Encryption !

12. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 12 GENIUS: file management OS authority Both authorities os and genius are distinct: we have services with differen authority!

13. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 13 GENIUS: file management Remote File Browser On the User Interface

14. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 14 GENIUS: file management

15. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 15 GENIUS: file management

16. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 16 GENIUS: file management Multiple Action in a Single Service View/Edit a File With this service you may edit a file in your home directory. <ef:option id="filetoedit" label="Select File to View/Edit" type="rfb" base="${HOME}" target="@genius"/> $EF_ROOT/plugins/genius/bin/geniusgrid.sh view-file $EF_ROOT/plugins/genius/bin/geniusgrid.sh edit-file

17. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 17 GENIUS: file management HTML fragment for better grouping of options in a service, i.e.

18. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 18 GENIUS: Grid Authentication with Myproxy myproxy-get-delegation

19. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 Grid authentication with MyProxy UI Local WS MyProxy Server GENIUS Server (UI) grid-proxy-init myproxy-init any grid service myproxy-get-delegation output the Grid execution WEB Browser Now, VOMS Extensions needed to run jobs on the GRID ( --voms )

20. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 20 GENIUS: security

21. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 21 GENIUS: security Transparent VOMS Support using the current proxy after authentication

22. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 22 GENIUS: preferences

23. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 23 GENIUS: preferences

24. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 24 GENIUS: preferences

25. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 25 GENIUS: preferences Custom MyProxy Server List Management

26. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 26 GENIUS: Job Submission Preferences are on the horinzontal bar after successful login to the Grid (under genius authority)

27. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 27 GENIUS: Job Submission

28. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 28 GENIUS: Job Submission

29. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 29 GENIUS: Job Submission

30. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 EF XML Dialect sample: job-submit service Job Submission Please, select a JDL file to submit. You may also choose a Resource Broker to connect to. $EF_ROOT/plugins/genius/bin/geniusgrid.sh job-submit-next Now you may also choose a specific Computing Element for your job. $EF_ROOT/plugins/genius/bin/geniusgrid.sh job-submit

31. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 31 GENIUS: Job Queue

32. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 32 GENIUS: Job Queue New Confirmation Message!

33. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 33 GENIUS: Job Queue - Spooler After get-output operation on the grid, the user is redirected to the spooler temporary area, ready to download on laptop. The spooler management has been improved on EF 4.1

34. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 34 GENIUS: Job Queue - Spooler Actions can be performed as facilities in downloading operations!

35. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 35 GENIUS: Job Queue - Spooler

36. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 36 GENIUS: Interactive Integrated Tight VNC for the first time during early GENIUS development!

37. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 37 GENIUS: Interactive

38. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 38 GENIUS: Data & Grid Catalogs Local Browse on laptop Remote Browse on UI (GENIUS Server) Extended Remote File Browse on Fireman/LFC Catalog

39. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 39 GENIUS: Data & Grid Catalogs Extended Multiple Remote File Browsing on Catalog!

40. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 40 GENIUS: Data & Grid Catalogs

41. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 41 GENIUS: Data & Grid Catalogs The download process clones on EF Spooler the tree present on Catalog starting from root directory.

42. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 42 GENIUS: Data & Grid Catalogs

43. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 43 GENIUS: Data & Grid Catalogs

44. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 44 GENIUS: Data & Grid Catalogs

45. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 45 DAG Jobs DAG job is a set of jobs where the input, output, or execution of one or more jobs depends on one or more other ones Dependencies are represented through Directed Acyclic Graphs, where the nodes are graphs, and the edges identify the dependencies nodeA nodeBnodeC NodeF nodeD

46. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 46 DAG Jobs [ type = "dag"; max_nodes_running = 4; nodes = [ nodeA = [ file ="nodes/nodeA.jdl" ; ]; nodeB = [ file ="nodes/nodeB.jdl" ; ]; nodeC = [ file ="nodes/nodeC.jdl" ; ]; nodeF = [ file ="nodes/nodeF.jdl"; ]; dependencies = { {nodeA, nodeB}, {nodeA, nodeC},{nodeA, nodeF}, { {nodeB,nodeC,nodeF}, nodeD } } ]; ; ] Node description could be done also here, instead of using separate file

47. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 47 GENIUS & TRIANA Graphic Workflow Editor

48. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 48 GENIUS & TRIANA Graphic Workflow Editor

49. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 49 GENIUS & TRIANA Graphic Workflow Editor

50. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 50 GENIUS & TRIANA Graphic Workflow Editor

51. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 51 GENIUS & TRIANA Graphic Workflow Editor

52. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 52 GENIUS & TRIANA Graphic Workflow Editor

53. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 53 GENIUS & TRIANA Graphic Workflow Editor

54. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 54 GENIUS & TRIANA Graphic Workflow Editor

55. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 55 GENIUS & TRIANA Graphic Workflow Editor

56. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 56 GENIUS & TRIANA Graphic Workflow Editor

57. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 57 GENIUS & TRIANA Graphic Workflow Editor

58. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 58 GENIUS & TRIANA Graphic Workflow Editor

59. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 59 GENIUS & TRIANA Graphic Workflow Editor

60. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 60 GENIUS & TRIANA Graphic Workflow Editor

61. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 61 GENIUS & TRIANA Graphic Workflow Editor

62. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 62 GENIUS & TRIANA Graphic Workflow Editor

63. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 63 GENIUS & TRIANA Graphic Workflow Editor

64. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 64 GENIUS & TRIANA Graphic Workflow Editor

65. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 65 Sample App: PATSearch

66. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 66 Sample App: PATSearch

67. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 67 Sample App: PATSearch

68. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 68 Sample App: PATSearch

69. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 69 Sample App: PATSearch

70. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 70 Sample App: PATSearch

71. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 71 Sample App: PATSearch

72. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 72 Sample App: PATSearch

73. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 73 Sample App: CODESA-3D

74. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 74 Sample App: CODESA-3D

75. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 75 Sample App: CODESA-3D

76. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 76 Sample App: CODESA-3D Final Output

77. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 GENIUS Installation on UI GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it Write an email message to alberto.falzone@nice-italy.com for an account request to download the GENIUS packagealberto.falzone@nice-italy.com root@gridba1:/opt/genius \>./genius_installer.sh Performing the GENIUS Installation process, please wait. Adding efnobody user Please, maintain this user GENIUS/EnginFrame Portal installed under /opt/genius path. Please: 1. Copy your server certificate (public key) under /opt/genius/apache/conf/ssl.crt and and (private key) under /opt/genius/apache/conf/ssl.key 2. Replace the host certificate names in /opt/genius/apache/conf/ssl.conf (yourservercertificate), and in /opt/genius/etc/genius.conf files. 3. Copy the EnginFrame license key file you have obtained by NICE srl company with /opt/genius/enginframe/license/license.ef filename 4. Open on your iptables firewall the following TCP ports in INPUT: 443 (https), 5901:5999 and 7801:7900 ranges. 5. Please, start GENIUS Portal service with following command: /etc/init.d/genius start6. Now GENIUS Portal can be reached at https://gridba1.ba.infn.it GENIUS Team. root@gridba1:/opt/genius \>

78. FP6−2004−Infrastructures−6-SSA-026409 E-infrastructure shared between Europe and Latin America Rio de Janeiro, Third EELA Tutorial, 26-30.06.2006 GENIUS References GENIUS Portal https://genius.ct.infn.it (Reference site)https://genius.ct.infn.it EnginFrame Framework http://www.enginframe.comhttp://www.enginframe.com GENIUS Repository at https://geniuscvs.ct.infn.ithttps://geniuscvs.ct.infn.it GENIUS based on gLite for GILDA at https://glite-tutor.ct.infn.ithttps://glite-tutor.ct.infn.it