Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari,

Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari, 09.03.2006

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 2 Contents EnginFrame: the Framework for Computing Portals –A lot of history from 3.2 to 4.1 version –The power and the flexibility in computing portals design GENIUS Portal powered by Enginframe: new features, improvements –The Changes in the backend and the stack on the gLite UI –New Features, Improvements –Management of User Security –The JSR168 compliance from GENIUS/EnginFrame DAG Jobs and TRIANA from GENIUS Portal A Sample of integrated Application Summary and conclusions Q & A

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 3 A grid portal: why and how It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, cell phone). It can keep the same user interface to several back- ends. It must be redundantly “secure” at all levels: –1) secure for web transactions, –2) secure for user credentials, –3) secure for user authentication, –4) secure at VO/VOMS level. All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly.

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 4 The Web-centric vision Licenses & Data Grid / Compute Farm Interactive Apps Inter/Intranet Server(s) Clients Standard Web Browser

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Data Grid (AFS, SRB, …) Storage Compute Grid (Globus, LSF, SGE, …) Internal HW/SW On-demand HW/SW WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Containers Skins / Themes Authentication – ACL management Custom XML Application Kits Data Management & Virtualization ISV n - XML Application Kit ISV 1 - XML Application Kit Accounting / Billing GUI Virtualization Single-Sign-On EnginFrame as Grid Gateway End users Presentation engine Portlet GW WS GW RSS GW

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Globus middleware Compute resources EGEE middleware LCG-2 / gLite Local Data Distributed Data WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Contaiers Classic GENIUS Authentication – ACL management General XML Application Kits Data Management & Virtualization VO n - XML Application Kit VO 1 - XML Application Kit Monitoring & Accounting VNC remote Desktop over SSL MyProxy auth. w/ VOMS extensions GENIUS architecture End users Presentation engine Portlet GW WS GW RSS GW

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 EnginFrame evolution 1.x - LAN Integration (1998)  Unix-Windows integration for simplified parallel and distributed computing 2.x - WAN MetaComputing interface (1999)  Standards convergence, dynamic contents management 3.x - Computing Portal (2000-2003)  Web architecture, JS independence, Globus integration 3.5 – Enterprise Grid Portal (2004)  Fault tolerance, Auth delegation, Authorization, EP integration 4.x – Enterprise Grid Gateway (2005)  WebServices, user mapping, enhanced Data management, RSS, JSR168

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Who uses EnginFrame? Mechanical – Ferrari, Audi, BMW, FIAT Auto, Elasis, Magneti Marelli, P+Z, Swagelok, Toyota, TRW Manufacturing – Bridgestone, Procter & Gamble, Galileo Avionica Oil&Gas – Slavneft, Schlumberger, TOTAL, VNIIGaz Electronics – STMicroelectronics, Accent, SensorDynamics, Motorola Biotech – ENEA, EGEE LS community Telecom – Telecom Italia Research – INFN, ASSC, CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN2P3, ENEA, FzU, ICI, IFAE, ITEP, JSC G.G.M., KU Leuven, SSC-Russia, SDSC Education – Dresda University, Ferrara University, ITU, Messina University, Politecnico of Milan, Technische Universität Dresden, Trinity College Dublin, Salerno University, S-PACI

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Application integrations

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 3D preview integration

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Corporate ASPs

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 HPC demo-center & ASP http://www.GridAge.com

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Joint activities…

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 E-Design collaboration Black-box for EDA simulation

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 ACL management Authorization system End users ServicesOption setsField contentsXML output

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 The Authorization Framework Authorization System is aimed to authorize user accesses to resources –allowing or denying operations –according to a set of predefined policies. Actor concept abstracts users. An actor can represent –single user –group of users EnginFrame resources are: –folders –services –service options –service action –service output Authorization policies are defined by Access Control Lists

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 17 GENIUS: new version Powered by EnginFrame 4.1 !! The Stack on the User Interface: –Apache + Mod_JK2 ; –Mod_SSL for secure encrypted connections ; –Tomcat as servlets container ; –EnginFrame 4.1 as core of 3-tier model Thanks to New EF 4.1 features, NOW GENIUS Portal can export its services in JSR168 as portlets! New Flexibility of EF 4.1 improves the ways to write the code for grid integrations: plugins as independent modules Improved virtualization of Remote File Browsing A power GridML has been created for Job details

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 18 GENIUS: how it works Apache + Tomcat EnginFrame GENIUS https+java/xml+rfb WEB Browser gLite UI Local WS the Grid M/W+GSI 3-tier model

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 19 The GENIUS hourglass model OS & Net services Basic Services High level GRID middleware ALICEATLASCMSLHCb Applications’ specific layer Other apps GLOBUS toolkit EGEE (LCG/gLite) architecture GENIUS web portal GENIUS Grid Enabled web eNvironment for site Independent User job Submission INFN / NICE Collaboration

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 20 GENIUS: layout restyling

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 21 GENIUS: layout restyling SSL Encryption !

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 22 GENIUS: files management OS authority Both authorities os and genius are distinct: we have services with different authority!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 23 GENIUS: files management Remote File Browser On the User Interface

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 24 GENIUS: files management

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 26 GENIUS: files management Multiple Action in a Single Service View/Edit a File With this service you may edit a file in your home directory. <ef:option id="filetoedit" label="Select File to View/Edit" type="rfb" base="${HOME}" target="@genius"/> $EF_ROOT/plugins/genius/bin/geniusgrid.sh view-file $EF_ROOT/plugins/genius/bin/geniusgrid.sh edit-file

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 28 GENIUS: files management HTML fragment for better grouping of options in a service, i.e.

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 29 GENIUS: Grid Authentication with Myproxy myproxy-get-delegation

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Grid authentication with MyProxy UI Local WS MyProxy Server GENIUS Server (UI) grid-proxy-init myproxy-init any grid service myproxy-get-delegation output the Grid execution WEB Browser Now, VOMS Extensions needed to run jobs on the GRID ( --voms )

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 31 GENIUS: security Transparent VOMS Support

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 32 GENIUS: security

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 36 GENIUS: preferences

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 39 GENIUS: preferences Custom MyProxy Server List Management

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 41 GENIUS: Job Editor for jdl files

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 42 GENIUS: Job Submission Preferences are on the horinzontal bar after successful login to the Grid (under genius authority)

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 43 GENIUS: Job Submission

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 46 GENIUS: Job Queue Code for Job Queue management rewritten using GridML tags

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 GridML sample: job andrea default DONE priority /usr/bin/spamc mail $HOME mail 0.0 0

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 48 GENIUS: Job Queue New Confirmation Message!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 49 GENIUS: Job Queue - Spooler After get-output operation on the grid, the user is redirected to the spooler temporary area, ready to download on laptop. The spooler management has been improved on EF 4.1

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 50 GENIUS: Job Queue - Spooler Actions can be performed as facilities in downloading operations!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 51 GENIUS: Job Queue - Spooler

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 52 GENIUS: Interactive Integrated Tight VNC for the first time during early GENIUS development!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 53 GENIUS: Interactive

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 54 GENIUS: Data & Grid Catalogs Local Browse on laptop Remote Browse on UI (GENIUS Server) Extended Remote File Browse on Fireman/LFC Catalog

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 55 GENIUS: Data & Grid Catalogs Extended Multiple Remote File Browsing on Catalog!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 56 GENIUS: Data & Grid Catalogs

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 57 GENIUS: Data & Grid Catalogs The download process clones on EF Spooler the tree present on Catalog starting from root directory.

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 61 DAG Jobs DAG job is a set of jobs where the input, output, or execution of one or more jobs depends on one or more other ones Dependencies are represented through Directed Acyclic Graphs, where the nodes are graphs, and the edges identify the dependencies nodeA nodeBnodeC NodeF nodeD

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 62 DAG Jobs [ type = "dag"; max_nodes_running = 4; nodes = [ nodeA = [ file ="nodes/nodeA.jdl" ; ]; nodeB = [ file ="nodes/nodeB.jdl" ; ]; nodeC = [ file ="nodes/nodeC.jdl" ; ]; nodeF = [ file ="nodes/nodeF.jdl"; ]; dependencies = { {nodeA, nodeB}, {nodeA, nodeC},{nodeA, nodeF}, { {nodeB,nodeC,nodeF}, nodeD } } ]; ; ] Node description could be done also here, instead of using separate file

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 63 GENIUS & TRIANA Graphic Workflow Editor

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 81 EnginFrame Portlets: How it works GENIUS Server (UI) GridSphere portlet container + EF Portlets https + genius auth Output of results in JSR168 Instance of portlets for any required service Every Genius Service can be considered as a singol atom to build a specific custom portal based on a portlet container like GridSphere that can dialog with the EGEE Grid Middleware !

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 82 GENIUSphere: files Files on the User Interface, where GENIUS is running!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 83 GENIUSphere: Jobs

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 84 GENIUSphere: Jobs

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 85 GENIUSphere: Jobs - Spoolers

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 86 Sample App: PATSearch

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 GENIUS Installation on UI GENIUS Repository at https://geniuscvs.ct.infn.it root@gridba1:/opt/genius \>./genius_installer.sh Performing the GENIUS Installation process, please wait. Adding efnobody user Please, maintain this user GENIUS/EnginFrame Portal installed under /opt/genius path. Please: 1. Copy your server certificate (public key) under /opt/genius/apache/conf/ssl.crt and and (private key) under /opt/genius/apache/conf/ssl.key 2. Replace the host certificate names in /opt/genius/apache/conf/ssl.conf (yourservercertificate), and in /opt/genius/etc/genius.conf files. 3. Copy the EnginFrame license key file you have obtained by NICE srl company with /opt/genius/enginframe/license/license.ef filename 4. Open on your iptables firewall the following TCP ports in INPUT: 443 (https), 5901:5999 and 7801:7900 ranges. 5. Please, start GENIUS Portal service with following command: /etc/init.d/genius start6. Now GENIUS Portal can be reached at https://gridba1.ba.infn.it GENIUS Team. root@gridba1:/opt/genius \>

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Important things TO DO in next release Add the VOMS compliance on MyProxy Init service to initialize the MyProxy directly from Portal in secure way Support new Job Collection type Improve the management of users profiles Open to collaborate for: –Integration of the Metadata technologies regarding Catalogs and DBs (AMGA, OGSA-DAI, G-DSE) –Integration in deep within EnginFrame framework of WorkFlow technologies like Taverna and Moteur Suggestions and feedback from users are very welcome!

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 Grid Gateway User Group proposal Open community for GENIUS and EnginFrame developers –Info forum on major happenings in the Grid Gateway arena –Blog / newsgroup for technical discussions Repository for portlet / plug-in developers –Industrial plug-ins for EnginFrame –Open source plug-ins for GENIUS –Other contributed components (e.g. GridSphere portlets, etc.) Volunteers are welcome! –Please contact:  roberto.barbera@ct.infn.it  livia@bio.dist.unige.it  info@nice-italy.com

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 97 Summary & Conclusions GENIUS Portal as a flexible and powerful tool to build comprhensive front-end of integrate applications for the end-users; It makes the web-ubiquity for users and grid-services logic-abstraction; Available many ways to manage user’s authentication (OS, GENIUS à la myproxy, Kerberos5, LDAP); Grid Security is transparent managed; Improved GridML to manage job queues; Easy installation; Powered by EnginFrame, Now it is a Web Services producer; New plugins can be wrote; We are OPEN to collaborate in order to better integrate in deep Metadata and WorkFlow technologies like Taverna and Moteur

Enabling Grids for E-sciencE GENIUS and TRIANA, Alberto Falzone – NICE srl BIOINFOGRID Initial training course, Bari, 09.03.2006 98 Q & AQ&A Thanks for your attention!

Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari,

Similar presentations

Presentation on theme: "Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari,"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari,

Similar presentations

Presentation on theme: "Enabling Grids for E-sciencE www.eu-egee.org Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl www.nice-italy.com Bioinfogrid Meeting Bari,"— Presentation transcript:

Similar presentations

About project

Feedback