FESR Trinacria Grid Virtual Laboratory Il portale GENIUS, EnginFrame e strumenti per il calcolo distribuito Alberto Falzone, Beppe Ugolotti NICE srl Primo Workshop TriGrid VL Catania,

Catania, Primo Workshop TriGrid VL, Contents About NICE GENIUS Portal –The stack on the gLite UI –New Features, Improvements, Logic Abstraction –Management of User Security EnginFrame: the Framework for Computing Portals –A lot of history from 3.2 to 4.1 version –The power and the flexibility in computing portals design Tools for Distributed Computing –Platform LSF & HPC Summary and conclusions Q & A

Catania, Primo Workshop TriGrid VL, NICE Mission Help our customers to address IT governance problems in… – Application delivery in distributed environment – HW/SW monitoring & information extraction to support decisions – HW/SW optimisation & orchestration …using reliable, scalable and innovative GRID technologies

Catania, Primo Workshop TriGrid VL, NICE Background Expertise in Grid deployment –Enterprise Compute & Data Grids –3rd party products, projects, consulting –10 years experience with legacy & comm. Apps –Long partnership with Platform Computing Core business: Access to Grid Services –EnginFrame Grid Portal product line Other relevant competencies –Consulting for Grid-oriented system tuning –Server-based computing –Expertise in Grid Intelligence

Catania, Primo Workshop TriGrid VL, NICE Customers MechanicalFerrari, Audi, BMW, FIAT Auto, CRF, Elasis, Lear, Magneti Marelli, Swagelok, Toyota ElectronicsSTMicroelectronics, Accent, Alcatel, Ericsson, Micron, Siemens, SensorDynamics, Motorola Manufacturing Procter & Gamble, Pirelli, Bridgestone AerospaceAlenia, Avio, CIRA, Galileo Avionica BioTechPharmacia, (ENEA) TelecomTelecom Italia Lab EnergyAgip, Shell, TOTAL, Slavneft ResearchENEA, ICTP, INFN, CASPUR, CILEA, CNR, CINECA, CRS4, Astronomical Obs. EducationScuola Normale Superiore di Pisa, Polit. Of Milan, Universities

Catania, Primo Workshop TriGrid VL, Contents About NICE GENIUS Portal –The stack on the gLite UI –New Features, Improvements, Logic Abstraction –Management of User Security EnginFrame: the Framework for Computing Portals –A lot of history from 3.2 to 4.1 version –The power and the flexibility in computing portals design Tools for Distributed Computing –Platform LSF & HPC Summary and conclusions Q & A

Catania, Primo Workshop TriGrid VL, A grid portal: why and how It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, cell phone). It can keep the same user interface to several back- ends. It must be redundantly “secure” at all levels: –1) secure for web transactions, –2) secure for user credentials, –3) secure for user authentication, –4) secure at VO/VOMS level. All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly.

Catania, Primo Workshop TriGrid VL, The Web-centric vision Licenses & Data Licenses & Data Grid / Compute Farm Grid / Compute Farm Interactive Apps Interactive Apps Inter/Intranet Server(s) Clients Standard Web Browser

Catania, Primo Workshop TriGrid VL, Data Grid (AFS, SRB, …) Storage Compute Grid (Globus, LSF, SGE, …) Internal HW/SW On-demand HW/SW WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Containers Skins / Themes Authentication – ACL management Custom XML Application Kits Data Management & Virtualization ISV n - XML Application Kit ISV 1 - XML Application Kit Accounting / Billing GUI Virtualization Single-Sign-On EnginFrame as Grid Gateway End users Presentation engine Portlet GW WS GW RSS GW

Catania, Primo Workshop TriGrid VL, Globus middleware Compute resources EGEE middleware LCG-2 / gLite Local Data Distributed Data WSDL/SOAP 3rd party Apps HTTP RSS Clients JSR168 Portlet Contaiers Classic GENIUS Authentication – ACL management General XML Application Kits Data Management & Virtualization VO n - XML Application Kit VO 1 - XML Application Kit Monitoring & Accounting VNC remote Desktop over SSL MyProxy auth. w/ VOMS extensions GENIUS architecture End users Presentation engine Portlet GW WS GW RSS GW

Catania, Primo Workshop TriGrid VL, GENIUS: new version Powered by EnginFrame 4.1 !! The Stack on the User Interface: –Apache + Mod_JK2 ; –Mod_SSL for secure encrypted connections ; –Tomcat as servlets container ; –EnginFrame 4.1 as core of 3-tier model Thanks to New EF 4.1 features, NOW GENIUS Portal can export its services in JSR168 as portlets! New Flexibility of EF 4.1 improves the ways to write the code for grid integrations: plugins as independent modules Improved virtualization of Remote File Browsing A power GridML has been created for Job details

Catania, Primo Workshop TriGrid VL, GENIUS: how it works Apache + Tomcat EnginFrame GENIUS https+java/xml+rfb WEB Browser gLite UI Local WS the Grid M/W+GSI 3-tier model

Catania, Primo Workshop TriGrid VL, The GENIUS hourglass model OS & Net services Basic Services High level GRID middleware ALICEATLASCMSLHCb Applications’ specific layer Other apps GLOBUS toolkit EGEE (LCG/gLite) architecture GENIUS web portal GENIUS Grid Enabled web eNvironment for site Independent User job Submission INFN / NICE Collaboration

Catania, Primo Workshop TriGrid VL, GENIUS Portal Reference Web Site:

Catania, Primo Workshop TriGrid VL, Grid authentication with MyProxy UI Local WS MyProxy Server GENIUS Server (UI) grid-proxy-init myproxy-init any grid service myproxy-get-delegation output the Grid execution WEB Browser Now, VOMS Extensions needed to run jobs on the GRID, transparently managed ( --voms )

Catania, Primo Workshop TriGrid VL, GENIUS Portal

Catania, Primo Workshop TriGrid VL, GENIUS: Job Queue - Spooler

Catania, Primo Workshop TriGrid VL, GENIUS Portal in brief Virtualization of Files Management: –Local, Remote and on Catalogs (fireman and lfc on EGEE MW); Transparent Security management and Secure Access; Job Management simplified for end-users; Integration of Tight VNC improved for Interactive applications; Preferences options added to better configure User Profiles Too much Flexibility on plugins code and functions for re-usability.

Catania, Primo Workshop TriGrid VL, Contents About NICE GENIUS Portal –The stack on the gLite UI –New Features, Improvements, Logic Abstraction –Management of User Security EnginFrame: the Framework for Computing Portals –A lot of history from 3.2 to 4.1 version –The power and the flexibility in computing portals design Tools for Distributed Computing –Platform LSF & HPC Summary and conclusions Q & A

Catania, Primo Workshop TriGrid VL, EnginFrame evolution 1.x - LAN Integration (1998)  Unix-Windows integration for simplified parallel and distributed computing 2.x - WAN MetaComputing interface (1999)  Standards convergence, dynamic contents management 3.x - Computing Portal ( )  Web architecture, JS independence, Globus integration 3.5 – Enterprise Grid Portal (2004)  Fault tolerance, Auth delegation, Authorization, EP integration 4.x – Enterprise Grid Gateway (2005)  WebServices, user mapping, enhanced Data management, RSS, JSR168

Catania, Primo Workshop TriGrid VL, Relevant features Security –Flexible authentication delegation (MyProxy, NIS, LDAP, AD, AFS, GSI, …) –User mapping (one to many, many to many, many to one) –Fine grained authorization system (up to option level) –Session-level Token management –Support for HTTPS encryption Connectivity –GridML abstraction to connect to the Grid (LSF, Globus, etc.) –Portlet Integration kit for WebSphere and Oracle Portal  SSO  Portlet content publishing & rendering –Integration with Citrix NFuse for SSO and application publishing –Integration with VNC Data Management –Working directory is virtualized via the “spooler” concept –Select, add, remove, zip, tar actions on spooler data –Automatic, renewable time-to- live management for spoolers Usability –Flexible remote file/item browsing –Multiple file upload (required client-side Java 1.1 support) –Context-based parameter expansion –Xforms 1.0 support –Correlation between job & data Performance and scalability –Service output caching –Support for Web farms –Dynamic on-demand loading of spooler data

Catania, Primo Workshop TriGrid VL, Key architectural benefits Job manager independence –Grid interface virtualized via our flexible GridML –Drivers for most popular middleware Enterprise class features –Authentication delegation –Support for web farms –Integration in Enterprise Portals Rapid evolutionary implementation –Intuitive XML service description language –Many applications already integrated –Trivial integration of custom scripts & apps –Can integrate & leverage telnet/ssh approach

Catania, Primo Workshop TriGrid VL, EnginFrame Benefits Hide the complexity of Technical Computing environments Instant migration to the Computing Portal paradigm from a command based world Enhance resource Manageability for System and Application managers Enable smooth NT-UNIX-appliance integration Ease deployment of new software and methodologies Automatic and transparent enforcement of company policies Simplify resource Accessibility in the Virtual Workplace Enhance collaboration with remote sites and partners Protect your Intellectual Property Shorten Time-To-Market/Time-To-User

Catania, Primo Workshop TriGrid VL, Who uses EnginFrame? Mechanical – Ferrari, Audi, BMW, FIAT Auto, Elasis, Magneti Marelli, P+Z, Swagelok, Toyota, TRW Manufacturing – Bridgestone, Procter & Gamble, Galileo Avionica Oil&Gas – Slavneft, Schlumberger, TOTAL, VNIIGaz Electronics – STMicroelectronics, Accent, SensorDynamics, Motorola Biotech – ENEA, EGEE LS community Telecom – Telecom Italia Research – INFN, ASSC, CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN2P3, ENEA, FzU, ICI, IFAE, ITEP, JSC G.G.M., KU Leuven, SSC-Russia, SDSC Education – Dresda University, Ferrara University, ITU, Messina University, Politecnico of Milan, Technische Universität Dresden, Trinity College Dublin, Salerno University, S-PACI

Catania, Primo Workshop TriGrid VL, Application integrations

Catania, Primo Workshop TriGrid VL, Contents About NICE GENIUS Portal –The stack on the gLite UI –New Features, Improvements, Logic Abstraction –Management of User Security EnginFrame: the Framework for Computing Portals –A lot of history from 3.2 to 4.1 version –The power and the flexibility in computing portals design Tools for Distributed Computing –Platform LSF & HPC Summary and conclusions Q & A

Catania, Primo Workshop TriGrid VL, What is the Platform LSF Family of Products?

Catania, Primo Workshop TriGrid VL, How It Works - Platform LSF Load Information Manager Host Workload Manager LSF Web Services Broker Web Application Job Submission API Plugin Schedulers Cluster Workload Manager Cluster Workload Manager Job Queue Intelligent Scheduler Fairshare Preemption Resource Reservation Advance Reservation License Scheduling SLA Scheduling Service Level Agreement MultiCluster Other Scheduling Modules

Catania, Primo Workshop TriGrid VL, Software license management Platform LSF Platform LSF License Scheduler API Platform LSF License Scheduler Macrovision FlexLM

Catania, Primo Workshop TriGrid VL, Multi-site orchestration

Catania, Primo Workshop TriGrid VL, Key Features - Platform LSF High Performing, Open, Scalable Architecture –Scalable Scheduler Architecture –External executable support –OGSI compliance Intelligent Scheduling Policies –Fairshare (user & project-based) –Policy-based preemption –Goal oriented SLA scheduling –Job Groups Advanced Self-Management –Flexible, comprehensive resource definitions –Job-level exception management –Automatic job migration and requeue –Master scheduler failover Heterogeneous Platform Support Extensive Application Support Comprehensive, Extensible and Standards-based Security

Catania, Primo Workshop TriGrid VL, ISV support is expanding Electronics Industrial Manufacturing Life Sciences

Catania, Primo Workshop TriGrid VL, Case study: ROI in IT Scenario: –Industrial companies try to fill the gap with competitors –CAE becomes strategic investment Problem: –expensive IT resources (HW, licenses, storage) –low ROA, unpredictable ROI –complex for management –complex deployment

Catania, Primo Workshop TriGrid VL, Grid + Grid Portal Grid metrics Sites: 1 Applications:10+ CPUs: Users: Grid metrics Sites: 1 Applications:10+ CPUs: Users:

Catania, Primo Workshop TriGrid VL, D preview integration

Catania, Primo Workshop TriGrid VL, HPC demo-center & ASP

Catania, Primo Workshop TriGrid VL, Joint activities…

Catania, Primo Workshop TriGrid VL, Customer Case Study: Intellectual Property Scenario: –Accent is designing a chipset for Customer –Customer needs to test the firmware & software for this chipset Problem: –Intellectual Property cannot be disclosed Chipset IP + + FirmWare

Catania, Primo Workshop TriGrid VL, E-Design collaboration Black-box for EDA simulation

Catania, Primo Workshop TriGrid VL, DataGate Grid metrics Sites: 1 Applications:<10 CPUs: 2! Users: 300+ Grid metrics Sites: 1 Applications:<10 CPUs: 2! Users: 300+

Catania, Primo Workshop TriGrid VL, Grid benefits Reduced costs in data exchange –Calculated ROI on shipment savings is 4 months! Complexity reduction for internal and external users Fault and license shortage tolerance Reduced network security issues Wide accessibility via virtually any browser Bottom line: –Full control of data coming in or out of the company

Catania, Primo Workshop TriGrid VL, Grid Portal within EP Centralized eDesign Services Grid metrics Sites: 3+ Applications:20+ CPUs: <20 Users: 100+ Grid metrics Sites: 3+ Applications:20+ CPUs: <20 Users: 100+ Citrix integration

Catania, Primo Workshop TriGrid VL, Grid benefits Flexible software license sharing between sites and projects IT assets optimization and consolidation Flexible and dynamic enforcement of policies and priorities Collaborative product design across different sites Strong management of usage peaks Low client TCO Bottom line: –Enables product development in distributed design teams –Cuts inefficiencies in license management

Catania, Primo Workshop TriGrid VL, Summary & Conclusions GENIUS Portal / EnginFrame as a flexible and powerful tool to build comprhensive front-end of integrate applications for the end-users; It makes the web-ubiquity for users and grid-services logic-abstraction; Available many ways to manage user’s authentication (OS, GENIUS à la myproxy, Kerberos5, LDAP); Grid Security is transparent managed; Improved GridML to manage job queues; Platform LSF Family Products as powerful and scalable tools for HPC; Integrations available for HPC environments.

Catania, Primo Workshop TriGrid VL, Q & AQ&A Thanks for your attention!