The GENIUS Grid Portal Giuseppe LA ROCCA INFN Catania ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia.

Slides:



Advertisements
Similar presentations
CHEP 2000, Roberto Barbera Roberto Barbera (*) GENIUS: a Web Portal for the GRID Meeting Grid.it, Bologna, (*) work in collaboration.
Advertisements

Building Portals to access Grid Middleware National Technical University of Athens Konstantinos Dolkas, On behalf of Andreas Menychtas.
ETSI Grid – Sophia Antipolis – 2005/05/24 A-WARE summary for ETSI Andrea Rodolico CTO, NICE srl, Italy.
The GENIUS Grid Portal Giuseppe LA ROCCA INFN Catania
1 CHEP 2000, Roberto Barbera Recent grid activities at INFN Catania (*) HEPiX/HEPNT 2002, Catania, (*) work in collaboration with.
 Copyright NICE srl, 2008 Workshop finale dei Progetti Grid del PON "Ricerca" Beppe Ugolotti, CEO
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
a Web interface for the GRID world Alice Off-line Meeting Erice (TP) - Italy, Sept 26th, 2000.
DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.
E-science grid facility for Europe and Latin America The GENIUS Grid Portal Riccardo Bruno INFN Catania
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
Integromics: a grid-enalbled platform for integration of advanced bioinformatics tools and data Luca Corradi Luca Corradi BIO-Lab,
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
Apache Airavata GSOC Knowledge and Expertise Computational Resources Scientific Instruments Algorithms and Models Archived Data and Metadata Advanced.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE07 Conference, Budapest, Hungary, 1-5 October ’07 Douglas McKinley
EGEE is a project funded by the European Union under contract IST GENIUS and EnginFrame Roberto Barbera Univ. of Catania and INFN SEE-GRID.
A Scalable Application Architecture for composing News Portals on the Internet Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta Famagusta.
Riccardo Bruno INFN.CT Sevilla, Sep 2007 The GENIUS Grid portal.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Grid Engine Riccardo Rotondo
 Copyright NICE srl, 2006 New Ideas for Computing Environments EnginFrame as framework for Grid enabled Web portals on Industrial and research contexts.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America GENIUS server installation and configuration.
INFSO-RI Enabling Grids for E-sciencE The GENIUS Grid portal Tony Calanducci INFN Catania - Italy First Latin American Workshop.
EGEE is a project funded by the European Union under contract IST The GENIUS portal Roberto Barbera University of Catania and INFN SEE-GRID.
 Copyright NICE srl, 2006 New Ideas for Computing Environments The Grid Portal for Industry and Research Alberto Falzone, Andrea Rodolico NICE srl
CHEP 2000, Giuseppe Andronico Grid portal based data management for Lattice QCD data ACAT03, Tsukuba, work in collaboration with A.
E-science grid facility for Europe and Latin America The GENIUS Grid Portal Giuseppe LA ROCCA INFN Catania
EGEE is a project funded by the European Union under contract IST The GENIUS portal Roberto Barbera University of Catania and INFN First Latinamerican.
November SC06 Tampa F.Fanzago CRAB a user-friendly tool for CMS distributed analysis Federica Fanzago INFN-PADOVA for CRAB team.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America The GENIUS Grid Portal Roberto Barbera Univ.
Enabling Grids for E-sciencE EGEE-III INFSO-RI Using DIANE for astrophysics applications Ladislav Hluchy, Viet Tran Institute of Informatics Slovak.
Migrating Desktop Marcin Płóciennik Marcin Płóciennik Kick-off Meeting, Santander, Graphical.
Enabling Grids for E-sciencE GENIUS and EnginFrame: A successful EGEE technology transfer Ing. Beppe Ugolotti NICE srl
Convert generic gUSE Portal into a science gateway Akos Balasko 02/07/
Holding slide prior to starting show. A Portlet Interface for Computational Electromagnetics on the Grid Maria Lin and David Walker Cardiff University.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
The GENIUS Portal Nicola Venuti – Livia Torterolo NICE srl Geneva, , 2nd Tutorial for Health e-Child Project New Ideas for Computing Environments.
Getting started DIRAC Project. Outline  DIRAC information system  Documentation sources  DIRAC users and groups  Registration with DIRAC  Getting.
E-infrastructure shared between Europe and Latin America GENIUS PORTAL Valeria Ardizzone INFN-Catania 1° EELA Grid School Itacuruçà Island, State of Rio.
EGEE is a project funded by the European Union under contract IST GENIUS and EnginFrame Roberto Barbera EGEE NA4 Generic Applications coordinator.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Application Porting INFN Giuseppe.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
Convert generic gUSE Portal into a science gateway Akos Balasko.
Application Specific Module Tutorial Zoltán Farkas, Ákos Balaskó 03/27/
EGEE is a project funded by the European Union under contract IST Enabling bioinformatics applications to.
Migrating Desktop Uniform Access to the Grid Marcin Płóciennik Poznan Supercomputing and Networking Center Poznan, Poland EGEE’07, Budapest, Oct.
Grid Execution Management for Legacy Code Architecture Exposing legacy applications as Grid services: the GEMLCA approach Centre.
Tutorial on Science Gateways, Roma, Catania Science Gateway Framework Motivations, architecture, features Riccardo Rotondo.
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
IST E-infrastructure shared between Europe and Latin America The GILDA t-Infrastructure and the GENIUS portal Christian Grunfeld,
 Copyright NICE srl, 2010 International Workshop on Science Gateways IWSG2010 A Service-Oriented Interface to the iRODS Data Grid Nicola Venuti
FESR Consorzio COMETA - Progetto PI2S2 Il portale grid: GENIUS Alberto Falzone NICE srl Consorzio COMETA – Progetto PI2S2 Tutorial.
EGEE is a project funded by the European Union under contract IST GENIUS and GILDA Guy Warner NeSC Training Team Induction to Grid Computing.
REST API to develop application for mobile devices Mario Torrisi Dipartimento di Fisica e Astronomia – Università degli Studi.
The EEPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Giuseppe Andronico, on behalf of Giuseppe LA ROCCA INFN Catania.
Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA The GENIUS Grid Portal and robot certificates Giuseppe LA ROCCA
Antonio Fuentes RedIRIS Barcelona, 15 Abril 2008 The GENIUS Grid portal.
Enabling Grids for E-sciencE Portals and Workflows: GENIUS and TRIANA Alberto Falzone NICE srl Bioinfogrid Meeting Bari,
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Convert generic gUSE Portal into a science gateway Akos Balasko.
GENIUS and EnginFrame GENIUS and EnginFrame: status and perspective The GENIUS Grid Portal: status and perspective Andrea Rodolico NICE srl
Operations Management Board 19th Dec. 2013
Giuseppe LA ROCCA INFN - Catania, Italy
Design and Manufacturing in a Distributed Computer Environment
The GENIUS portal Roberto Barbera University of Catania and INFN
GENIUS and EnginFrame: what’s next in the Grid Portal business
The GENIUS portal and the GILDA t-Infrastructure
Grid Engine Diego Scardaci (INFN – Catania)
The GILDA t-Infrastructure and the GENIUS portal
Il Portale GENIUS Alberto Falzone NICE srl Primo Tutorial TriGrid VL
Presentation transcript:

The GENIUS Grid Portal Giuseppe LA ROCCA INFN Catania ACGRID-II School 2-14 November 2009 Kuala Lumpur - Malaysia

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions Outline

Grid technology allows users to share a wide pletora of distributed computational resources regardless of their geographical location. Virtual services are exposed to the users through rather complex Command Line Interfaces or API languages. Grid security is indeed based on the Public Key Infrastructure (PKI) of X.509 certificates and the procedure to get and manage those certificates is unfortunately not straightforward; Up to now, the high security policy requested to access distributed computing resources has been a rather big limiting factor when trying to broaden the usage of Grids into a wide community of users; + +

User has to adhere to a Virtual Organization (VO) User needs an account on one of the trusted User Interface (UI) + + = Grid portals provide an added value to make Grids more appealing for non-expert users.

5 A grid portal: why and how It can be accessed from everywhere and by “everything” (desktop, laptop, PDA, cell phone). It can keep the same user interface to several back-ends. It must be redundantly “secure” at all levels: –1) secure for web transactions, –2) secure for user credentials, –3) secure for user authentication, –4) secure at VO/VOMS level. All available grid services must be incorporated in a logic way, just “one mouse click away”. Its layout must be easily understandable and user friendly.

A Grid Portal improves usability of Grids –Lowering end-user requirements for accessing the Grid –Hiding the complexity of data and job services management in the Grid A Grid Portal improves utilization of Grids –Making the Grid (r)evolution transparent to the end-user –Providing an appealing user-friendly Web interface –Enforcing Grid utilization policies Grid Portal Benefits

Interactive Applications Intranet Clients Win LX UXMac Grid / Compute Farm Internal Users Batch Applications Storage and Data Grid Portal / Gateway Project Managers Client Apps Standard protocols Licenses Home Users The Grid Portal / Gateway

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

What is EnginFrame ? It is a web-based technology able to expose Grid services running on Grid infrastructures It allows organizations to provide application- oriented computing and data services to both users (via Web browsers) and applications (via SOAP/WSDL and/or RSS) ‏ It’s a Grid gateway It greatly simplifies the development of Web Portals exposing computing services that can run on a broad range of different computational Grid systems

Spoolers HTML page Custom plugin Script Browser SDF XML EnginFrame Server HTML XSLT Grid Compute Farm Grid Compute Farm MetaFrame + NFuse MetaFrame + NFuse Application Server Application Server EnginFrame Agent Execute Service Req XML output Service Req User Authorize Groups, ACLs XML Layout XSL Service Submission EnginFrame Working Environment

gzip sample maximum medium none EF_SPOOLER_NAME="gzip $file” export EF_SPOOLER_NAME ${EF_ROOT}/plugins/lsf/bin/bsub -o output.txt gzip -$level \"$FILE\” Service example

EnginFrame snapshots Services are XML description defining –Input parameters –The action to accomplish (Unix/Windows script, Java, …)

User friendly, Application-oriented Job submission Flexible and efficient Input file management Hide complexity of underlying scheduler EF Customizable Job Submission

EF Monitoring & control Global Job monitoring Cluster & host monitoring Job details & control

Output management Data lifecycle managemnet Comprehensive output File manipulation (view, edit, delete, zip, …) Follow-up actions support RESUBMIT jobs – Rapidly edit input files and re-submit with same parameters/settings‏

16 A growing number of customers… Automotive & Industrial Equipment Audi, ARRK, Bridgestone, Bosch, Corus Automotive, Delphi, Elasis/CRF, Ferrari, Brawn GP, Jaguar-LandRover, Lear, Magneti Marelli, McLaren, P+Z, PSA, RedBull Engineering, Swagelok, Suzuki, Toyota, TRW, Volkswagen Life Sciences LitBio project, DEISA project, Biolab, Swiss Institute for Bioinformatics, Partners Healthcare, M.D. Anderson Cancer Center Energy & Utilities Addax Petroleum, AECL, Amerada Hess, British Gas, CC of Water Resources, Chevron, Conoco-Phillips, DSC-Libya, ENI/Agip, GazPromNeft, Marathon Oil, Nexen, Rosneft, Schlumberger, Sibneft, Sinopec, Slavneft, Sonatrach, Statoil, Talisman Energy, Telecom Italia, TNK-BP, TNNC, TOTAL, TyumenNIIGaz, VNIIGaz, Xinjiang Oil Research & Education ASSC, CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN2P3, ENEA, FzU, ICI, IFAE, INFN, ITEP, Harvard Business School, SSC- Russia, SDSC, Ferrara Uni, ITU, T.U.Dresden, Trinity College Dublin, Huazhong Normal Uni, Yale University, UPM High Tech STMicroelectronics, Accent, Samsung SDI, SensorDynamics, Motorola Aerospace & Manufacturing AIRBUS, Air Products and Chemicals, Procter&Gamble, Galileo Avionica, Hamilton Sunstrand, Kimberly Clark, Magellan Aerospace, MTU, Northrop Grumman, P&W, Raytheon, Simpson Strong-Tie

GENIUS is a powerful Grid Portal that allows scientists to exploit Grid resources only using a conventional Web browser It has been built on top of the EnginFrame framework It’s a gateway to European EGEE Project middle-ware It allows to expose gLite-enabled applications via Web- browser as well as Web Services What is GENIUS ?

GENIUS: Grid Preferences

20 GENIUS: Job Submission

21 GENIUS: Job Submission

Code for Job Queue management rewritten using GridML tags GENIUS: Job(s) Queue

New Confirmation Message! GENIUS: Job Retrieving

GENIUS: Data Spooler

Tight VNC GENIUS: Interactive Services

26 Local Browse on laptop Remote Browse on UI (GENIUS Server) Extended Remote File Browse on LFC Catalog GENIUS: Data Management

27 Extended Multiple Remote File Browsing on Catalog!

28 GENIUS: Workflow

29

30

31

32

33

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

VOMS Proxy Init Service A CAPTCHA Code is required to start the VOMS Proxy Applet for the proxy initialization The Java plugin or higher is mandatory required.

Jointly developed by NICE and INFN Catania

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

Robot certificates have been introduced to permit users, who are not familiar with deal personal certificates and don’t belong to any VOs, to experience the Grid paradigm for research activity and reduce the initial barriers. –They are extremely useful for instance to automate grid service monitoring, data processing production, distributed data collection systems. –Basically these certificates can be used to identify a person responsible for an unattended service or process acting as client and/or server. Robot certificates in a nutshell

In order to strong reduce the risks to have the portal certificate compromised the INFN CA decided to issue this new certificate on board of the Aladdin eToken PRO 32K smart card. Each smart card can support several robot certificates: one for each application user wants to share with the other. –An user’s PIN is prompted every time user try to read the certificate stored on the smart card to generate a proxy. –A first prototype of Grid Portal using robot certificate to generate an user’s proxy has been successfully designed. Robot certificates in a nutshell

1. ask for a service 2. create a proxy with the robot certificate 5. get the results 3. execute action 4. get output 2’,3’. track user User Admin The GENIUS Portal & Robot Certificates

The Users Tracking System (UTS) ACL-based services that enable easier access control customization for users not belonging to any group!

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

Porting the „MrBayes” application to Grid Case study from CNR - ITB

General Introduction MrBayes is a program for the Bayesian estimation of phylogeny. Bayesian inference of phylogeny is based on the posterior probability distribution of trees, which is the probability of a tree conditioned on the observations. –To approximate the posterior probability distribution of trees MrBayes uses a simulation technique called Markov Chain Monte Carlo (or MCMC). The program takes as input a character matrix in a NEXUS file format. The output is several files with the parameters that were sampled by the MCMC algorithm. The application is CPU demanding, especially if the MPI version of the software is used.

WMS LFC Catalog SE Phylogenetic analysis on large scale Robot Certificate UI + GENIUS Portal Job Submission Tool GRID

Job Submission Tool: is driven by the concept of “Task” as the applications are – Each task could be independent or could be described as depended from another “Task” – Each task is described by a “status” – The task is executed by a wrapper that takes care of monitoring the task: If the task is correctly executed the wrapper can change the status of the task from “Free” to “Done” If a single step on the job execution fails, the whole task is considered failed and automatically rescheduled JST tool takes care of submitting jobs, retrieving the output and monitoring the status of each task It is able to deal with accidental failure of grid services It is possible to change at run time the priority of each task/application JST characteristics

EF Monitoring with JST

Case study from Porting the „ASTRA” application to Grid

The ASTRA project in a nutshell The ASTRA (Ancient instrument Sound/Timbre Reconstruction Application) projects aims to reconstruct the sound or timbre of ancient instruments using archaeological data as fragments from excavations, written descriptions, pictures... The technique used is the physical modeling synthesis, a complex digital audio rendering technique which allows to recreate a model of the musical instrument and produce the sound by simulating its behavior as a mechanical system.

Computer model The Grid Network Reconstructed sounds Archaeological findings Load the sounds on a piano keyboard and play 30 sec. of audio sound => 90min. on 3.73Ghz, 2 GB RAM Modeling and computation on the Grid

How does ASTRA reconstruct the sound of the instruments ? The modeling process is known as Physical Modeling Synthesis Physical modeling creates a virtual model of the instrument and reproduces its sound by simulating its behaviour as a mechanical system. »This approach is also referred to as “synthesis by rule”. The higher is the quality of the audio files, the longer is the time required To have an idea of the needed time for simulation, on a Pentium IV 3.73 Ghz, 2GB RAM Personal Computer to correctly reproduce a sound lasting for 30 seconds it could be required more than 90 min. (image unisa.it)

ASTRA project is involved on..

GRID UI ASTRA software VOMS Server WMS User

In June 2009, the concert Fisica", was organized by the Catania Division of the Italian National Institute of Nuclear Fisica" This world premiere showcased the sounds of the Epigonion, an instrument of the past, reconstructed via computer-intensive modelling, being performed alongside real instruments such as baroque cello and percussions Purcell’sPurcell’s “The Sparrow and the Gentle Dove”, (a reconstructed Epigonion played live with percussion and baroque cello) Fisica

Introduction GENIUS/EnginFrame: new version 4.1 VOMS Proxy Init Service Robot Certificates Success Stories Summary and Conclusions

GENIUS offers the following advantages: it is a complete production-ready environment which combines the concepts of “user portal” and “science portal”; absolutely no client software needs to be installed on the user’s workstation apart from the web browser with its usual plug-ins like Java (at least JRE or higher); it provides a new unique tool to authorize users, in a very strong secure way, into the grid environment with or without VOMS support as well, easy to use; it includes support for both single and composite jobs (including DAG’s); interactive analysis and web access to personal spooling areas are possible; environment and settings customizable for the users; security for data management and sessions.

References NICE web-site EnginFrame Framework GENIUS Portal GENIUS Repository at GENIUS based on gLite at tutor.ct.infn.ithttps://glite- tutor.ct.infn.it GENIUS Installation GENIUS Repository at Write an message to italy.com or for an account request to download the GENIUS

Hands-on practicals Open your browser and connect to login=kualalumpurXX PassWord=GridKUAXX where XX = 001,..,050 Connect to the gLite User Interface