1 Globe adapted from wikipedia/commons/f/fa/ Globe.svg IDGF-SP International Desktop Grid Federation - Support Project SZTAKI.

Slides:



Advertisements
Similar presentations
P-GRADE and WS-PGRADE portals supporting desktop grids and clouds Peter Kacsuk MTA SZTAKI
Advertisements

WS-PGRADE: Supporting parameter sweep applications in workflows Péter Kacsuk, Krisztián Karóczkai, Gábor Hermann, Gergely Sipos, and József Kovács MTA.
1 Application Specific Module for P-GRADE Portal 2.7 Application Specific Module overview Akos Balasko MTA-SZTAKI LPDS
Workflows Information Flows Prof. Silvia Olabarriaga Dr. Gabriele Pierantoni.
“Cloud bursting” on SZTAKI Cloud Attila Csaba Marosi Cloud Computing Research Group MTA SZTAKI LPDS 1 Summer School on Grid.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI WS-PGRADE/gUSE Supporting e-Science communities in Europe Zoltan Farkas.
Connecting Workflow-Oriented Science Gateways to Multi-Cloud Systems Zoltán Farkas, Péter Kacsuk, Ákos Hajnal MTA SZTAKI.
CloudBroker integration to WS- PGRADE/gUSE Zoltán Farkas MTA SZTAKI LPDS
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI gUSE Services Remote API, DCI Bridge, Data Bridge, Robot Certificate Zoltán.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI CloudBroker Platform integration into WS-PGRADE/gUSE Zoltán Farkas MTA.
The EDGeS project receives Community research funding 1 EDGeS Bridge technology to interconnect EGEE and BOINC grids Peter Kacsuk MTA.
EDGI European Desktop Grid Initiative EDGI Portal usage and use-case studies EDGI is supported by the FP7 Capacities Programme under contract.
1 Developing domain specific gateways based on the WS- PGRADE/gUSE framework Peter Kacsuk MTA SZTAKI Start date: Duration:
Sharing, integrating and executing different workflows in heterogeneous multi-cloud systems Peter Kacsuk MTA SZTAKI SCI-BUS is supported.
The EDGI project receives Community research funding 1 EDGI Brings Desktop Grids To Distributed Computing Interoperability Etienne URBAH
07/06/11 New Features of WS-PGRADE (and gUSE) 2010 Q Q2 Miklós Kozlovszky MTA SZTAKI LPDS.
From P-GRADE to SCI-BUS Peter Kacsuk, Zoltan Farkas and Miklos Kozlovszky MTA SZTAKI - Computer and Automation Research Institute of the Hungarian Academy.
GILDA testbed GILDA Certification Authority GILDA Certification Authority User Support and Training Services in IGI IGI Site Administrators IGI Users IGI.
1 IDGF International Desktop Grid Federation How can you benefit from joining IDGF? Hannover, Peter Kacsuk, MTA SZTAKI, EDGI.
Sharing Workflows through Coarse-Grained Workflow Interoperability : Sharing Workflows through Coarse-Grained Workflow Interoperability G. Terstyanszky,
Introduction to SHIWA Technology Peter Kacsuk MTA SZTAKI and Univ.of Westminster
The EDGeS project receives Community research funding 1 SG-DG Bridges Zoltán Farkas, MTA SZTAKI.
The EDGeS project receives Community research funding 1 Bridging EGEE to BOINC and XtremWeb GIN : From interoperation to interoperability.
Introduction to WS-PGRADE and gUSE Tutorial Akos Balasko 04/17/
Supported by EU projects 12/12/2013 Athens, Greece Open Data in Agriculture Hands-on with data infrastructures that can power your agricultural data products.
WS-PGRADE portal and its usage in the CancerGrid project M. Kozlovszky, P. Kacsuk Computer and Automation Research Institute of the Hungarian Academy of.
Convert generic gUSE Portal into a science gateway Akos Balasko 02/07/
AgINFRA science gateway for workflows and integrated services 07/02/2012 Robert Lovas MTA SZTAKI.
The EDGeS project receives Community research funding 1 EDGeS infrastructure for the EGEE user community Peter Kacsuk MTA SZTAKI.
The EDGeS project receives Community research funding 1 Porting Applications to the EDGeS Infrastructure A comparison of the available methods, APIs, and.
How to Read gUSE Documents Orange Docs Series for General Pruposes RELEASE ISSUE POLICY LICENSE HOW TO READ GUSE DOCUMENTS GUSE IN A NUTSHELL by Tibor.
02/06/2014, ESRF – Grenoble Potential connections to e-Infrastructure projects …focusing on science gateways and volunteer computing Robert Lovas Institute.
Conference name Company name INFSOM-RI Speaker name The ETICS Job management architecture EGEE ‘08 Istanbul, September 25 th 2008 Valerio Venturi.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Services for advanced workflow programming.
1 IDGF International Desktop Grid Federation How can you benefit from joining IDGF? Lyon, Peter Kacsuk, MTA SZTAKI, EDGI is.
Convert generic gUSE Portal into a science gateway Akos Balasko.
SHIWA and Coarse-grained Workflow Interoperability Gabor Terstyanszky, University of Westminster Summer School Budapest July 2012 SHIWA is supported.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Accessing Cloud Systems from WS-PGRADE/gUSE Zoltán Farkas MTA SZTAKI LPDS.
11 Introduction to EDGI Peter Kacsuk, MTA SZTAKI Start date: Duration: 27 months EDGI.
SHIWA: Is the Workflow Interoperability a Myth or Reality PUCOWO, June 2011, London Gabor Terstyanszky, Tamas Kiss, Tamas Kukla University of Westminster.
One Click Cloud Orchestrator (OCCO): bringing Complex Complete Infrastructures Effortlessly to IaaS Clouds Peter KACSUK, Adam VISEGRADI, Jozsef KOVACS.
Application Specific Module Tutorial Zoltán Farkas, Ákos Balaskó 03/27/
1 Further information and next steps Further information and next steps Gergely Sipos MTA SZTAKI
1 WS-PGRADE/gUSE generic DCI gateway framework for EGI user communities Zoltan Farkas and Peter Kacsuk MTA SZTAKI SCI-BUS is supported.
Introduction to the program of the summer school Peter Kacsuk MTA SZTAKI SCI-BUS is supported by the FP7 Capacities Programme under contract.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI MTA SZTAKI background for the DARIAH CC Zoltan Farkas MTA SZTAKI LPDS,
OpenNebula: Experience at SZTAKI Peter Kacsuk, Sandor Acs, Mark Gergely, Jozsef Kovacs MTA SZTAKI EGI CF Helsinki.
EGI Technical Forum Madrid COMPSs in the EGI Federated Cloud Daniele Lezzi – BSC EGI Technical Forum Madrid.
Cloud-enabled, scalable Data Avenue service to process very large, heterogeneus data Péter Kacsuk, Ákos Hajnal MTA SZTAKI Francesco Tusa, Junaid Arshad.
SHIWA Simulation Platform (SSP) Gabor Terstyanszky, University of Westminster EGI Community Forum Munnich March 2012 SHIWA is supported by the FP7.
Usage of WS-PGRADE and gUSE in European and national projects Peter Kacsuk 03/27/
1 International Desktop Grid Federation at the fingertips of EGI scientists Revised version for ‘Promoting Desktop Grids’ virtual team status report Robert.
RI EGI-TF 2010, Tutorial Managing an EGEE/EGI Virtual Organisation (VO) with EDGES bridged Desktop Resources Tutorial Robert Lovas, MTA SZTAKI.
11 Extending EMI middleware with DGs Peter Kacsuk, MTA SZTAKI Start date: Duration:
Occopus and its usage to build efficient data processing workflow infrastructures in clouds József Kovács, Péter Kacsuk, Ádám Novák, Ádám Visegrádi MTA.
New developments of gUSE & WS-PGRADE to support e-science gateways Peter Kacsuk and Miklos Kozlovszky MTA SZTAKI - Computer and Automation Research Institute.
DIRAC for Grid and Cloud Dr. Víctor Méndez Muñoz (for DIRAC Project) LHCb Tier 1 Liaison at PIC EGI User Community Board, October 31st, 2013.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Accessing cloud resources through the WS-PGRADE/gUSE and CloudBroker integrated.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf.
OCCI-compliant Occopus orchestrator and experiences with using it with the EGI Federated Cloud Jozsef Kovacs and Peter Kacsuk MTA SZTAKI.
Convert generic gUSE Portal into a science gateway Akos Balasko.
Convert generic gUSE Portal into a science gateway Akos Balasko.
Exposing WS-PGRADE/gUSE for large user communities Peter Kacsuk, Zoltan Farkas, Krisztian Karoczkai, Istvan Marton, Akos Hajnal,
SHIWA SIMULATION PLATFORM = SSP Gabor Terstyanszky, University of Westminster e-Science Workflows Workshop Budapest 09 nd February 2012 SHIWA is supported.
Flowbster: Dynamic creation of data pipelines in clouds
WS-PGRADE for Molecular Sciences and XSEDE
Peter Kacsuk MTA SZTAKI
DARIAH Competence Centre: architecture and activity summary
Introduction to the SHIWA Simulation Platform EGI User Forum,
Presentation transcript:

1 Globe adapted from wikipedia/commons/f/fa/ Globe.svg IDGF-SP International Desktop Grid Federation - Support Project SZTAKI Use Cases to dynamically create gLite and BOINC based DCIs in the EGI FedCloud Peter Kacsuk, Sandor Acs, Peter Kotzauer, Jozsef Kovacs

2 Private network gLite UI 3G Bridge BOINC Server WMSMCE WS- PGRADE Portal Use Case 1 architecture (SCI-BUS) First successful experiment on CESNET and SZTAKI Clouds gLite and BOINC infrastructure dynamically deployed in the same cloud in order to test WS-PGRADE portal with gLite and BOINC BOINC client

3 Portal CESNET Cloud GWDG Cloud SZTAKI Cloud DCI Bridge / Glite UI Glite VOMS Glite mCE BOINC server +client Glite WMS Virtual Private network 3 gLite and BOINC infrastructure deployed on different cloud providers Extended Use Case 1 architecture with 3 clouds

4 IDGF-SP 17/09/ Use case 2 for gLite VO communities (EDGI, IDGF-SP)  Extend existing gLite VOs with desktop grids having large number of clients  Two options: – The desktop grid is maintained by IDGF – The desktop grid is set up by the gLite VO community  We show the second option as use case for EGI CloudFed

5 IDGF-SP 17/09/ SZTAKI Desktop Grid server gLite VO modified gLite CREAM CE 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC The scenario to implement: using dynamically instantiated desktop grid resources through gLite EGI Cloud

6 IDGF-SP 17/09/ Result  If you want your own dynamically instantiated computational “cluster” – Setup a gLite CE with EDGI executor (mCE) – Instantiate the DG server image in the FedCloud – Adjust the mCE’s configuration – …and submit your jobs! – Due to the GBAC virtualization any application can run in the DG

7 IDGF-SP 17/09/ If the cloud resources are not enough you can ask volunteers to join your project BOINC client Virtualisation by GBAC volunteers SZTAKI Desktop Grid server gLite VO modified gLite CREAM CE 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC EGI Cloud

8 IDGF-SP 17/09/ Use Case 3 for biologists using autodock (SCI-BUS, IDGF-SP)  Biologist can create the necessary infrastructure in the cloud by one click in order to run an autodock experiment  The created infrastructure contains: – An autodock gateway (customized from WS- PGRADE) – A desktop grid with 2 clients  Biologist can specify the molecule files and initiate the autodock calculation via the autodock gateway  Volunteers can connect if more clients are needed  Grid certificate is not required

9 IDGF-SP 17/09/ SZTAKI Desktop Grid server WS-PGRADE Autodock gateway 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC Autodock gateway with BOINC desktop Grid in the cloud EGI Cloud Since biologists have no idea how to create such a cloud system the whole infrastructure is created by 1 click

10 IDGF-SP 17/09/ Generic architecture of the 1-click solution Cloud BOINC Server Client BOINC Clients DG Infrastructure CO framework Queue handler DB Infrastructure Queue Service Description Language Orchestrator Queue Messaging Infrastructure handler VM Handlers Cloud Interfaces Cloud Interfaces Tests & Test Descriptions Tests & Requirement s Handler Executors User Username;Password; gUSE Username;Password;3GB_ URL; Username;Password; BOINC_URL;

11 IDGF-SP 17/09/ Use Case 4 for agINFRA project members  1-Click cloud infrastructure  The current implementation deploys a – workflow engine (and WS-PGRADE based GUI) with access to external clouds, grids, and data components as well, – with Remote API to provide external access e.g. from a DRUPAL module, – an extensible Distributed Computing Infrastructure based on Desktop Grid server, and workers, – one BioVEL application example (BBGC)  Grid certificate is not required.

12 IDGF-SP 17/09/ VM2 VM1 WS-PGRADE WF Developer UI gUSE workflow engine DCI Bridge WS-PGRADE End-User UI Remote API A B BES interface BBGC Workflow (BIOVEL) SZTAKI Desktop Grid server 3G Bridge BOINC project BBGC Binaries BBGC APP VM4 BOINC client BBGC job VM3 BOINC client BBGC job External access from other tools/GUIs BBGC job BOINC client External Grid / Cloud resources, agINFRA data components, registries (CIARD RING) Use Case 4 for agINFRA project members Volunteers

13 IDGF-SP 17/09/ Conclusions  The 1-click concept enables the set up of various DCI infrastructures for various types of communities  Very important to provide a centralized marketplace from where the required DCI service images can be accessed  Gradually, new and new DCI infrastructure images can be provided and used by the 1-click solution according to the needs of various user communities  Further topics interesting for the before mentioned communities: – Brokering, auto scaling, market place development  Based on the previous experiences SZTAKI is ready to actively work in these areas and to provide support for different user communities