1 Globe adapted from wikipedia/commons/f/fa/ Globe.svg IDGF-SP International Desktop Grid Federation - Support Project SZTAKI Use Cases to dynamically create gLite and BOINC based DCIs in the EGI FedCloud Peter Kacsuk, Sandor Acs, Peter Kotzauer, Jozsef Kovacs

2 Private network gLite UI 3G Bridge BOINC Server WMSMCE WS- PGRADE Portal Use Case 1 architecture (SCI-BUS) First successful experiment on CESNET and SZTAKI Clouds gLite and BOINC infrastructure dynamically deployed in the same cloud in order to test WS-PGRADE portal with gLite and BOINC BOINC client

3 Portal CESNET Cloud GWDG Cloud SZTAKI Cloud DCI Bridge / Glite UI Glite VOMS Glite mCE BOINC server +client Glite WMS Virtual Private network 3 gLite and BOINC infrastructure deployed on different cloud providers Extended Use Case 1 architecture with 3 clouds

4 IDGF-SP 17/09/ Use case 2 for gLite VO communities (EDGI, IDGF-SP)  Extend existing gLite VOs with desktop grids having large number of clients  Two options: – The desktop grid is maintained by IDGF – The desktop grid is set up by the gLite VO community  We show the second option as use case for EGI CloudFed

5 IDGF-SP 17/09/ SZTAKI Desktop Grid server gLite VO modified gLite CREAM CE 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC The scenario to implement: using dynamically instantiated desktop grid resources through gLite EGI Cloud

6 IDGF-SP 17/09/ Result  If you want your own dynamically instantiated computational “cluster” – Setup a gLite CE with EDGI executor (mCE) – Instantiate the DG server image in the FedCloud – Adjust the mCE’s configuration – …and submit your jobs! – Due to the GBAC virtualization any application can run in the DG

7 IDGF-SP 17/09/ If the cloud resources are not enough you can ask volunteers to join your project BOINC client Virtualisation by GBAC volunteers SZTAKI Desktop Grid server gLite VO modified gLite CREAM CE 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC EGI Cloud

8 IDGF-SP 17/09/ Use Case 3 for biologists using autodock (SCI-BUS, IDGF-SP)  Biologist can create the necessary infrastructure in the cloud by one click in order to run an autodock experiment  The created infrastructure contains: – An autodock gateway (customized from WS- PGRADE) – A desktop grid with 2 clients  Biologist can specify the molecule files and initiate the autodock calculation via the autodock gateway  Volunteers can connect if more clients are needed  Grid certificate is not required

9 IDGF-SP 17/09/ SZTAKI Desktop Grid server WS-PGRADE Autodock gateway 3G BridgeBOINC project DC-API plugin Metajob plugin WS SOAP IF BOINC client Virtualisation by GBAC Autodock gateway with BOINC desktop Grid in the cloud EGI Cloud Since biologists have no idea how to create such a cloud system the whole infrastructure is created by 1 click

10 IDGF-SP 17/09/ Generic architecture of the 1-click solution Cloud BOINC Server Client BOINC Clients DG Infrastructure CO framework Queue handler DB Infrastructure Queue Service Description Language Orchestrator Queue Messaging Infrastructure handler VM Handlers Cloud Interfaces Cloud Interfaces Tests & Test Descriptions Tests & Requirement s Handler Executors User Username;Password; gUSE Username;Password;3GB_ URL; Username;Password; BOINC_URL;

11 IDGF-SP 17/09/ Use Case 4 for agINFRA project members  1-Click cloud infrastructure  The current implementation deploys a – workflow engine (and WS-PGRADE based GUI) with access to external clouds, grids, and data components as well, – with Remote API to provide external access e.g. from a DRUPAL module, – an extensible Distributed Computing Infrastructure based on Desktop Grid server, and workers, – one BioVEL application example (BBGC)  Grid certificate is not required.

12 IDGF-SP 17/09/ VM2 VM1 WS-PGRADE WF Developer UI gUSE workflow engine DCI Bridge WS-PGRADE End-User UI Remote API A B BES interface BBGC Workflow (BIOVEL) SZTAKI Desktop Grid server 3G Bridge BOINC project BBGC Binaries BBGC APP VM4 BOINC client BBGC job VM3 BOINC client BBGC job External access from other tools/GUIs BBGC job BOINC client External Grid / Cloud resources, agINFRA data components, registries (CIARD RING) Use Case 4 for agINFRA project members Volunteers

13 IDGF-SP 17/09/ Conclusions  The 1-click concept enables the set up of various DCI infrastructures for various types of communities  Very important to provide a centralized marketplace from where the required DCI service images can be accessed  Gradually, new and new DCI infrastructure images can be provided and used by the 1-click solution according to the needs of various user communities  Further topics interesting for the before mentioned communities: – Brokering, auto scaling, market place development  Based on the previous experiences SZTAKI is ready to actively work in these areas and to provide support for different user communities