SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI CloudBroker Platform integration into WS-PGRADE/gUSE Zoltán Farkas MTA SZTAKI LPDS

Outline Motivation WS-PGRADE/gUSE CloudBroker Platform Integration: – Architecture, features – DCI Bridge modifications – WS-PGRADE modifications – WFI modifications SaaS vs IaaS execution modes

Motivation There are many user communities who would like to access compute resources in a transparent way through science gateways Cloud resources are getting more and more popular Clouds are more reliable than grids Additionally, cloud resource can provide a vast amount of resources (flexible resource allocation) A customizable science gateway with cloud support is capable of satisfying compute needs of complex scientific computations Aim: provide access to cloud resources in a transparent way

WS-PGRADE/gUSE in SCI-BUS

The CloudBroker Platform Easy, scalable, secure, integrable and pay-per-use access to scientific and technical applications in the cloud High performance computing application store and marketplace with accounting and billing Deployment and execution of applications in the cloud with the same parameters and files as for local execution Accessible through the internet via a web browser UI Integration possibilities into third party tools via web service-based APIs Public version available under Using infrastructure as a service (IaaS) from cloud providers Offering platform as a service (PaaS) for software vendors Providing software as a service (SaaS) to end users Available compute resources: Amazon, IBM, OpenStack, OpenNebula Available storage resources: Amazon, Walrus, Rados S3

CloudBroker Platform Architecture CloudBroker Platform Amazon Cloud IBM Cloud … Cloud Chemistry Appli- cations Biology Appli- cations Medicine Appli- cations Web Browser UI Engineering Appli- cations Web Service API Generic Workbenches CloudBroker Integration Domain-Specific Gateways R&D End Users and Software Vendors CLI … Appli- cations

Integrated architecture

Integration features I. Slight modification in the fully featured UI Also available in every customization level, completely hiding the fact that cloud is being used: – ASM – Remote API – End-user view – DCI Bridge

Integration features II. Support for commercial clouds with costs (prices configured in CloudBroker Platform): – Estimated job cost before submission – Actual job and workflow cost after execution

Integration features III. IaaS modelSaaS model Enables users to run their own executablesYesNo What has to be pre-deployed in the virtual machine image A single wrapper applicationEvery application that users would like to use What has to be configured in the CloudBroker Platform Only the wrapper applicationExecutables for the different application Level of securityLow: user can run anythingHigh: users can run only pre-registered, tested apps Ease of use (user’s perspective)Very easy: very similar to existing WS- PGRADE/gUSE interface Very easy: user simply selects from the pre- defined Software Ease of use (portal administrator’s perspective) Easy: only two additional properties have to be set in the DCI Bridge configuration (see previous slide) Very easy: no need to set additional properties Ease of use (CloudBroker administrator’s perspective) Easy: only one application has to be configured for each cloud resource Hard: a number of applications have to be configured Easy of use (cloud administrator’s perspective) Easy: only one VM image has to be deployedHard: either a number of VM images have to be deployed or one VM image must be updated a number of times.

DCI Bridge modifications I. New Configuration tab for CloudBroker

DCI Bridge modifications II. New CloudBroker plugin to manage CloudBroker jobs Using slightly extended JSDL Workflow Test Applications 1.0 1f803ac0-8cfd-47de-9b9a-718f2a287f20 Workflow Test Applications 1.0 Workflow Test Applications 1.0 ARInew.sh Workflow Test Applications 1.0 ARInew.sh A job.out job.err OpenNebula MTA SZTAKI SZTAKI OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula OpenNebula MTA SZTAKI SZTAKI SZTAKI OpenNebula x86_64

WS-PGRADE modifications I. CloudBroker Authentication Portlet

WS-PGRADE modifications II. IaaS and SaaS-based configuration

WFI modifications Produce JSDL document for DCI Bridge based on workflow XML containing CloudBroker- specific settings

Accessible Cloud Resources Access provided by the CloudBroker Platform Commercial: – Amazon EC2 – IBM OpenSource/Free: – OpenStack – OpenNebula Currently, within SCI-BUS accessible: – MTA SZTAKI OpenNebula (400 cores) – BIFI OpenStack (50 cores)

Example applications - IaaS University-level cloud resource access Run any application

Example applications - SaaS Predeployed application execution Docking portal: AutoGrid, AutoDock, AutoDock Vina (should be deployed)

Summary WS-PGRADE/gUSE offers production-level cloud access through the CloudBroker Platform Mainstream cloud resources can be used (both IaaS and SaaS model is supported) Slight UI changes (can be completely hidden for end-users) Commercial exploitation available

Questions? ?