1. Introduction to the VPH-Share Atmosphere Cloud Platform Piotr Nowakowski, Tomasz Bartyński, Marian Bubak, Tomasz Gubała, Daniel Harężlak, Marek Kasztelnik, Maciej Malawski ACC CYFRONET AGH UvA Seminar Amsterdam, 30 June 2015

2. VPH-ShareUvA Amsterdam30-06-2015 A (very) brief introduction Scientist I need to compute something. The challengeWhere we are Developer Office PC Scientist MySpine (client application) Requires: MS Windows There’s a better way! MySpine (cloud service) Requires: MS Windows Computational cloud Developer Scientist Computations are an inherent part of modern e-science, particularly within the life sciences domain. As the available IT tools grow ever more sophisticated, domain scientists require help from scientific programmers and other IT specialists to be able to perform their research. When faced with a computational task, the first reaction is to either install the necessary software by oneself or call in help from the IT department. Either way, such traditional setups carry serious drawbacks: You need to provision your own hardware (typically an office PC) A locally running application is only accessible from one place (typically your office) Applications and data cannot be easily transferred to other computers Computational clouds enable us to avoid these problems entirely. A cloud-based service can perform all the functions of a locally running application, with the following benefits: The hardware is provided by the cloud operator (and can be vastly more powerful than any local resources you have access to!) A cloud application is available from anywhere Once deployed, the application can be accessed by many users at the click of a button

3. VPH-ShareUvA Amsterdam30-06-2015 Install/configure each application service (which we call a Cloud Service or an Atomic Service) once – then use them multiple times in different workflows; Direct access to raw virtual machines is provided for developers, with multitudes of operating systems to choose from (IaaS solution); Install whatever you want (root access to cloud Virtual Machines); The cloud platform takes over management and instantiation of Cloud Services; Many instances of Cloud Services can be spawned simultaneously; Large-scale computations can be delegated from the PC to the cloud/HPC via a dedicated interface. Basic functionality of the cloud platform Developer Application Install any scientific application in the cloud End user Access available applications and data in a secure manner Administrator Cloud infrastructure for e-science Manage cloud computing and storage resources Managed application

4. Cloud service instance: A running instance of a cloud service, hosted in the cloud and capable of being directly interfaced, e.g. by the workflow management tools or VPH-Share GUIs. ! Virtual Machine: A self-contained operating system image, registered in the Cloud framework and capable of being managed by VPH-Share mechanisms. ! Cloud service: A VPH-Share application (or a component thereof) installed on a Virtual Machine and registered with the cloud management tools for deployment. ! Raw OS OS VPH-Share app. (or component) External APIs OS VPH-Share app. (or component) External APIs Cloud host A (very) short glossary

5. VPH-ShareUvA Amsterdam30-06-2015 5 VPH-Share Computational Cloud: developers’ view MySpine v22 (cloud service)ViroLab DRS (cloud service)GIMIAS (cloud service)Ubuntu Linux (OS template)MS Windows (OS template) VPH-Share Computational Cloud Service instance Ubuntu Linux (OS template) Hardware: 4 cores, 8 GB RAM, 30 GB HDD 1.The developer, selects and instantiates a service. This can be an existing application (e.g. MySpine) or a „raw” OS image (e.g. MS Windows Server) Service instance OpenLab v1 (development) Hardware: 4 cores, 8 GB RAM, 30 GB HDD 2. The VPH-Share cloud platform creates a virtual machine which hosts an instance of the selected template and provides the required quantity of hardware resources. 2’. The developer may securely log into the machine and install any software of his/her choosing – in this case, we’re installing the OpenLabyrinth virtual patient management application. OpenLab v1 (cloud service) 3. Once the developer is satisfied with the application, he/she may instruct the VPH- Share cloud platform to package and upload it as a new service to the cloud. OpenLab v1 (cloud service) 4. The new service is automatically registered in VPH-Share and can be shared with platform users (including other developers, who may wish to continue development work on the application). Developer

6. VPH-ShareUvA Amsterdam30-06-2015 6 VPH-Share Computational Cloud: end users’ view MySpine v22 (cloud service)ViroLab DRS (cloud service)GIMIAS (cloud service) VPH-Share Computational Cloud 1.The scientist browses the available services and requests access to a given service. Only published services (not OS templates) are visible to non-developers. Service instance OpenLab v1 (cloud service) Hardware: as required by the application 2. The service’s owner (typically its developer) approves the sharing request, following which the scientist may spawn instances of the service in the cloud infrastructure 2’. The cloud platform creates an instance of the service with the appropriate hardware allocation (as set by the developer) and provides the user with access information. OpenLab v1 (cloud service) 3. The user is free to interact with the service. Once the instance is no longer needed, it can be shut down in order to conserve computational resources. Scientist OpenLab v1 (cloud service) Authorized users: Developer

7. Worker Node Head Node Image store VPH-Share cloud site at ACC CYFRONET AGH Worker node w/large resource pool („fat node”) Head Node Image store VPH-Share cloud site at UNIVIE Hybrid cloud environment API host Image store Amazon Elastic Compute Cloud (EC2) – European availability zone 96 CPU cores 184 GB RAM 4 TB storage private IP space Worker node w/large resource pool („fat node”) 128 CPU cores 256 GB RAM 4 TB storage private IP space Worker Node Massive (functionally limitless) hardware resource pool public IP space Cloud Management Portlets VPH-Share Master Interface host portal.vph-share.eu Provide GUI elements which enable service developers and end users to interact with the Atmosphere platform and create/deploy services on the available cloud resources WP2 Core Services Host vph.cyfronet.pl user accounts Atmosphere Registry (AIR) available cloud sitesservices and templates Atmosphere Core Secure RESTful API (Cloud Facade) Authentication and authorization logic Communication with underlying computational clouds Launching and monitoring service instances Creating new service templates Billing and accounting Logging and administrative services

8. VPH-ShareUvA Amsterdam30-06-2015 User authentication The OpenID architecture enables the Master Interace to delegate authentication to any public identity provider (e.g. BiomedTown). Following authentication the MI obtains a secure user token containing the current user’s roles. This token is then used to authorize access to Cloud Service Instances, in accordance with their security policies. VPH-Share Master Int. Authentication widget Login feature Admin Developer Scientist Portlet BiomedTown Identity Provider Authentication service 2. Open login window and delegate credentials VPH-Share Cloud Service Instance Security Proxy 1. User selects „Log in with BiomedTown” Users and roles Security Policy Service payload (VPH-Share application component) 3. Validate credentials and spawn session cookie containing user token (created by the Master Interface) 5. Parse user token, retrieve roles and allow/deny access to the CSI according to the security policy 6’. Relay request if authorized 6’. Report error (HTTP/401) if not authorized 4. When invoking CS, pass user token along with request header

9. VPH-ShareUvA Amsterdam30-06-2015 Security key management Atmosphere provides a mechanism for developers to manage and access their Cloud Services in a secure manner. Prior to starting development work on a Cloud Service the developer opens their favorite SSH client software and generates a pair of RSA security keys. The public key is uploaded into Atmosphere using the Key Manager extension in the Cloud Manager interface. The developer keeps the private key in a safe place and does not share it with anyone. Public key authentication is supported by all popular SSH clients and enables the user to obtain shell access to their development-mode Cloud Service Instances without relying on „magic” accounts or pre-shared root credentials. Atmosphere takes care of managing public keys. Any number of keys may be registered by a single developer. Developer SSH key generator Public keyPrivate key 1. Open SSH client software and generate a pair of security keys VPH-Share Master Int. Cloud Manager Development Mode Key Manager 2. Upload your public key to Atmosphere using the Key Manager Core Component Host (149.156.10.143) Cloud Facade (API) Atmosphere Internal Registry 3. Key Manager asks Cloud Facade to store key Keystore 4. Cloud Facade stores key in AIR

10. Instantiating a Cloud Service Template (1/2) The Cloud Manager portlet enables developers to create, deploy, save and instantiate Cloud Service Instances on cloud resources. Developer VPH-Share Master Int. Cloud Manager Development Mode Start Cloud Service Core Component Host (149.156.10.143) Cloud Facade (API) Atmosphere AMS Atmosphere Internal Registry AIR DB Comp. model Keystore Nova Head Node (149.156.10.132) OpenStack (API) Nova management interface Glance image store CS Images 1. Start CS 2. Request instantiation of Cloud Service 3. Get CS VM details OpenStack WN (10.100.x.x) WN hypervisor (KVM) Mounted network storage Per-WN storage 6. Upload VM image to WN storage 5. Stage CS image on WN Cloud Service Instance 7. Boot VM 7. 4. Call Nova to instantiate selected VM 8. Inject security key (development mode) 8. Retrieve security key Virtual HDD

11. Developer VPH-Share Master Int. Cloud Manager Development Mode Core Component Host (149.156.10.143) Cloud Facade (API) Atmosphere AMS Atmosphere Internal Registry AIR DB OpenStack (API) Nova management interface 17. Retrieve CSI status, port mappings and access credentials 13. Register CSI as booting/running 11. Poll Nova for VM status OpenStack WN (10.100.x.x) Cloud Service Instance Virtual HDD WN hypervisor 9. Report VM is booting 12. Delegate query and relay reply IP Wrangler host (149.156.10.151) DNAT Port mapping table 14. Configure DNAT to enable port forwarding 16. Poll for CSI status and update view CSI details Atmosphere takes care of interpreting user requests and managing the underlying cloud platform. The platform honors resource allocation requests. 15. Register port mappings for this CSI 10. Report VM is running Nova Head Node (149.156.10.132) Comp. model Keystore Instantiating a Cloud Service Template (2/2)

12. VPH-ShareUvA Amsterdam30-06-2015 Obtaining access to a Cloud Service Instance in development mode Note: Cloud Service Instances typically do not have public IPs The role of the IP Wrangler is to facilitate user interaction on arbitrary ports (e.g. SSH, VNC etc.) with VMs deployed on a computing cluster (such as is the case at CYFRONET) Accessing Cloud Service Instances in development mode requires the user to present his/her private key The preinjected public key enables the SSH server residing on the CSI to perform user authentication Developer VPH-Share Master Int. Cloud Manager Development Mode OpenStack WN (10.100.x.x) KVM hypervisor Cloud Service Instance (Virtual Machine) Virtual HDD IP Wrangler host (149.156.10.151) IP Wrangler Port mapping table CSI metadata Standard IP stack (accessible via public IP) 1. Look up CSI details (including IP Wrangler IP, port mappings and access credentials, if needed) 2. Initiate interaction. Use private key to authenticate self 3. Relay4. Call ASI Local shell SSH host Public key 5. Perform authentication

13. Saving the Instance as a new Cloud Service Developers are able to save existing instances as new Cloud Services. Once saved, a Cloud Service can be instantiated by clients. Developer VPH-Share Master Int. Cloud Manager Development Mode Save Cloud Service Core Component Host (149.156.10.143) Cloud Facade (API) Atmosphere AMS Atmosphere Internal Registry AIR DB Nova Head Node (149.156.10.131) OpenStack (API) Nova management interface Glance image store CS Images 1. Create CS from CSI specifying service name, requirements and flags 2. Request storage of Cloud Service 8. Register CS as available. 3. Call Nova to persist CSI OpenStack WN (10.100.x.x) WN hypervisor (KVM) Mounted network storage Per-WN storage 6. Upload VM image to Glance 4. Store VM image in Glance Cloud Service Instance Assigned local storage 5. Image selected VM (incl. user space) 5. 7. Report success CS metadata 3’. Register CS as being saved. Comp. model Keystore

14. Cloud platform interfaces All operations on cloud hardware are abstracted by the Atmosphere platform which exposes a unified RESTful API (with a suitable set of developer’s documentation available). For end users, the API is concealed by a layer of platform GUIs embedded in the VPH-Share portal and providing a user-friendly work environment - for domain scientists and service developers alike. The API can also be directly invoked by external services as long as they possess the required security credentials (Atmosphere relies on the well-known OpenID authentication standard with the VPH site at www.biomedtown.org acting as its identity provider).www.biomedtown.org Application -- or -- Workflow environment End user A full range of user-friendly GUIs is provided to enable service creation, instantiation and access. A comprehensive online user guide is also available. Atmosphere Registry (AIR) Atmosphere Ruby on Rails controller layer (core Atmosphere logic) Cloud sites The GUIs work by invoking a secure RESTful API which is exposed by the Atmosphere host. We refer to this API as the Cloud Facade. Any operation which can be performed using the GUI may also be invoked programmatically by tools acting on behalf of the platform user – this includes standalone applications and workflow management environments (which VPH-Share also provides).

15. VPH-ShareUvA Amsterdam30-06-2015 Advanced features: smart utilization of hardware resources Published services become visible to non-developers and can be instantiated using the Generic Invoker. Developers are free to spawn „snapshot” images of their cloud services (e.g. for backup purposes) without exposing them to external users. Scientist Developer Atmosphere Cloud Platform Cloud Service Published Atmosphere Cloud Service Shared Cloud WN Shared VM Scientist Atmosphere Cloud Service Scalable Cloud WN Separate VM Scientist Cloud WN Separate VM A Shared service is backended by a single virtual machine which „mimics” multiple instances from the users’ point of view. Shared services greatly conserve hardware resources and can be instantiated quickly. When a Scalable service is overloaded with requests Atmosphere can spawn additional instances in the cloud to handle the additional load. The process is transparent from the user’s perspective.

16. Advanced features: template propagation in a hybrid cloud environment Developer OpenLab v1 (cloud service) VPH-Share cloud site at ACC CYFRONET AGH Template repository (GLANCE) Service templates (qcow2 format) 1. Developer creates a new service on one of the available cloud sites VPH-Share cloud site at UNIVIE Template repository (GLANCE) Service templates (qcow2 format) Atmosphere Amazon Elastic Compute Cloud (EC2) – EU availability zone Template repository Service templates (AMI format) Template converter 2. Upon request, the service template may be automatically propagated to other federated sites so that it can be instantiated and accessed there 3.The template image can be directly copied to OpenStack sites which use the qcow2 image format 4.In order to deploy the template in a public cloud, the template must first undergo (automated) conversion The Atmosphere platform handles propagation and registration of templates in an automatic fashion. The VPH-Share computational cloud is a hybrid environment composed of multiple cloud sites – „private” sites maintained by Project partners as well as the public (commercial) resources purchased from Amazon EC2. In order to enable instantiation of services on arbitrary cloud sites, the Atmosphere platform implements template propagation. New services can be uploaded (upon developer’s request) to selected cloud sites. This feature creates a truly federated cloud with scale-out capability for applications which require more computational power than a single site can provide. Additionally, services which manage sensitive data may be restricted to certain (private) sites so that such data is processed in a controlled environment, in accordance with legal restrictions.

17. VPH-ShareUvA Amsterdam30-06-2015 17 Advanced features: Domain-based service proxy Private cloud site (OST/CYF)Atmosphere core Redirus controller Head Node Worker Node CS Instance: Google Refine Local IP: 10.100.8.34 Exposes external interface on port 3333 Exposes HTTP endpoint with path app/ through the interface listed above Redirus worker (proxy webhost) 1. Atmosphere instructs the proxy host to set up a redirection 2. A downstream link is opened to the Worker Node hosting the service instance HTTP proxy host 149.156.10.135 Private intranet (10.100.8.0/22)Public Internet (149.156.10.132/28) Redirus worker 3. The domain-based proxy creates a dedicated domain on the fly, and can do so using secure (HTTPS) channels: User The HTTP proxy is a component of the cloud platform which enables external access to HTTP-based service instances deployed in a private cloud site. This includes (in particular) web applications which expose graphical interfaces to end users, but also SOAP/RESTful services which perform computations as part of larger application workflows. The proxy resolves the IP address crunch in a graceful manner, ensuring that each HTTP endpoint is matched to a unique domain (created on the fly by a dedicated Atmosphere component). This change enables seamless deployment of web applications and services which reference themselves through HTTP/HTTPS hyperlinks. https://web-7936.vph.cyfronet.pl/app/

18. VPH-ShareUvA Amsterdam30-06-2015 18 Advanced features: Interface monitoring Time elapsed Atmosphere core Worker Node Spawn ASI 1. User starts new service instance Worker Node OS booting „Your service is initializing” 2. Cloud reports service is initializing User is asked to wait. API not responding Worker Node OS active App initializing API not ready „Your service is ready!” 3. Cloud reports service is running – but the API is not yet operational. Worker Node OS active App ready API active 4. Atmosphere monitors CSI APIs and updates their status. API responding A crucial problem has been resolved in the scope of CSI initialization sequences: Cloud middleware will report services as „active” as soon as the underlying OS has booted – even though the resident application may require additional time to become responsive. This has the potential to mislead users into trying to interact with services which are still being activated (resulting in errors). Atmosphere monitors the availability of HTTP endpoints and provides users with information regarding their status.

19. VPH-ShareUvA Amsterdam30-06-2015 19 Advanced features: cloud billing logic Fund1 User 2 User 1 User 3 Fund 2 Instance #1 Running at: CYF Launched by: User 1 Paid for by: Fund 1 Requirements: med Hourly cost: $0.0114 Atmosphere Internal Registry Instance #2 Running at: CYF Launched by: User 2 Paid for by: Fund 1 Requirements: low Hourly cost: free Instance #3 Running at: AWS Launched by: User 3 Paid for by: Fund 2 Requirements: hi Hourly cost: $0.2660 Instance #4 Running at: AWS Launched by: User 3 Paid for by: Fund 2 Requirements: v. hi Hourly cost: $0.5320 Atmosphere core Billing log Billing service Determines who can run what and on which sites; Periodically bills all running instances; Can shut down/suspend instances when out of funds. Registers all billing actions for post-mortem analysis and debugging. Each user may be assigned to one or more Funds. Each fund contains a set quantity of monetary resources and can be drawn upon when launching instances. The cloud billing service is a dedicated module built into Atmosphere which monitors financial expenditures associated with the use of cloud resources. Each service instance must be assigned to a Fund which is periodically charged for continued operation of that service. Hourly charges depend on the quantity of hardware resources (CPU, memory etc.) required by the service. Atmosphere interfaces with the underlying billing APIs exposed by popular cloud providers; however system administrators may assign arbitrary hourly rates to each service flavor.

20. VPH-Share#Y3Review23-May-14 20 User 1 Atmosphere core 1. User 1 wishes to launch an instance of the MySpine CS. MySpine r17 (Cloud Service) Can run at: Amazon AWS Requires: 8 cores, 16 GB RAM, 25 GB HDD Operating system family: MS Windows Atmosphere Internal Registry 2. Atmosphere polls AIR for information related to this CS. Flavor name: m3.xlarge Cloud site: Amazon AWS Provides: 8 cores, 16 GB RAM, 80 GB HDD Hourly cost of operation: $0.5320 3. AS requirements are matched to an optimal instance flavor. User: Jane Doe Organization: MySpine Consortium Roles: User, developer Can access funds: Fund 1 4. Atmosphere checks which funds are available to the user Fund: Fund 1 Balance: $31,117.25 Overdraft limit: $0 On exhaustion: Suspend all services 5. Finally, a decision is made on whether to allow AS instantiation Billing service The billing mechanism comes into play whenever a user (or an application acting on behalf of a user) wishes to instantiate a Cloud Service. Before the request can proceed, the billing service determines the cost of launching the CSI. A new instance will be launched if the user has access to sufficient financial resources to pay for 1 hour of the instance’s operation (as most public cloud providers run a hourly billing cycle). A single user may have access to multiple funds in which case Atmosphere will honor the client’s fund selection via an appropriate API call – otherwise the user’s default fund is used. The Optimizer (another Atmosphere component – not pictured here) will select the cheapest hardware flavor which matches user requirements, all other considerations being equal. Advanced features: cloud billing logic (continued)

21. VPH-Share#Y3Review23-May-14 21 Atmosphere core Billing service 2. Funds are debited and administrative actions may be triggered depending on the applicable policies. Fund: Fund 1 Ample credit available On exhaustion: No action Fund: Fund 2 Exhausted On exhaustion: Suspend all services Atmosphere Internal Registry 1. The billing service performs periodic billing of all CSIs (this usually happens once per hour but the billing operation may be triggered at any time. ASI: Jane’s service Paid from: Fund 1 Hourly cost: $0.5320 ASI: Joe’s service Paid from: Fund 1 Hourly cost: $0.2660 ASI: Tom’s service Paid from: Fund 2 Hourly cost: $0.2660 Status: Suspended – out of funding Prepaid until: n/a Total charge: $217.0560 Cloud site Tom’s service 3. As Tom’s service has run out of funding, Atmosphere instructs the cloud site to suspend it. Status: OK Prepaid until: Now + 1 hour Total charge: $39.6340 Status: OK Prepaid until: Now + 1 hour Total charge: $6.3840 The billing service also performs periodic billing actions on all active Cloud Service Instances. By default, each service is charged for 1 additional hour of operation (although this behavior can be changed by system administrators). If a CSI runs out of funding, Atmosphere may take administrative action, depending on its fund’s exhaustion policy – the CSI may be terminated, suspended or left running (generating an overdraft). All billing actions are logged in a dedicated billing log (not pictured). Advanced features: cloud billing logic (continued)

22. VPH-ShareUvA Amsterdam30-06-2015 LOBCDER host (149.156.10.143) LOBCDER service backend Resource catalogue WebDAV servlet Resource factory Storage driver Storage driver Storage driver (SWIFT) SWIFT storage backend Core component host (vph.cyfronet.pl) Data Manager Portlet (VPH-Share Master Interface component) Cloud Service Instance (10.100.x.x) Service payload (VPH-Share application component) External host Generic WebDAV client GUI-based access Mounted on local FS (e.g. via davfs2) LOBCDER (the VPH-Share federated data storage component) enables data sharing in the context of VPH- Share applications The system is capable of interfacing with various types of storage resources and supports SWIFT cloud storage (support for Amazon S3 is under development) LOBCDER exposes a WebDAV interface and can be accessed by any DAV-compliant client. It can also be mounted as a component of the local client filesystem using any DAV-to-FS driver (such as davfs2). Advanced features: binary data storage federation

23. VPH-ShareUvA Amsterdam30-06-2015 23 Cloud Service Instance Atmosphere core Master Interface User security token CS initial configuration …&token=#{ } 1. MI passes user token when calling Atmosphere API Linux OS Local filesystem 2. Atmosphere injects the token into the initial configuration of the Cloud Service which is being launched. 3. On bootup, a dedicated OS script reads the token from the initial configuration (exposed by cloud mechanisms) and invokes a DAVFS2 mount call on LOBCDER. LOBCDER Authentication component 4. The lobcder-automount script extracts the client token from the initial configuration and uses it to establish a secure connection to LOBCDER. 5. If successful, a LOBCDER directory is added to the local FS, containing the user’s resources. Secure storage federation Atmosphere resolves the problem of exposing LOBCDER as part of the local filesystem on Linux machines owing to the concept of dynamic service configurations – i.e. initial configurations, where selected parameters may be specified dynamically during service initialization. This mechanism is used to pass client tokens to the LOBCDER servers, enabling LOBCDER to be securely mounted in the name of the user who launched the given CSI. WebDAV-compliant API Advanced features: LOBCDER automount

24. VPH-ShareUvA Amsterdam30-06-2015 Provides a policy-driven access system for the security framework. Open-source based access control system based on fine-grained authorization policies. Implements Policy Enforcement, Policy Decision and Policy Management Ensures privacy and confidentiality of sensitive scientific data Capable of expressing eHealth requirements and constraints in security policies (compliance) Tailored to the requirements of public clouds VPH Security Framework ApplicationWorkflow managemen t service DeveloperEnd userAdministrator VPH clients VPH Security Framework VPH Cloud Service Instances Public internet (or any authorized user capable of presenting a valid security token) Atmosphere Security Framework

25. VPH-ShareUvA Amsterdam30-06-2015 Security as applied to Cloud Services The application API is only exposed to localhost clients Calls to Cloud Services are intercepted by the Security Proxy Each call carries a user token (passed in the request header) The user token is digitally signed to prevent forgery. This signature is validated by the Security Proxy The Security Proxy decides whether to allow or disallow the request on the basis of its internal security policy Cleared requests are forwarded to the local service instance VPH-Share Cloud Service Instance Security Proxy Security Policy Service payload (VPH-Share application component) Public CS API (SOAP/REST) 1. Incoming request Actual application API (localhost access only) Exposed externally by local web server (apache2/tomcat) 2. Intercept request a6b72bfb5f2466512a b2700cd27ed5f84f99 1422rdiaz!developer! rdiaz,Rodrigo Diaz,rodrigo.diaz@at osresearch.eu,,SPAIN, 08018 User token digital signature unique username assigned role(s) additional info 3. Decrypt and validate the digital signature with the VPH-Share public key. 4. If the digital signature checks out, consult the security policy to determine whether the user should be granted access on the basis of his/her assigned roles. 6. Intercept service response 7. Relay response 3’, 4’ Report error 3’, 4’. If the digital signature is invalid or if the security policy prevents access given the user’s existing roles, the Security Proxy throws a HTTP/401 (Unauthorized) exception to the client. 5. Relay original request (if cleared) 5. Otherwise, relay the original request to the service payload. Include the user token for potential use by the service itself. 6-7. The service response is relayed to the original client. This mechanism is entirely transparent from the point of view of the person/application invoking the Cloud Service.

26. VPH-ShareUvA Amsterdam30-06-2015 VPH-Share applications Not just a proof-of-concept deployment: a real production infrastructure with real-world applications and services. Over 150 service templates currently registered, with approximately 50 instances launched on a daily basis across three computational cloud sites. A ticketing system is in place and technical support is available on a regular basis both to service developers and end users. Online user manuals and platform API documentation is available.

27. VPH-ShareUvA Amsterdam30-06-2015 A more complex case study – the DataFluo Application DataFluo Listener RabbitMQ DataFluo Server CS RabbitMQ Worker CS RabbitMQ Worker CS Cloud Facade Atmosphere Management Service (Launches server and automatically scales workers) Atmosphere Scientist Launcher script Secure API Problem: Cardiovascular sensitivity study: 164 input parameters (e.g. vessel diameter and length) First analysis: 1,494,000 Monte Carlo runs (expected execution time on a PC: 14,525 hours) Second Analysis: 5,000 runs per model parameter for each patient dataset; requires another 830,000 Monte Carlo runs per patient dataset for a total of four additional patient datasets – this results in 32,280 hours of calculation time on one personal computer. Total: 50,000 hours of calculation time on a single PC. Solution: Scale the application with cloud resources. VPH-Share implementation: Scalable workflow deployed entirely using VPH-Share tools and services. Consists of a RabbitMQ server and a number of clients processing computational tasks in parallel, each registered as a service. The server and client services are launched by a script which communicates directly withe the Cloud Facade API.

28. VPH-ShareUvA Amsterdam30-06-2015 28 Summary: challenges and solutions The Atmosphere framework provides a way to port scientific applications to the cloud A layer of abstraction is created over cloud-based virtual machines, enabling the platform to automatically select the best hardware resources upon which to deploy application services Automatic load balancing allows applications to scale up as needed VPH-Share also provides access to cloud hardware upon which scientific applications can be deployed A whole range of applications – from Linux-based SOAP/REST services all the way to rich graphical clients running under MS Windows have been successfully ported, proving the usefulness and versatility of our solution The platform is fully integrated with the wider VPH ecosystem, including its authentication/authorization, sharing and data management mechanisms A billing model has been implemented in order to ensure long-term sustainability of the Project architecture We provide ongoing technical support and advice to application developers and end users

29. VPH-ShareUvA Amsterdam30-06-2015 For further information… A more detailed introduction to the Atmosphere cloud platform (including user manuals) can be found at https://vph.cyfronet.pl/tutorial https://vph.cyfronet.pl/tutorial You’re also welcome to visit the DIstributed Computing Environments (DICE) team homepage at http://dice.cyfronet.pl and our brand new GitHub site at http://dice-cyfronet.github.iohttp://dice.cyfronet.plhttp://dice-cyfronet.github.io