A scalable and flexible platform to run various types of resource intensive applications on clouds ISWG June 2015 Budapest, Hungary Tamas Kiss, Hannu Visti, Gabor Terstyanszky, Gregoire Gesmier

Funded by the European Commission FP7 programme, FoF: Factories of the Future July 2013 – December 2015 EUR 4.5 million overall funding Coordinated by the University of Westminster 29 project partners from 8 European countries 24 companies (SMEs) and 5 academic/research institutions An example for intensive industry collaboration CloudSME - EU funded industry oriented research project

* The CloudSME project develops a cloud-based, one-stop-shop solution providing a scalable platform for small or larger scale simulations, and enable the wider take-up of simulation technologies in manufacturing and engineering SME’s. The CloudSME project Cloud-based Simulation platform for Manufacturing and Engineering * Defines generic and concrete business models for SMEs in the manufacturing/engineering sector to facilitate the take-up of cloud-based simulation solutions * CloudSME builds a simulation platform that allows seamless access to multiple heterogeneous cloud resources and provides a high level of abstraction to users when accessing these resources for simulations in a one-stop-shop solution. * Provides a Platform as a Service (PaaS) solution to build customised cloud applications * Enables simulation software providers to offer Software as a Service (SaaS) simulation solutions * Enables SMEs in the manufacturing and engineering domain to access simulation services * Provides seamless access to HPC resources in order to speed up the simulations on-demand

The CloudSME Simulation Platform

Targeted application areas * High Performance Computing (HPC) * relatively small number of end-users with high computation demand * many used simulation before * enable software vendors to extend their product with cloud support * High Throughput Computing – parameter sweep (HTC) * use of multiplier entities (e.g. consultant companies) and technologies (e.g. templates) * although smaller in scale, still computationally intensive, typically parameter sweep * Scalable Web Applications (SWA) * Use CloudSME Simulation Platform (CSSP) to deploy and manage web services on public Internet.

High Performance Computing * Example Case: Fluid dynamics simulation * Benefits * Provides access to various clouds, grids and clusters – allowing execution of code where it is optimal either from performance or cost perspective * Challenges * Scales well within a single node by adding more cores. Communication between instances is a problem if more than one node is needed. CloudSigma are working on a solution in CloudSME project, and there is also supercomputer access from the platform.

High Throughput Computing * Example Case: Repast * Repast: Recursive Porous Agent Simulation Toolkit * Requires a model (Java code) and a parameter file * Benefits * CloudSME infrastructure provides parameter sweep workflow functionality and possibility to orchestrate jobs to different clouds. * Challenges: * Each model is different. Some are computationally heavy and a single run takes a long time to complete, while others complete in fractions of a second. * Currently each run runs exactly one simulation with a certain model and parameter file. If thousands or tens of thousands of runs are required, this needs to be optimised.

Scalable Web Applications * Example Case: Outlandish and Tidybooks * Three-tier architecture (proxy server with public IP address, application server and database server) * Can run LAMP or MEAN stack – not limited to any particular architecture * Can be managed from Cloudbroker GUI or Cloudbroker API * Benefits * CloudSME infrastructure allows access to different clouds. A Cloudbroker job is a representation of application functionality and data, which fits to Web application architecture neatly. * Setting up the architecture is basically a workflow

HTTP proxy1 HTTP proxy 2 HTTP proxy N App server 1 App server 2 App server 3 App server N DB server 1 DB server 2 Replication Internet Public IP Private IP Admin server Admin server Scalable Web Applications

* Challenges: * Needs public IP address. Academic clouds generally do not provide this. * Cost and performance optimisation + need to add mechanisms of measuring. * If the web service traffic volume varies a lot, it would need to be scaled up or down on the fly. Managing this, especially between clouds, would require a portable representation of the application, its configuration and also data. Means of transport exist already in Cloudbroker input-output parameters, but there are problems (potentially massive volumes of data – financial optimisation would need a way of querying this) * Currently not tested with WS-PGRADE portal – the setup workflow is implemented as a Python script. There is not currently an easy way to run Cloudbroker API commands from workflows. This could be solved by a front-end application deployment but doing this would not be trivial.

Any questions?