1. An easier path? Customizing a “Global Solution”
Fernando Aguilar
Giacinto Donvito

2. Towards a “Global Solution”
We have a number of solutions/services in INDIGO platform.
There are a number of use cases that use a large set of INDIGO services. Different communities, different uses.
Alternatives to solve the same problem in different ways.
The possibility to include INDIGO services in already available solution/platform
Groups of solutions to match requirements (Yesterday's session).
We propose matching at Level 0, Level 1 or Level 2. Depending on how they use the solutions.

3. INDIGO-DataCloud Solutions for Earth Sciences

4. The user “logins” directly in the infrastructure
Login to the web interface of OpenStack / OpenNebula and submit a job
The system administrators may install the INDIGO products, that allow:
Improved scheduling for allocation of resources: scheduling algorithms and support for spot-instances.
Support for improved IaaS resource orchestration capabilities using standards orchestration engines (Heat) OpenStack and OpenNebula.
Improved and transparent support for Docker containers. This includes for example the introduction of native container support in OpenNebula, and enhanced featurs to nova-docker.
Login in a HPC system or cluster:
Run containers in user space using an INDIGO product: udocker
The sysadmins could install, bdocker, an INDIGO product that provides support for docker containers to be run on the batch system.

5. Using the PaaS: INDIGO architecture
EGI Federated Cloud
Commercial Providers
HPC systems
Orchestrator
Services
Future
Gateway
CLI (orchent)
Data Management
services
AAI INDIGO SOLUTION

6. Using the PaaS via command line interface “orchent”
Orchent is a tool in “go”, that allows submitting job to the PaaS, without going through a graphical interface
Is presented as a user CLI tool in the form of a script.
Could be used by applications to submit durectly to the PaaS Orchestrator
*the user can download the orchent code from github
and then compile it to create the executable (needs go environment)
*A dockerized version is available here:

7. Using the PaaS via FutureGateway
onezone.cloud.ba.infn.it
provider-RECAS-BARI
Local OneClient 7

8. INDIGO CASE STUDY: TRUFA
TRUFA ( A web-based RNA-seq application
RNA-seq is used to analyze the continually changing cellular transcriptome
→ RNA comparison between healthy and unhealthy organisms can lead to better understanding
diseases such as cancer, diabetes, alzheimer...
→ Also useful for comprehending several biological mechanisms (i.e. venom production, silk production...)
Application of next-generation sequencing (NGS) methods for transcriptome analysis (RNA-seq) has become increasingly accessible in recent years and are of great interest to many biological disciplines including: evolutionary biology, ecology, biomedicine, and computational biology.
Although virtually any research group can now obtain RNA-seq data, only a few have the bioinformatics knowledge and computation facilities required for transcriptome analysis.
INDIGO CASE STUDY: TRUFA

9. PaaS Orchestrator or uDocker INDIGO CASE STUDY: TRUFA
TRUFA ( A web-based RNA-seq application
The main advantage of TRUFA is that it allows to gather together the most advanced tools in RNA-seq in a friendly and interactive way
→ Completely transparent. No informatics knowledge needed by the user.
→ Highly parallelized
The typical file size used in this kind of analysis goes from 5 GB to 40 GB per file.
→ Large data both for processing and for storing
Most of the analysis take more than 3 days to be completed.
→ A lot of CPU time
CPU time per year:
2015:
2016:
2017:
INDIGO Solutions:
PaaS Orchestrator or uDocker
INDIGO CASE STUDY: TRUFA

11. TRUFA with Orchestrator
Deployment & Orchestrator Service
Future Gateway
Data mngt. Service (OneData)
Disk Volume (FS)
Input/output
WNs
TOSCA Recipes
Monitoring
Resource provider (e.g. EGI FedCloud)
TRUFA Web
AAI
TOSCA Expert
Repositories
TRUFA Manager
TRUFA User

12. TRUFA with Orchestrator
INDIGO IAM
AAI for all the services: Onedata, FGW, TRUFA web (OpenID).

13. TRUFA with Orchestrator
Future Gateway API
Definition of TOSCA templates
Connecting with PaaS
Option B: Orchent

14. TRUFA with Orchestrator
PaaS Orchestrator
Mesos and Chronos
Two alternatives: Virtual cluster or Dynamic deployment
Autoscaling Vs. Workflow management.

15. TRUFA with Orchestrator
Data Management Solution (OneData)
Accessible by: User, TRUFA web, Computing nodes.

16. TRUFA with Orchestrator
Fairshare scheduler on Cloud Management Framework
TOSCA translator at the IaaS level
Storage provider for Onedata
Infrastructure Manager

17. Data mngt. Service (OneData)
TRUFA with uDocker
Scheduler
Monitoring
Data mngt. Service (OneData)
Disk Volume (FS)
Input/output
WNs
uDocker
TRUFA Web
AAI
Repositories
TRUFA Manager
TRUFA User

18. Thank you!
Fernando Aguilar