Stakeholders consultation Ignacio Blanquer-Espert EUBrazilCC project coordinator www.eubrazilcloudconnect.eu

EU Brazil Cloud Connect Aims and benefits The main objective is the creation of a federated e-infrastructure for research using a user-centric approach. To achieve this, we need to pursue three objectives: Adaptation of existing applications to tackle new scenarios emerging from cooperation between Europe and Brazil relevant to both regions. Integration of frameworks and programming models for scientific gateways and complex workflows. Federation of resources, to build up a general-purpose infrastructure comprising existing and heterogeneous resources Additionally, EUBrazilCC will: perform an active dissemination campaign, analyse innovation, foster the involvement of Brazilian institutions in cloud standards definition, and bring the EU Cloudscape series to broader international audience. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

Architecture UC1 UC2 UC3 Users mc2 Programming Frameworks e-SC API COMPSs Execution Environments & Services CSGRID e-SC PDAS COMPSs PMES Resources Management OCCI CDMI IM VMRC AAI SLURM FogBow Infrastructure Providers Opportunistic Cloud Private Cloud HPC COLFLEB CLIOC Data Providers Landsat CMIP5

Whole Vascular System Simulation EUBrazilCC targets three use cases which require the collaboration of Europe and Brazil Leishmaniasis Virtual Laboratory - LVL Anticipating outbreaks Whole Vascular System Simulation A Pureblood simulator Simulating a heartbeat is a complex, multi-scale problem. EUBrazilCC will deploy a complete blood simulation system with an accuracy beyond the state of the art by integrating the ALYA heart simulation system with the complete vascular simulation system of ADAN . Biodiversity & Climate Change Breaking the vicious circle Understanding the mutual interaction at a global scale between climate change & biodiversity dynamics is needed. EUBrazilCC will integrate two workflows com- ______ Every year 1-2 million new cases of Leishmaniasis occur. The LVL aims to improve the knowledge on the susceptibility of outburst in Leishmaniasis disease by integrating species distributed databases of parasite vectors and biomolecular data from bining models of plant spe- cies distribu- tion & multi- level imaging data & proce- ssing in a scientific ga- teway. data from the parasites with bioinf. and Ecological Niche Modelling processing pipelines. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

What Most researchers have A few cores in a set of medium-sized computers. Configured by the researcher or by a local system administrator. Own resources. Persistent POSIX filesystem where own and reference data is located. Working for my applications. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

What researchers have to do Cloud infrastructures Get results Scale down and undeploy Run and scale up Transfer data to local disks Adapt applications Prepare, deploy and configure infrastructure Identify provider Local own resources 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

What Most researchers have What cloud is offering them A few cores in a set of medium-sized computers. Configured by the researcher or by a local system administrator. Own resources. Persistent POSIX filesystem where own and reference data is located. Working for my applications. A large number of smaller- scale instances. VMIs need to be configured, properly installed and managed. Pay-per-use. Non-straightforward file storage. Applications do not work faster just by deploying on the cloud. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

What do they expect? Case 1: They need MORE computing Example case: Climate Change Processing of new remote sensing data takes less than 1K hours (although increasing). Historical data however takes tens of CPU years. On the other side, processing pipelines (e.g. barcoding, orthology, molecular phylogeny) require a variable type and number of computing nodes Replicate their infrastructure on ext. provider Work as usual Do not care about scaling / deploying Really pay as you go Cloud infrastructures 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

What do they expect? Case 2: Their systems are small They need a “bigger” machine Users do not want to pay for the memory when not using it. Example case: Biocomputing Assembling of Next Generation Sequencing requires >64GB RAM computers. Mapping may require tens of GB RAM depending on the complexity of the reads and the reference, not necessarily bounded to the size of data or arguments. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

Needs posed by the users Minimum difference with their infrastructure and the infrastructure on the cloud Vision: Infrastructure to “move” from my resources to an external cloud, and eventually come back. Simplify deployment and elasticity Deploying an customizing a cluster in the cloud should not require administrator skills. Horizontal elasticity must take into account the automatic reconfiguration of the virtual cluster. Vertical elasticity can enable running applications eventually demanding large memory. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

Addressing the challenge Automatic Deployment Deployments based on recipes, configuration and contextualization services. www.grycap.upv.es/im Automatic Elasticity Virtual elastic computing clusters. www.grycap.upv.es/clues 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

Conclusions Users do not expect infrastructures to provide a single machine with a commodity software, but a fully-configured cluster. Users do not expect to assume the administration or configuration Pre-configured VMs are hard to maintain. Market-like offer-vs-demand models start to be successfully users and can adjust resources and make users pay for what they really use. 17/6/2014 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC

More Info and Contact www.eubrazilcloudconnect.eu Open Call for papers BR Coordinator Francisco Vilar Brasileiro fubica@dsc.ufcg.edu.br Universidade Federal de Campina Grande Brazil EU Coordinator Ignacio Blanquer-Espert iblanque@dsic.upv.es Universitat Politècnica de València Spain www.eubrazilcloudconnect.eu 4/5/2019 Stakeholders consultation on computing and data for the WP 2016-17 - EUBrazilCC