User Interfaces: Science Gateways, Workflows and Toolkits Marcin Plociennik RIA-653549

Scenarios

User interfaces, toolkits and services Ophidia OMT FutureGateway Portlets Kepler Orchent, TTSc

Checklist FG OMT Ophidia Kepler P0 CSIC LifeWatch Y P1 UPV EuroBioImaging P2 CIRMMP INSTRUCT P3a INAF, LBT P3b INAF,CTA P4 U. Utrecht P5 CMCC ENES P6 ICCU N Y? P7 EGI.eu P8 CNR ELIXIR‐ITA P9 INGV EMSO MOIST P10 RBI DARIAH P11 INFN CMS P12 LATTICE QCD

The FutureGateway (FG) A set of components to build Science Gateways The core is a RESTful API service Manage applications and resources for user communities Provide a highly customisable environment targeting different portal Frameworks and their technologies Components and customisation provided for Liferay Modular architecture to support different back-end infrastructures (Cloud, Grid, local HPC sites)

Integration: Galaxy for ELIXIR use case A FG portlet for Liferay has been developed to deploy Galaxy based application on INDIGO PaaS Application is managed by the FG API service Interaction with PaaS through the INDIGO Orchestrator    

Generic portlet - What it is? Generic portlet is a part of FutureGateway Portal, based on Liferay, responsible for submitting tasks. Portlet allows to use all applications available at FutureGateway server 7

How to use it? Register your application using FutureGateway admin portlet Register application in generic-portlet-config.json file at portletConfigFiles directory (will be available from the FG admin) Add parameters.json file as a template for submitting new tasks: list of the parameters-default values, descriptions

Example: Generic portelt

Example: Submitting new task

Open Mobile Toolkit Set of libraries to be used by mobile applications based on FG API Server the library for the Android/iOS platform

Scientific workflows A scientific workflow system is usually defined as a system designed to compose and execute a series of computational or data manipulation steps, or workflow, in a scientific application Scientific workflows automates tedious jobs traditionally performed by hand for each data set, and usually contain a large number of tasks Scientific workflows are a useful representation for managing the execution of large-scale computations Facilitates the creation and management of computation Provenance is captured at runtime

INDIGO-Kepler Kepler is a scientific workflow system Workflows = scenarios realized by actors i.e. components with I/O linked together Lots of actors are available(500+), even more are developed by domain-specific communities 13

INDIGO-Kepler indigo-dc/indigokepler Kepler actors and examples to communicate with FutureGateway API Server Allows to: List applications defined on FG Manage tasks i.e. jobs or resources allocations Perform remote I/O operations 14

INDIGO-Kepler Why use INDIGO-Kepler? Graphical design of computational experiment, even very complex Access to other INDIGO components (e.g. VM instantiation) A workflow is a snapshot of scenario – can be shared and reused by the community 15

INDIGO-Kepler 16

INDIGO-Kepler Ready to use in a couple of minutes: indigodatacloudapps/kepler in Docker Hub You can install it on your own VM via Ansible: indigo-dc/kepler in Ansible Galaxy Codes available: indigo-dc/indigoclient in GitHub indigo-dc/indigokepler in GitHub 17

INDIGO-Kepler Scenario 1: build your own workflow Define & add your application on the FutureGateway (JSON format) Compose workflow core: CreateTask UploadFiles GetTask [repeat until DONE] DownloadFiles Add additional logic (remote file fetch, local database read, etc.) 18

INDIGO-Kepler Scenario 2: run batch workflow Use specialized indigo-dc/kepler-batch Docker image Define all inputs in the TOSCA template Fill inputs via web, native or mobile GUI Send TOSCA, parameters & Kepler workflow to the Orchestrator 19

Ophidia - Overview Big data stack for the analysis of scientific, multidimensional data Time series analysis, data subsetting, data aggregation, data intercomparison, predicate evaluation, OLAP support, etc. Designed for eScience Multiple interfaces available Support for both batch & interactive data analysis as well as for analytics workflows Exploits parallel paradigms (both MPI & OpenMP) to address performance Link to International Working Groups RDA Array Database Assessment WG ESGF-Compute Working Team Included in the 1st INDIGO release

Ophidia Several communities are exploiting/integrating Ophidia in their case study: EMSO: analysis of very long time series recorded by a seafloor observatory, starting from seismic data in SAC format. LTB: integration of the astronomical FITS image file format into Ophidia for massive data reduction. Import routine has been written and a preliminary workflow implementing the reduction procedure on CUBE files has been tested. Lifewatch: analysis of Hydrodynamic and Water Quality. Activity is ongoing to compare different variables to find correlations and study the differences using Ophidia. ENES: couple model intercomparison data analysis experiment. The proposed solution is paving the way towards the integration of INDIGO services into the ESGF community stack. Links with ESGF and RDA. PTA = Precipitation Trend Analysis

Orchent – The Orchestrator CLI Extensive help at the command line Full use of the REST interface of the orchestrator List all deployments, showing a specific deployment List all resources, showing a specific resource Create and update of deployments Delete deployments Single binary executable Completely new development carried out in INDIGO-DataCloud 22

Token Translation Service Client (TTSc) Command line client for REST interface of TTS Support the complete interface Get list of all services supported by TTS Get list of credentials of a user Request credentials for a specific service Revoke a specific credential Completely new development carried out in INDIGO-DataCloud 23

Admin Portlet FutureGateway manages services and applications for the users Applications have to be registered in advance Infrastructure end-points have to be specified The admin portlet provide a GUI for the application/service and infrastructure management Allows to verify the user activities It is aimed at the scientific domain admin

Requirements The admin portlet is available in the control panel of the Liferay implementing the Science Gateway Requires a specific role to be accessible: assigned to scientific domain administrators TOSCA template has to be provided in order to create an application Can be either a file or a URL

The interface Infrastructure List Application Detail New Application