Population Imaging Use Case - EuroBioImaging

Slides:



Advertisements
Similar presentations
Microsoft Dynamics AX Technical Conference 2013
Advertisements

An Approach to Secure Cloud Computing Architectures By Y. Serge Joseph FAU security Group February 24th, 2011.
NCS Grid Service Ken Meacham, IT Innovation Crystal Grid Workshop, Sept 2004.
FI-WARE – Future Internet Core Platform FI-WARE Cloud Hosting July 2011 High-level description.
Ocean Observatories Initiative Common Execution Environment Kate Keahey OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego,
 Cloud computing  Workflow  Workflow lifecycle  Workflow design  Workflow tools : xcp, eucalyptus, open nebula.
Building service testbeds on FIRE D5.2.5 Virtual Cluster on Federated Cloud Demonstration Kit August 2012 Version 1.0 Copyright © 2012 CESGA. All rights.
Flexibility and user-friendliness of grid portals: the PROGRESS approach Michal Kosiedowski
INFSO-RI Module 01 ETICS Overview Alberto Di Meglio.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
CLASS Information Management Presented at NOAATECH Conference 2006 Presented by Pat Schafer (CLASS-WV Development Lead)
| nectar.org.au NECTAR TRAINING Module 2 Virtual Laboratories and eResearch Tools.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
ETICS An Environment for Distributed Software Development in Aerospace Applications SpaceTransfer09 Hannover Messe, April 2009.
UPV-IBM’S BIG DATA OBSERVATORY & HADOOP INFRASTRUCTURE MANAGEMENT Damian Segrelles, Germán Moltó & Ignacio Blanquer,
Overview of the global architecture Giacinto DONVITO INFN-Bari.
Agility Dev TestDeploy Learn Agility.
EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number WG on Python and WG on Workflows.
DIRAC for Grid and Cloud Dr. Víctor Méndez Muñoz (for DIRAC Project) LHCb Tier 1 Liaison at PIC EGI User Community Board, October 31st, 2013.
Federated Cloud: Computing UPVLC-I3M Effort allocated: 6 pms. Proposal: Integration of an Infrastructure Broker with self-configuration and auto-scaling.
Deploying Elastic Java EE Microservices in the Cloud with Docker
EGI-InSPIRE RI EGI Compute and Data Services for Open Access in H2020 Tiziana Ferrari Technical Director, EGI.eu
EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.
A Data Engine for Grid Science Gateways Enabling Easy Transfers and Data Sharing Dr. Marco Fargetta (1), Mr. Riccardo Rotondo (2,*), Prof. Roberto Barbera.
A cloud-based architecture for supporting medical imaging analysis through INDIGO-DataCloud Ignacio Blanquer, Germán Moltó, Miguel Caballer (UPV) Luis.
CERN Openlab Openlab II virtualization developments Havard Bjerke.
Petr Škoda, Jakub Koza Astronomical Institute Academy of Sciences
PaaS services for Computing and Storage
CMS Experience with Indigo DataCloud
Job Scheduling and Runtime in DLWorkspace
Accessing the VI-SEEM infrastructure
Building ARM IaaS Application Environment
Ecological Niche Modelling in the EGI Cloud Federation
User Interfaces: Science Gateways, Workflows and Toolkits
The First INDIGO-DataCloud Software Release
StratusLab First Periodic Review
Tools and Services Workshop
Joslynn Lee – Data Science Educator
The PaaS Layer in the INDIGO-DataCloud
Overview of the global architecture
Infrastructure Orchestration to Optimize Testing
Working With Azure Batch AI
IaaS Layer – Solutions for “Enablers”
Docker Birthday #3.
Management of Virtual Machines in Grids Infrastructures
StratusLab Final Periodic Review
StratusLab Final Periodic Review
Donatella Castelli CNR-ISTI
PaaS Core Session (Notes from UPV)
Bridges and Clouds Sergiu Sanielevici, PSC Director of User Support for Scientific Applications October 12, 2017 © 2017 Pittsburgh Supercomputing Center.
Processing of Images: Orchestrating an Elastic Cloud (
Cloud Computing Platform as a Service
Grid Portal Services IeSE (the Integrated e-Science Environment)
Action U-E-5 Technical Coordination – User Technical Support
EGI Applications Database VM Operations Dashboard
Serverless Architecture in Azure
An easier path? Customizing a “Global Solution”
Management of Virtual Machines in Grids Infrastructures
PROCESS - H2020 Project Work Package WP6 JRA3
Acutelearn Azure Administration Training in Hyderabad Classroom Training Instructor led trainings at Acutelearn premises Corporate Training Custom tailored.
Securing Cloud-Native Applications Jason Schmitt CEO
Case Study: Algae Bloom in a Water Reservoir
Module 01 ETICS Overview ETICS Online Tutorials
Cloud Computing: Concepts
Overview of Workflows: Why Use Them?
MMG: from proof-of-concept to production services at scale
OpenStack Summit Berlin – November 14, 2018
Harrison Howell CSCE 824 Dr. Farkas
Introduction to the SHIWA Simulation Platform EGI User Forum,
Photon & Neutron working meeting
Presentation transcript:

Population Imaging Use Case - EuroBioImaging Ignacio Blanquer Ferrán Borreguero Universitat Politècnica de València

Medical Imaging Biobanks External subproj. team Description Resource needs SW dependencies Data Request Access Request Scientific & Technical Committee Regional PACS Customised Volume Data Manager Tech. Manager Data request Updated SW deps & resource needs Evaluate Mount Infrastructure Brokering SW image Deploy Provision SW deps BIMCV (an EuroBioImaging ESFRI node) manages a population database from an area of 5 Million people. BIMCV will receive applications for projects E.g. Training a set of models for the automatic segmentation of bone tissues in osteoporotic women of an age above 70, including sound control subjects. BIMCV will provide research data for those projects and a infrastructure to process them However, BIMCV has limited resources to deal with the computations of the different pilots. BIMCV is seeking for a model to provide services rather than data and implementing a closer follow-up of the activities. ceib.san.gva.es/bimcv EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

User Stories US#1777: I will need to store data permanently. US#1771: I will need convenient access to this storage. US#1756: I will need to protect access to data to selected users. US#1774: I need remote access to the data produced without downloading it all. US#1753: I will need to run legacy applications that use POSIX filesystems. End-User / Application developer US#1765: My executions will involve high-throughput jobs. I will need that the amount of resources to dynamically fit the workload. US#1762: I will need to compose and customize processing workflows with my own parameters and tools. US#1768: I need direct access to the terminal of the processing resources. EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

Image Processing integration Overview Onedata /upload Cluster /apply web service api Virtual Machine slurm Radiologist /create EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

Medical Imaging Biobanks – Elastic batch queue Adds the complexity of deploying an elastic batch queue New TOSCA: https://github.com/indigo-dc/tosca-types/blob/master/examples/eubiosteo_server.yaml Deals with both VM for the nodes of the batch queue and containers for the execution of the biomarkers Skips the need of running Docker under privileged mode. Addresses the need of respecting IPR of users’ code by injecting it on generic containers. Uses ONEData volumes for both images and software code. Runs jobs on a batch queue submitted through a web service. EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

Data volume integration slurm mount Virtual Machine No privileges required on Docker ubuntu 16 EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

Medical Imaging Biobanks – architecture HIS Proj Volume GitHub Docker Hub Dockerfile .../indigo- dc/eubiosteo Automated Docker Built image Ansible Play Role …/indigo- dc/ansible-role- eubiosteo Ansible Galaxy Ansible Recipe trigger calls includes Extract 1 Proj. Volume gitHub DockerHub Galaxy Verification Annotation Anonymization Research Repository Upload Image Data Bank OneData Extract Nifty & Health Metadata Proj. Volume Own containers Own Images Data Manager Local Disk IaaS mount pull 3 Define Processing code Terms of Usage Deploy WN WN Front/end Orchestrator Generic INDIGO components Submit Project Technical Manager …/tosca-types/ …/eubiosteo_server.yaml 4 Web Portal End-users TOSCA EUBIOSTEO INDIGO pilot 2 OneData Project Director

EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017 Web interface EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017 Conclusions Despite that EUBIOSTEO use case is tailored for Medical Imaging, it shares many general requirements with most of the applications INDIGO-DC offers BIMCV Availability of an efficient batch queue. Convenient & secure access to Remote Volumes accessible through POSIX. Convenient integration of user-specific code, including IPR restrictions. Availability of a comprehensive endpoint to interact with. Single sign-on, integrated deployment, QoS. It constitutes the access model and application delivery pattern for the BIM-CV node. EGI Conf & INDIGO Summit 2017 - Catania - 10/5/2017

We are ready to share our experience! https://www.indigo-datacloud.eu Better Software for Better Science.