Remote Data Analysis Service for Photon Science Calypso+ - JRA2.

Slides:

Advertisements

Similar presentations

Prof. Natalia Kussul, PhD. Andrey Shelestov, Lobunets A., Korbakov M., Kravchenko A.

Advertisements

Introduction on WP7/WP9 Dominique PORTE 29/05/2008 Menu What is WP7? What is WP9? Goal of the brainstorming Introduction on WP7/WP9.

DevOps: Why you should care Bruce Vincent Senior Technology Strategist and IT Architect.

1 October 2013 APF Summary Oct 2013 John Hover John Hover.

DevOps and Private Cloud Automation 23 April 2015 Hal Clark.

Big Data Open Source Software and Projects Unit 0 Part B: Class Introduction Data Science Curriculum March Geoffrey Fox

1 1 Hybrid Cloud Solutions (Private with Public Burst) Accelerate and Orchestrate Enterprise Applications.

Making the Internet a Better Place for Business NIST PKI Steering Committee March 14, 2002.

Interoperability in the Cloud By Alex Espinoza

INFSO-RI Enabling Grids for E-sciencE Status of LCG-2 porting Stephen Childs, Brian Coghlan and Eamonn Kenny Grid-Ireland/EGEE October.

Windows Azure: Microsoft’s Cloud Platform By Shahed Chowdhuri.

EGEE is a project funded by the European Union under contract IST Testing processes Leanne Guy Testing activity manager JRA1 All hands meeting,

INFSO-RI Module 01 ETICS Overview Alberto Di Meglio.

European Grid Initiative Federated Cloud update Peter solagna Pre-GDB Workshop 10/11/

Eucalyptus 3 (&3.1). Eucalyptus 3 Product Overview – Govind Rangasamy.

INFSO-RI Module 01 ETICS Overview Etics Online Tutorial Marian ŻUREK Baltic Grid II Summer School Vilnius, 2-3 July 2009.

How to use Remote Desktop and Remote Support. What is remote desktop? Remotely control your computer from another office, from home, or while traveling.

FutureGrid Connection to Comet Testbed and On Ramp as a Service Geoffrey Fox Indiana University Infra structure.

Quick Introduction to NorduGrid Oxana Smirnova 4 th Nordic LHC Workshop November 23, 2001, Stockholm.

The II SAS Testbed Site Jan Astalos - Institute of Informatics Slovak Academy of Sciences.

09/02 ID099-1 September 9, 2002Grid Technology Panel Patrick Dreher Technical Panel Discussion: Progress in Developing a Web Services Data Analysis Grid.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

Building the e-Minerals Minigrid Rik Tyer, Lisa Blanshard, Kerstin Kleese (Data Management Group) Rob Allan, Andrew Richards (Grid Technology Group)

Jonathan Carroll-Nellenback.

Recipes for Success with Big Data using FutureGrid Cloudmesh SDSC Exhibit Booth New Orleans Convention Center November Geoffrey Fox, Gregor von.

Enabling Grids for E-sciencE Batch System Integration Jan Just Keijser Nikhef Amsterdam EGEE III SA3 Kickoff Meeting, May

Information Initiative Center, Hokkaido University North 11, West 5, Sapporo , Japan Tel, Fax: Management.

1 10 th February 2016 JRA2 SOLEIL MOTIVATIONS FOR ITS PARTICIPATION IN JRA Alain BUTEAU : Software for Controls and Data Acquisition group leader.

StratusLab is co-funded by the European Community’s Seventh Framework Programme (Capacities) Grant Agreement INFSO-RI Demonstration StratusLab First.

STFC in INDIGO DataCloud WP3 INDIGO DataCloud Kickoff Meeting Bologna April 2015 Ian Collier

Platform & Engineering Services CERN IT Department CH-1211 Geneva 23 Switzerland t PES Agile Infrastructure Project Overview : Status and.

Information Initiative Center, Hokkaido University North 11, West 5, Sapporo , Japan Tel, Fax: General.

DIRAC for Grid and Cloud Dr. Víctor Méndez Muñoz (for DIRAC Project) LHCb Tier 1 Liaison at PIC EGI User Community Board, October 31st, 2013.

Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.

DIRAC Distributed Computing Services A. Tsaregorodtsev, CPPM-IN2P3-CNRS FCPPL Meeting, 29 March 2013, Nanjing.

EGI-InSPIRE EGI-InSPIRE RI The European Grid Infrastructure Steven Newhouse Director, EGI.eu Project Director, EGI-InSPIRE 29/06/2016CoreGrid.

Multi-community e-Science service connecting grids & clouds R. Graciani 1, V. Méndez 2, T. Fifield 3, A. Tsaregordtsev 4 1 University of Barcelona 2 University.

EGI-InSPIRE RI EGI Compute and Data Services for Open Access in H2020 Tiziana Ferrari Technical Director, EGI.eu

EPAM Cloud Orchestration

EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.

HTCondor Accounting Update

@ Bucharest DevOps Hacker Meetup

Umbrella ID Status Mirjam van Daalen.

Budget JRA2 Beneficiaries Description TOT Costs incl travel

Deploying Galaxy in a secure environment to analyse sensitive data

Infrastructure Orchestration to Optimize Testing

Status and Challenges: January 2017

Quattor in Amazon Cloud

StratusLab Final Periodic Review

StratusLab Final Periodic Review

Bridges and Clouds Sergiu Sanielevici, PSC Director of User Support for Scientific Applications October 12, 2017 © 2017 Pittsburgh Supercomputing Center.

Revamping IdP in the Cloud pilot activities

Mirjam van Daalen:: Paul Scherrer Institut

Leanne Guy EGEE JRA1 Test Team Manager

FCT Follow-up Meeting 31 March, 2017 Fernando Meireles

Secure DevOps for Government in MOC

Connecting the European Grid Infrastructure to Research Communities

NSF : CIF21 DIBBs: Middleware and High Performance Analytics Libraries for Scalable Data Science PI: Geoffrey C. Fox Software: MIDAS HPC-ABDS.

Patrick Dreher Research Scientist & Associate Director

EOSCpilot All Hands Meeting 8 March 2018 Pisa

What’s New from Platform Computing

Platform for the long tail of science

Brian Matthews STFC EOSCpilot Brian Matthews STFC

MMG: from proof-of-concept to production services at scale

gLite The EGEE Middleware Distribution

Installation/Configuration

Status JRA2 WP24 Demonstrator of a Photon Science Analysis Service (DaaS) Mirjam van Daalen 6/28/2019 Mirjam van Daalen PSI.

Data analysis IT infrastructure status report IT Systems - Computing

Welcome to (HT)Condor Week #19 (year 34 of our project)

Umbrella ID Federated Identity for PaN facilities

Presentation transcript:

Remote Data Analysis Service for Photon Science Calypso+ - JRA2

List of expertise 1.Project Coordination + workshops (PSI/ESRF obviously) By the way: Workshop at ALBA Synchrotron: HTCondor / ARC CE Workshop 29 February to 4 March (registration open till 22/2) 2.Use (science) case definition and collation (multiple disciplines, facilities and user/industry) (Scientists) 3.Packaging for applications (SysAdmins) - Linux rpms – Extensive experience packaging – HPC analysis examples: autoproc, ccp4, xds-viewer,... - Docker – Experience in container management & packaging - KVM VM image-analysis applications packaging experience on PBS/MAUI - Distribution generation experience: Yocto distro (embedded), Debian based distro. 4.Cloud setup and deployment (SysAdmins) Some experience in private & public cloud (AWS, OpenStack, OpenNebula) 5.Configuration of site and test sites (DevOps) - Experience with EGEE SA3 with Condor Cream testing. - Experience configuring HPC clusters with SaltStack, LCGng, Puppet, Ansible and Quattor (Queue systems: Slurm, HTCondor, PBS, TorqueMAUI) i.e. Alba HPC Slurm Cluster deployed & managed with SaltStack & Cobbler. 6.Port and package applications and examples (Software engineer) Integration of existing applications and packages such as MxCube, EDNA (ESRF)… 7.Umbrella authentication (AAI+security) Under testing. Un production shortly in the User Office Portal at ALBA. 8.User portal development (Web engineers) Django-AngularJS and webservices (REST) expertise and other Javascript frameworks. Twitter Bootstrap (responsive) websites expertise as well. 9.Definition, development and distribution of mini demonstrator platform to test sites (DevOps) Interested to collaborate in definition. Not in development. In distribution as part of the testbed sites. 10.Report on results and links to HNSciCloud + EOSC (IT Engineer+SysAdmin+Software+Scientists) No experience in HNSciCloud + EOSC

HPC, HTC Cluster

Remote access… (today…) NX enterprise (remote operation) Citrix Data repatriated via SFTP… …or disks at the BL.

BL REMOTE OPERATION 1 st design. NX ENTERPRISE