1. Status JRA2
WP24 Demonstrator of a Photon Science Analysis Service (DaaS)
Mirjam van Daalen
6/28/2019
Mirjam van Daalen PSI

2. Partners JRA2 Kick off meeting 24-26 June 2017 ESRF JRA2 CALIPSOplus
Beneficiaries
PSI (leader) Mirjam van Daalen
ESRF (co-leader) Andy Goetz
DESY Frank Schluenzen
DLS Alun Ashton
ALBA Daniel Salvat
Elettra Roberto Pugliese
EUXFEL observer
Partners JRA2
Kick off meeting June 2017 ESRF
JRA2 CALIPSOplus
Umbrella / LEAPS WG3 IT
6/28/2019
Mirjam van Daalen PSI 2

3. Objectives and structure of JRA 2 -1
Build up demonstrators for remote data analysis for typical experiments on new platform.
- The demonstrators will build on the HPC platforms of the different partners.
- Cloud based where cloud technology is deployed.
- In the other institutes it will run on standard HPC hardware.
- A web portal will be designed
- Gives users a common user experience to access analysis software
- Future connection to EOSC
6/28/2019
Mirjam van Daalen PSI 3

4. JRA2 activities up to now - 1
JRA2 Demonstrator of a Photon Science Analysis Service (DaaS)
June 24-26th:
Future Umbrella
Kick off JRA2 (D24.1)
Divison of tasks between the partners
Organisation and logistics
kick off LEAPS WG3
Preparation for document Brussels LEAPS role out
Deliverable D24.1 accomplished report on kick-off workshop
Deliverable D24.3 accomplished cross site use case requirement
Telco‘s every month 1st telco :00-17:00
LEAPS Initiative:
Current Working Group 3 „LEAPS IT“
6/28/2019
Mirjam van Daalen PSI 4

5. Task 24.1 Lead ESRF
Design of a platform for Remote Data Analysis as a Service.
The design will take into account the current situation at the partner institutes.
Increase re-usability of solutions like component based software engineering and development of scientific/algorithmic libraries.
Evaluate different container concepts as deployment strategies, like classical installation procedures, shipping of virtual machines and/or Docker containers.
D24.2 Blueprint on implementing a platform and manuals for the implementation at the different sites (M12)
Postponed by 4 Months as hiring of personnel was retarding the work
ESRF hired an expert now that will start work at the beginning of July 2018
ESRF started working already on the topic
Satellite meeting to discuss the blueprint is organised adjacent to the JRA2 face-to-face meeting at ESRF

6. Task 24.2 Lead PSI
Collect data analysis requirements and existing software solutions
Analyse which use cases are suitable candidates for providing harmonised solutions which can be transferred from one site to another site.
Do this for the use case domains selected i.e. SAXS, Tomography, Diffraction (including MX)

7. D 24.3 Cross site use case requirement report including comparison of existing solutions. (M12)
5 different use cases were selected:
CrsytFEL leading house DESY: CrystFEL is a suite of programs for processing diffraction data acquired "serially" in a "snapshot" manner, such as when using the technique of Serial Femtosecond Crystallography (SFX) with a free-electron laser source.
Ptycho Shelves leading house PSI: Ptycho Shelves is an innovative and conceptually-simple modular framework for ptychography reconstructions.
Savu leading house Diamond: Savu is a Python package to assist with the processing and reconstruction of parallel-beam tomography data.

8. D 24.3 Cross site use case requirement report including comparison of existing solutions. (M12)
5 different use cases were selected:
pyFAI leading house ESRF: The software package pyFAI has been designed to reduce SAXS, WAXS and XRPD images taken with detectors into 1D curves (azimuthal integration)
PyMca leading House ESRF: It offers all what users can expect in that field (qualitative and quantitative analysis, interactive and batch processing, large variety of data formats support -including HDF5 NeXus-) while being the only free and open source alternative among the reference applications.
D 24.3 Cross site use case requirement report including comparison of existing solutions accomplished

9. Task 24.3 Lead ESRF
Implement DaaS on the local platform at the each site with remote access to compute and storage resources using dedicated hardware resources supplied by each site (23M)
No work done yet

10. Task 24.4 Lead ALBA
Implement DaaS on the local platform at the each site with remote access to compute and storage resources using dedicated hardware resources supplied by each site
Person hired to do this at ALBA
Platform already exists
Progress on the Frontend development and integration of an example open source software as well as integration of Authentication with Umbrella.
A test will be done. Starting to have first experience with dummy processes to start docker and destroy.

11. Task 24.5 Lead DESY
Package and deploy data analysis packages for at least two Use Cases / site
Hired person to work on container environments
Experiences in setting up containerised analysis environments.

12. Task 24.6 Lead PSI
Extend and deploy the Umbrella authentication system as the standard authentication mechanism to fully support different non-web-based authentication mechanisms to be able to service a broader range of facilities (eduGain + JISC Assent) with differing needs and capabilities. (18M)
Umbrella is connected to EduGain.
The EduTeam pilot (GEANT) attribute handling can now be exploited.
Umbrella is used as a critical service at PSI and the first company from ParkInnovaare is now connected via Umbrella to use PSI’s infrastructure.
It is planned to use Umbrella
With the EduGain connection Umbrella can now bridge to various other communities such as: Elixir, Orcid, EGI CheckIn
Subcontractor is being hired. Participation in the FIM4R editors group were Umbrella is one of the involved user communities.

13. Task 24.7 Lead Elettra
Test data analysis software for Use Cases with test data and users
Will take place as soon as use cases are ready

14. JRA1 next activities May 23-24th 2018: meeting @ESRF
JRA2 Demonstrator of a Photon Science Analysis Service (DaaS)
May 23-24th 2018:
BluePrint/Umbrella/JRA2
JRA2

15. Current and future related projects
HZ2020 CALIPSOplus JRA2 DaaS for Photon Sciences
League of accelerator based photon sources LEAPS
WG3 IT next generation IT for Photo science
EU commission European Open Science Cloud EOSC
BEATS Beamline for Tomography at SESAME