1. The EGI Applications On Demand (AoD) service
Giuseppe La Rocca Technical Outreach Expert
Gergely Sipos Customer and Technical Outreach Manager
Webinar:
Service:
Documentation:
Contact

2. The EGI Applications on Demand (AoD) service
Agenda
Agenda:
The EGI Applications on Demand (AoD) service

3. Objectives of this webinar
1.) Introduce the EGI Applications On Demand service to:
Architect of RIs
Researchers
NGIs/User Support Teams
2.) Invite:
Application developers to extend the the «Service» with new applications
Science Gateways/portals providers to extend the «Service» with new Virtual Research Environments (VREs)/Science Gateways;
Cloud and HTC providers to join the «Service» with their capacities;
NGI representatives to become members of the EGI User Support team.
The EGI Applications on Demand (AoD) service

4. The EGI Applications on Demand service – Motivations

5. EGI Foundation (Amsterdam)
EGI today
EGI Foundation (Amsterdam)
820,000 CPU Cores
560 PB of disk and tape storage
23 Cloud providers
325 resource providers
48,000 users
+4,000 research papers
The EGI Applications on Demand (AoD) service

6. EGI serves researchers and innovators
Size of individual groups
WLCG
ELI
CTA
ELIXIR
EPOS
EISCAT_3D
BBMRI
CLARIN
LOFAR
EMSO
LifeWatch
ICOS
CORBEL
ENVRIplus …
VRE projects
OpenDreamKit
WeNMR
DRIHM
VERCE
MuG
AgINFRA
CMMST
LSGC
SuperSites Exploitation
Environmental sci.
neuGRID …
PeachNote
CEBA Galaxy eLab
Semiconductor design
Main-belt comets
Quantum pysics studies
Virtual imaging (LS)
Bovine tuberculosis spread
Convergent evol. in genomes
Geography evolution
Seafloor seismic waves
3D liver maps with MRI
Metabolic rate modelling
Genome alignment
Tapeworms infection on fish …
Agroknow
CloudEO
CloudSME
Ecohydros
gnubila
Sinergise
SixSq
TEISS
Terradue
Ubercloud …
ESFRIs, FET flagships
Multinational communities, (e.g. H2020 projects)
Industry, SMEs
‘Long tail of science’
The EGI Applications on Demand (AoD) service

7. Barriers of application access and reuse
Fragmented landscape of ‘catch-all Virtual Organisation’ services (35 VOs at the moment)
Barrier for broad adoption
Abandoned/un-sustained services in some of these VOs
Barrier for use
‘Reinventing the wheel’ (same applications, frameworks in several VOs)
Waste of effort
Complicated access (X.509)
Barrier for usability
The EGI Applications on Demand (AoD) service

8. Main design requirements for the service
100% coverage: anyone with internet access can become a user (no X509 certificate; no NGI/VO affiliation)
Secure: provide high level of traceability of user’s activities
Open and scalable: can scale up to support large number of application providers, resource providers, technology providers and users
Resource providers can join with their own resources
Science Gateways/VRE providers can join with custom tools
Reuse existing technological building blocks as much as possible
User-friendly: intuitive GUI, local user support
The EGI Applications on Demand (AoD) service

9. The EGI Applications on Demand (AoD) service

10. What does the Service offer ?
Scientific applications that are offered “as a services” through online graphical environments
Science Gateways and Virtual Research Environments that offer integrated development environments to port custom applications with High-throughput computing and cloud resources
Cloud and High-throughput compute resources suited for both compute/data intensive applications and for hosting of scientific services
Online storage resources for storing scientific data that serve as input and output for computational jobs
Access management system and a network of Consultants who can provide guidance on the use of the service
The EGI Applications on Demand (AoD) service

11. The EGI Applications on Demand (AoD) service
Service components
The User Registration Portal (URP)
Application portals, science Gateways, VREs
Catania Science Gateway (CSG)
WS-PGRADE/gUSE
Elastic Cloud Computing Cluster (EC3)
Scientific applications (hosted in the gateways/portals/VREs)
R, GNU Octave, The Semantic Search Engine, Chipster, ClustalW2, Galaxy, AutoDock Vina, NAMD, Jupyter
A pool of resources that the applications can use
Cloud
HTC (batch computing)
A Credentials Management System
The EGI Applications on Demand (AoD) service

12. The User Registration Portal (URP)
The URP is the entry point to access the Service and uses the available applications
URP AUP
User Registration Portal (URP) access.egi.eu
User UID
The user:
Logs in using his/her social accounts OR with EGI SSO accounts
Provides background information (affiliation, research use case, links)
Submits a request for application access with resource allocation
EGI/NGIs:
Approves the user request (remote identity vetting whenever possible)
Monitors application/resource usage
After the user’s request is approved, an unique UID is assigned to uniquely identify the user across all the service components
The EGI Applications on Demand (AoD) service

13. How to access the AoD service ? http://access.egi.eu
Approval by distributed Support Team
1. Register with social or EGI SSO account
2. Complete user profile and submit access request
3. Access or develop applications
The EGI Applications on Demand (AoD) service

14. Compute and storage resources
The pre-allocated, open pool of resources is operated by various geographically distributed EGI Resource Centres
both cloud and HTC resource centres have accepted to enable the vo.access.egi.eu VO on their resources.
The EGI Applications on Demand (AoD) service

15. Science Gateways/VREs
To access and use pre-defined applications
To port additional applications, such as
Computational workflows or parameter sweeps  WS-PGRADE
Applications with rich GUI  Catania Science Gateway
Cloud applications  EC3
Science Gateways/Portals are configured to:
Consume authorization information from the User Registration Portal (user UID) and,
Generate short-term proxies from robot certificates
For the traceability of user identities proxies have been extended creating Per-User Sub-Proxies (PUSP) (see next slide)
The EGI Applications on Demand (AoD) service

16. The EGI Applications on Demand (AoD) service
Science Gateway/CSG
The Catania Science Gateway (CSG) provides small groups/researchers with the possibility to seamlessly execute scientific applications on the resources supporting the catch-all VO
The EGI Applications on Demand (AoD) service

17. Science Gateway/WS-PGRADE
The WS-PGRADE Science Gateway offers a workflow-oriented Science Gateway framework with different customization methodologies. - Developed by MTA SZTAKI and released under the Apache 2.0 license.
The EGI Applications on Demand (AoD) service

18. The EGI Applications on Demand (AoD) service
Platform/EC3
The Elastic Cloud Computing Cluster (EC3) is an open-source software platform to dynamically deploy complex scientific virtual computing infrastructures on top of Infrastructure as a Service Clouds - Developed by the Polytechnic University of Valencia (UPV)
The EGI Applications on Demand (AoD) service

19. OLA agreements with resource providers
OLA agreements with cloud resource providers
Cloud Provider
Resources
OLA Status
INFN-CATANIA
20 vCPU cores, 50GB RAM, 1TB storage
Signed
INFN-BARI
15 vCPU cores, 30GB RAM, 1TB storage
CESGA
32 vCPU cores, 64GB RAM, 2TB of storage
BIFI
100 vCPU cores, 100GB RAM, 2TB of storage
OLA agreements with HTC resource providers
HTC Provider
Resources
OLA Status
INFN-CATANIA
1M HEPSPEC, 1GB RAM per core, 100GB storage
Signed
INFN-BARI
0.5M HEPSPEC, 2GB RAM per core, 100GB storage
CESGA
1M HEPSPEC, 1GB RAM per core, 2TB storage
CYFRONET-LCG2
5M HEPSPEC, 3GB RAM per core, 500GB storage
BEgrid-ULB-VUB
5M HEPSPEC, 2GB RAM per core, 500GB storage
The EGI Applications on Demand (AoD) service

20. OLA agreements with technology providers
Service
GOCDB registration
OLA Status
Monitor in ARGO
GGUS Support Unit
User Registration Portal (URP)
Registered
Signed
Active
Created
Catania Science Gateway (CSG)
WS-PGRADE/gUSE
Elastic Cloud Computing Cluster (EC3)
The EGI Applications on Demand (AoD) service

21. Credential Management System
In the back-end there is still X509, but this is transparent for the users
Credentials Management System generates a per-user-sub-proxy (PUSP) from an X509 robot certificate.
Per-User Sub-Proxy (PUSP) identifies the user
Proxy CN includes the user ID
Accounting information can be grouped by this ID
Users can be suspended by the ID
The EGI Applications on Demand (AoD) service

22. Applications/tools integrated so far
The EGI Applications on Demand (AoD) service

23. The EGI Applications on Demand (AoD) service
User requests so far
29 requests from 16 countries
Last updates June 13, 2017
NIL contact: Macedonia, Croatia, Netherlands and Czech Republic.
The EGI Applications on Demand (AoD) service

24. Value proposition Cloud, HTC, storage sites Users Support teams
For Application Developers:
> Add application to Science Gateway/hosting framework and share with other users
For Researchers
> Use applications from provided library
Cloud, HTC, storage sites
Application hosting framework 1
Application hosting framework 2
Application hosting framework 3
Application hosting framework X
User Registration Portal (URP)
Users
Support teams
App. 1
App. K …
App. L
App. M
App. N
Identity vetting
EGI Accounting system
Usage stats
Approve,suspend users
For NGIs
Facilitate collaboration with users
For resource providers
> Offer applications to your own user groups by plugging own compute/storage in the backend
Scientific publications
The EGI Applications on Demand (AoD) service

25. The EGI Applications on Demand (AoD) service
How did we get here ?
Designed in late 2014
Demonstrated in Nov. 2015
Reached Alpha in Jan. 2017
Opened in Beta in May 2017
Webinar Jun 2017
What’s coming
Improve the list of scientific applications available:
JupyterHub aaS, Galaxy aaS
Add AppDB VMOps as another ‘gateway’
The EGI Applications on Demand (AoD) service

26. The EGI Applications on Demand (AoD) service
Get involved
AoDs is not here to replace national services
AoDs is here to facilitate reuse and collaboration
How can You participate:
Apply for access as a user
Join the user vetting and support team
Integrate additional applications
Connect a portal/VRE/gateway
Join with cloud or HTC resources
The EGI Applications on Demand (AoD) service

27. Do you want to get in touch ? Contact us at support@egi.eu
Credits
Do you want to get in touch ? Contact us at
The EGI Applications on Demand (AoD) service

28. The EGI Applications on Demand (AoD) live demo