1. Diego Scardaci EGI Technical Outreach Expert
EGI Technical Roadmap
Diego Scardaci
EGI Technical Outreach Expert

2. Technical Coordination
Technical coordination boards:
Coordination of the requirements, assessment, delivery and verification of software technology
technical roadmaps definition
contribute to continual service improvement.
Current TCBs:
AAI (lead by Nicolas Liampotis)
Cloud (lead by Enol Fernandez)
Data (lead by Matthew Viljoen)
Core and collaborative services (lead by Diego Scardaci)
EGI Conference 2017

3. TCB AAI
Includes representatives from various EGI service areas, UCST, EGI operations, EGI Security Group and technology providers.
Goals:
Enable users to access EGI services and resources using their existing credentials from their Home Organisations (e.g. eduGAIN)
Support authorised access to protected resources based on VO/group membership and role information
Aggregate user attributes from different sources, including community-managed attribute providers
Support the linking of multiple external identities to a persistent, non-reassignable, unique user identifier within the EGI infrastructure
Associate a Level of Assurance (LoA) to each authenticated identity in the EGI infrastructure
Provide protocol translation mechanisms to hide the complexity of different protocols/technologies from EGI services
EGI Conference 2017

4. Attribute Authorities
AAI CheckIn service
SAML 2.0
OpenID Connect
EGI Services
Supported IdPs:
SAML2.0: eduGAIN
OIDC/OAuth2: Google, Facebook, LinkedIn, ORCID
Supported RPs:
SAML2.0 & OIDC
Attribute Authorties
SAML2.0 Attr. Query, REST, LDAP, SQL
Support for LoA
IdP SP
User Enrollment
IdP Discovery
User Consent
User
CheckIn
IdP/SP Proxy
SAML 2.0
SAML 2.0 REST
LDAP
SQL
Social IDs
eGOV IDs
Attribute Authorities
OpenID Connect
JRA1 E-Infrastructure Commons

5. AAI Roadmap Short term (2017) Long term (2018-2020)
Translation of VO information (SAML, OIDC) into VOMS proxies
Provisioning of VOMS information through SAML and OIDC interfaces (from VOMS to SAML/OIDC)
Provide user documentation code for getting certificates through RCauth.eu
User enrolment and account linking
Long term ( )
(New) RCAuth CA
Master Portal Enhanced High Availability Support
Web UI harmonization and branding support for the EGI CheckIn Service
(De-)provisioning and continuous update of user account information
Interoperability with EUDAT B2ACCESS
Self-service interface for managing OIDC access tokens
Self-service web interface for registering OIDC & SAML based SPs
Standalone VO/Group Management Service
Support for centralised fine grained authorization
EGI Conference 2017

6. TCB Cloud
Includes representatives from various EGI user support, EGI operations, Cloud providers and technology providers.
Goal:
Provides the focus for the technologies that will be used within the EGI production infrastructure to deliver distributed cloud computing services for the research communities
Liasons:
EGI Fedcloud task force, EGI Security Coordination team, TCB AAI, UCB, OMB, SSB, Commercial cloud providers (see next slide)
EGI Conference 2017

7. EGI FederatedCloudArchitecture IaaS Federation
EGI Federation services:
Accounting, Monitoring, Configuration Database, Information Discovery, VM Marketplace
EGI AAI
Cloud Management Framework
IaaS API
IaaS Federated Access Tools
Community Platforms
AppDB VMOps
EGI Conference Cloud Roadmap

8. IaaS Federated Access Tools
Provide access to the heterogeneous IaaS frameworks:
IaaS provisioning systems that allow to define infrastructure as code and manage and combine resources from different providers, thus enabling the portability of application deployments between them
Smart brokers providing matchmaking for workloads to available providers
Cloud Management Software that provides a unified console for accessing resources and deploy workloads following a set of user-defined established policies (e.g. Scalr or RightScale)

9. Cloud Roadmap Short term (2017) Long term (2018-2020)
Usability: UIs, IaaS Federated Access Tools, Data Management
Integration with commercial cloud
Integration tools: EGI CheckIn, VM Image replication, Monitoring, Information discovery, Accounting
Improve relationship with other initiatives: survey the landscape, increase participation in OpenStack (scientific WG, Open Research Cloud Declaration)
Long term ( )
maintenance and evolution of components enabling the federation
New execution models: Preemptible instances, Fair-share scheduling
OCCI evolution
Automated Security check of VM images
New features: Filesystem as a Service, Container orchestration, Data Analytics, HTC cluster, VPN as a Service, DNS as Service, LB as a Service, Enable HPC/bare metal/accelerators on cloud
EGI Conference 2017

10. TCB Data
Includes representatives from various EGI user support, EGI operations and technology providers
Goals:
Publication of open research data based on policies
Make large data sets available without transferring them completely
Enabling complex metadata queries
Integration of the open data access data management with community portals
Data identification, linking and citation
Enabling sharing of data between researchers under certain conditions
Sharing and accessing data across federations
Data provenance
EGI Conference 2017

11. Open Data Platform Interactions
Public Services
For Data Discovery
2: opendata publish
collection Data-set-1.1
-> DOI.1
3: discover data
-> DOI.1
3: discover data
-> DOI.1
Snapshot
Data-set-1.1
Data-set-1.1
Mounted to
/localdir/
4: Visit Collection
Web Page (HTTP)
5: opendata mount remote
DOI.1 /localdir/
1: opendata create
snapshot Data-set-1
6: opendata fork DOI.1
Lazy Replication
Private Resources
Data–set-1
Cloned
Data-set-1.1
Private Resources

12. DataHub comes online (Oct ‘16)
Host experimental or temporary scientific data and enable easy access to it by appropriate scientific applications.
Distributed platform for discovering publicly available data collection available on EGI Infrastructure
Unified access to reference scientific data of public interest.
Data here could mean datasets ­ a collection of data/files/filesets at a level of granularity considered useful to user communities.
Data repository
Data provider 3
Storage provider

13. EGI DataHub benefits existing data providers
Greater visibility for existing data providers
Discoverability and exploitation of data
Publishing and long term preservation
Domain level
Data repository
EGI FedCloud VM VM VM
From the point of view of a domain level data repository. Many store data, very few bring the data to computing – an issue with big datasets.
POSIX/S3
Shared storage`

14. Don’t forget data intensive computing across multiple clouds
Bringing data to computing (e.g. FedCloud)
Private cloud 1
Institutional cloud 2
Public cloud VM VM VM VM VM VM VM VM VM
POSIX/S3
POSIX/S3
POSIX/S3
Shared storage
Brings the possibility of scalable, data intensive computing to communities regardless of their domain.
Seemless access to multiple clouds – private, institutional, public.
Multiple VMs accessing the same data via shared storage SPANNING these clouds.
Accessible via the EGI DataHub
Possibility of publishing data via a DOI and depositing it into a long term archive such as B2SAFE from EUDAT

15. Data Roadmap Initial proposed activities: Short term (2017)
Data and Computing unification, outreach and dissemination, pilots and testing, policy, transfer, preservation, DMPs,
Short term (2017)
All INDIGO-DataCloud use cases planning to use EGI infrastructure are migrated from INDIGO pilot preview testbed to EGI infrastructure
EGI ODP and EGI DataHub fully production ready serving end users
Long term ( )
Prototype developed and running demonstrating value added services of exploiting DOIs to open data (and possibly software) using OpenAire
EGI Data Preservation (DP) services implemented and available to end users making use of EUDAT or Zenodo as external long term archives suitable for data preservation.
EGI ODP fully tested and usable by all EOSC pilots for active computation of data intensive workflows and for long term preservation and continuing exploitation of data.
EGI infrastructure used for providing a European Data Orphanage
EGI Conference 2017

16. Core and Collaborative services TCB
New! First meeting in June 2017
Includes representatives from various EGI user support, EGI operations and technology providers
Goals:
Maintenance and evolution of the EGI Core platform: Service registry, Monitoring, Accounting, Ops Portal, Messaging, Security tools
Maintenance and evolution of the EGI Collaborations platform: AppDB, Applications on Demand service, Marketplace
Integration of the new EGI AAI in the EGI tools
EGI Conference 2017

17. References EGI Technical coordination boards AAI Roadmap Cloud Roadmap
AAI Roadmap
Cloud Roadmap
EGI Conference 2017