Diego Scardaci EGI Technical Outreach Expert EGI Technical Roadmap Diego Scardaci EGI Technical Outreach Expert

Technical Coordination Technical coordination boards: Coordination of the requirements, assessment, delivery and verification of software technology technical roadmaps definition contribute to continual service improvement. Current TCBs: AAI (lead by Nicolas Liampotis) Cloud (lead by Enol Fernandez) Data (lead by Matthew Viljoen) Core and collaborative services (lead by Diego Scardaci) EGI Conference 2017

TCB AAI Includes representatives from various EGI service areas, UCST, EGI operations, EGI Security Group and technology providers. Goals: Enable users to access EGI services and resources using their existing credentials from their Home Organisations (e.g. eduGAIN) Support authorised access to protected resources based on VO/group membership and role information Aggregate user attributes from different sources, including community-managed attribute providers Support the linking of multiple external identities to a persistent, non-reassignable, unique user identifier within the EGI infrastructure Associate a Level of Assurance (LoA) to each authenticated identity in the EGI infrastructure Provide protocol translation mechanisms to hide the complexity of different protocols/technologies from EGI services EGI Conference 2017

Attribute Authorities AAI CheckIn service SAML 2.0 OpenID Connect EGI Services Supported IdPs: SAML2.0: eduGAIN OIDC/OAuth2: Google, Facebook, LinkedIn, ORCID Supported RPs: SAML2.0 & OIDC Attribute Authorties SAML2.0 Attr. Query, REST, LDAP, SQL Support for LoA IdP SP User Enrollment IdP Discovery User Consent User CheckIn IdP/SP Proxy SAML 2.0 SAML 2.0 REST LDAP SQL Social IDs eGOV IDs Attribute Authorities OpenID Connect JRA1 E-Infrastructure Commons

AAI Roadmap Short term (2017) Long term (2018-2020) Translation of VO information (SAML, OIDC) into VOMS proxies Provisioning of VOMS information through SAML and OIDC interfaces (from VOMS to SAML/OIDC) Provide user documentation code for getting certificates through RCauth.eu User enrolment and account linking Long term (2018-2020) (New) RCAuth CA Master Portal Enhanced High Availability Support Web UI harmonization and branding support for the EGI CheckIn Service (De-)provisioning and continuous update of user account information Interoperability with EUDAT B2ACCESS Self-service interface for managing OIDC access tokens Self-service web interface for registering OIDC & SAML based SPs Standalone VO/Group Management Service Support for centralised fine grained authorization EGI Conference 2017

TCB Cloud Includes representatives from various EGI user support, EGI operations, Cloud providers and technology providers. Goal: Provides the focus for the technologies that will be used within the EGI production infrastructure to deliver distributed cloud computing services for the research communities Liasons: EGI Fedcloud task force, EGI Security Coordination team, TCB AAI, UCB, OMB, SSB, Commercial cloud providers (see next slide) EGI Conference 2017

EGI FederatedCloudArchitecture IaaS Federation EGI Federation services: Accounting, Monitoring, Configuration Database, Information Discovery, VM Marketplace EGI AAI Cloud Management Framework IaaS API IaaS Federated Access Tools Community Platforms AppDB VMOps EGI Conference 2017 - Cloud Roadmap

IaaS Federated Access Tools Provide access to the heterogeneous IaaS frameworks: IaaS provisioning systems that allow to define infrastructure as code and manage and combine resources from different providers, thus enabling the portability of application deployments between them Smart brokers providing matchmaking for workloads to available providers Cloud Management Software that provides a unified console for accessing resources and deploy workloads following a set of user-defined established policies (e.g. Scalr or RightScale)

Cloud Roadmap Short term (2017) Long term (2018-2020) Usability: UIs, IaaS Federated Access Tools, Data Management Integration with commercial cloud Integration tools: EGI CheckIn, VM Image replication, Monitoring, Information discovery, Accounting Improve relationship with other initiatives: survey the landscape, increase participation in OpenStack (scientific WG, Open Research Cloud Declaration) Long term (2018-2020) maintenance and evolution of components enabling the federation New execution models: Preemptible instances, Fair-share scheduling OCCI evolution Automated Security check of VM images New features: Filesystem as a Service, Container orchestration, Data Analytics, HTC cluster, VPN as a Service, DNS as Service, LB as a Service, Enable HPC/bare metal/accelerators on cloud EGI Conference 2017

TCB Data Includes representatives from various EGI user support, EGI operations and technology providers Goals: Publication of open research data based on policies Make large data sets available without transferring them completely Enabling complex metadata queries Integration of the open data access data management with community portals Data identification, linking and citation Enabling sharing of data between researchers under certain conditions Sharing and accessing data across federations Data provenance EGI Conference 2017

Open Data Platform Interactions Public Services For Data Discovery 2: opendata publish collection Data-set-1.1 -> DOI.1 3: discover data -> DOI.1 3: discover data -> DOI.1 Snapshot Data-set-1.1 Data-set-1.1 Mounted to /localdir/ 4: Visit Collection Web Page (HTTP) 5: opendata mount remote DOI.1 /localdir/ 1: opendata create snapshot Data-set-1 6: opendata fork DOI.1 Lazy Replication Private Resources Data–set-1 Cloned Data-set-1.1 Private Resources

DataHub comes online (Oct ‘16) Host experimental or temporary scientific data and enable easy access to it by appropriate scientific applications. Distributed platform for discovering publicly available data collection available on EGI Infrastructure Unified access to reference scientific data of public interest. Data here could mean datasets ­ a collection of data/files/filesets at a level of granularity considered useful to user communities. Data repository Data provider 3 Storage provider

EGI DataHub benefits existing data providers Greater visibility for existing data providers Discoverability and exploitation of data Publishing and long term preservation Domain level Data repository EGI FedCloud VM VM VM From the point of view of a domain level data repository. Many store data, very few bring the data to computing – an issue with big datasets. POSIX/S3 Shared storage`

Don’t forget data intensive computing across multiple clouds Bringing data to computing (e.g. FedCloud) Private cloud 1 Institutional cloud 2 Public cloud VM VM VM VM VM VM VM VM VM POSIX/S3 POSIX/S3 POSIX/S3 Shared storage Brings the possibility of scalable, data intensive computing to communities regardless of their domain. Seemless access to multiple clouds – private, institutional, public. Multiple VMs accessing the same data via shared storage SPANNING these clouds. Accessible via the EGI DataHub Possibility of publishing data via a DOI and depositing it into a long term archive such as B2SAFE from EUDAT

Data Roadmap Initial proposed activities: Short term (2017) Data and Computing unification, outreach and dissemination, pilots and testing, policy, transfer, preservation, DMPs, Short term (2017) All INDIGO-DataCloud use cases planning to use EGI infrastructure are migrated from INDIGO pilot preview testbed to EGI infrastructure EGI ODP and EGI DataHub fully production ready serving end users Long term (2018-2020) Prototype developed and running demonstrating value added services of exploiting DOIs to open data (and possibly software) using OpenAire EGI Data Preservation (DP) services implemented and available to end users making use of EUDAT or Zenodo as external long term archives suitable for data preservation. EGI ODP fully tested and usable by all EOSC pilots for active computation of data intensive workflows and for long term preservation and continuing exploitation of data. EGI infrastructure used for providing a European Data Orphanage EGI Conference 2017

Core and Collaborative services TCB New! First meeting in June 2017 Includes representatives from various EGI user support, EGI operations and technology providers Goals: Maintenance and evolution of the EGI Core platform: Service registry, Monitoring, Accounting, Ops Portal, Messaging, Security tools Maintenance and evolution of the EGI Collaborations platform: AppDB, Applications on Demand service, Marketplace Integration of the new EGI AAI in the EGI tools EGI Conference 2017

References EGI Technical coordination boards AAI Roadmap Cloud Roadmap https://wiki.egi.eu/wiki/Technology_Coordination_Board AAI Roadmap http://go.egi.eu/AAI-roadmap Cloud Roadmap http://go.egi.eu/cloud-roadmap EGI Conference 2017