EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Federated Cloud Status David Wallom University of Oxford Chair, EGI Federated Cloud

EGI-InSPIRE RI Value proposition The EGI Federated Cloud, a federation of institutional private Clouds, offering Cloud Services to researchers in Europe and worldwide A single cloud system able to Scale to user needs Integrate multiple different providers to give resilience Prevent vendor lock-in Enable resource provision targeted towards the research community Standards based federation of IaaS cloud: Exposes a set of independent cloud services accessible to users utilising a common standards profile Allows deployment of services across multiple providers and capacity bursting

EGI-InSPIRE RI EGI Cloud Infrastructure EGI Core Platform Federated AAI Service Registry MonitoringAccounting EGI Cloud Infrastructure Platform Instance Mgmt 3 Information Discovery GSI GLUE2 OCCI Storage Management CDMI Cloud Management Stacks (OpenStack, OpenNebula, Synnefo, …) SAM UR Help and Support Security Co- ordination Training and Outreach EGI Collaboration Tools EGI Application DB Image Repository EGI Cloud Service Marketplace OVF Sustainable Business Models

EGI-InSPIRE RI EGI Cloud Infrastructure EGI Core Platform Federated AAI Service Registry MonitoringAccounting EGI Cloud Infrastructure Platform Instance Mgmt 4 Information Discovery GSI GLUE2 Cloudinit Storage Management CDMI Cloud Management Stacks (OpenStack, OpenNebula, Synnefo, …) SAM UR Help and Support Security Co- ordination Training and Outreach EGI Collaboration Tools EGI Application DB Image Repository EGI Cloud Service Marketplace OVF Sustainable Business Models OCCI

EGI-InSPIRE RI Cloud Capabilities 1.Manage VM instances Consistent operations across multiple different CMF Inclusion of Contextualisation Integrated with OCCI V1.2 More from Boris 3.Data access/transfer interface Minimal useful data operation set available More from Christos 4.Cloud service information federation Standardise GLUE2 extensions 5.Resource consumption management Upgraded to latest SSM and rolling out Need storage accounting 6.Cloud service availability monitoring Continue probe development for new services and new capabilities 5

EGI-InSPIRE RI Cloud Capabilities 7.Federated AAI User requirements are key, survey ongoing 8.VM Image Management Production marketplace More from Marios 9.Brokering Get user communities using these! 10.SLA & Business Models - Sustainability Part of p4u activity 11.Security Surveying technologies and resource providers as part of certification 6

EGI-InSPIRE RI Geographical dispersion 5/14 12 countries provide 15 certified resources –Czech Republic, Germany, Greece, Hungary, Italy, Macedonia, Poland, Slovakia, Spain, Sweden, Turkey, United Kingdom 2 countries currently integrating –Croatia, Finland 5 countries interested –Bulgaria, France, Israel*, The Netherlands, Switzerland Worldwide interest –South Africa* (SAGrid) –South Korea* (KISTI) * Not shown on map

EGI-InSPIRE RI Geographical dispersion 9/14 12 countries provide 19 certified resources –Czech Republic, Germany, Greece, Hungary, Italy, Macedonia, Poland, Slovakia, Spain, Sweden, Turkey, United Kingdom 5 countries currently integrating –Croatia, France, Finland, Portugal, South Korea* (KISTI) 5 countries interested –Bulgaria, Croatia, Israel*, The Netherlands, Switzerland, + more resources + technologies from countries already engaged Worldwide interest –Australia* (NeCTAR) –South Africa* (SAGrid) –United States* (NIST, NSF Centres TACC, Arizona) * Not shown on map

EGI-InSPIRE RI FedCloud in Numbers 15 certified Resource Providers –100 Percent IT Ltd, United Kingdom –The University of Zaragoza’s Institute for Biocomputation and Physics of Complex Systems (BIFI), Spain –Academic Computer Centre CYFRONET AGH (Cyfronet), Poland –The Supercomputing Centre of Galicia (CESGA), Spain –The Czech Education and Scientific NETwork (CESNET), Czech Republic –Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG), Germany –Consejo Superior de Investigaciones Científicas (CSIC) / Instituto de Física de Cantabria (IFCA), Spain –Okeanos by the Greek Research and Technology Network (GRNET), Greece –Institute of Informatics, Slovak Academy of Sciences (II SAS), Slovakia –Italian National Institute of Nuclear Physics (INFN), Division of Catania, Italy –The Royal Institute of Technology, Sweden (KTH), Sweden –Italian National Institute of Nuclear Physics (INFN), Division of Bari, Italy –The Hungarian Academy of Sciences Institute for Computer Science and Control (MTA SZTAKI), Hungary –Turkish Academic Network and Information Centre (ULAKBIM), Turkey –The European Information & Innovation Centre (UKIM), Macedonia 152 individuals participating and contributing to the Federated Cloud group –130 conference calls (Sep 2011 – May 2014) –14’800 VM instantiations accounted for in the last year

EGI-InSPIRE RI FedCloud in Numbers 19 certified Resource Resources from 17 Providers –100 Percent IT Ltd, United Kingdom –The University of Zaragoza’s Institute for Biocomputation and Physics of Complex Systems (BIFI), Spain –Barcelona Supercomputing Centre, Spain –Academic Computer Centre CYFRONET AGH (Cyfronet), Poland –The Supercomputing Centre of Galicia (CESGA), Spain –The Czech Education and Scientific NETwork (CESNET), Czech Republic –Forschungzentrum Juelich, Germany –Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG), Germany –Consejo Superior de Investigaciones Científicas (CSIC) / Instituto de Física de Cantabria (IFCA), Spain –~Okeanos by the Greek Research and Technology Network (GRNET), Greece –Institute of Informatics, Slovak Academy of Sciences (II SAS), Slovakia –Italian National Institute of Nuclear Physics (INFN), Division of Catania, Italy –The Royal Institute of Technology, Sweden (KTH), Sweden –Italian National Institute of Nuclear Physics (INFN), Division of Bari, Italy –Italian National Institute of Nuclear Physics (INFN), Division of Padova, Italy –The Hungarian Academy of Sciences Institute for Computer Science and Control (MTA SZTAKI), Hungary –Turkish Academic Network and Information Centre (ULAKBIM), Turkey –The European Information & Innovation Centre (UKIM), Macedonia –Centro Extremeño de Tecnologías Avanzadas CIEMAT (CETA-CIEMAT), Spain 158 individuals participating and contributing to the Federated Cloud group –>150 conference calls (Sep 2011 – Sept 2014) –78,422 VM instantiations accounted for in the last year –190,513 CPU Hours on certified resources Currently no storage accounting though sites have examples of heavy utilisation

EGI-InSPIRE RI Accounting VMs launched across 24 clouds Current Usage Record Content Site User: local user/group, Digital Certificate DN VMUUID (VM instance identifier) Start (and End) Time Status Wall and CPU Duration VM source image Cloud Type

EGI-InSPIRE RI Accounting VMs launched across 25 clouds Usage Record Content Site User: local user/group, Digital Certificate DN VMUUID (VM instance identifier) Start (and End) Time Status Wall and CPU Duration VM source image Cloud Type

EGI-InSPIRE RI EGI FedCloud Communities 9/2014 Ecology – BioVeL: Biodiversity Virtual e-Laboratory Structural biology – WeNMR: a worldwide e-Infrastructure for NMR and structural biology Linguistics – CLARIN: ‘British National Corpus’ service (BNCWeb) Earth Observation – SSEP: European Space Agency’s Supersites Exploitation Platform for volcano and earthquakes monitoring (Collaboration with Helix Nebula) Software Engineering – SCI-BUS: simulated environments for portal testing Software Engineering – DIRAC: deploying ready-to-use distributed computing systems Software Engineering – Catania Science Gateway Framework Musicology – Peachnote: dynamic analysis of musical scores Earth Observation – ENVRI: Common Operations of Environmental Research infrastructures (collaboration with EISCAT3D) Geology – VERCE: Virtual Earthquake and seismology Research Ecology – LifeWatch: E-Science European Infrastructure for Biodiversity and Ecosystem Research High Energy Physics – CERN ATLAS: ATLAS processing cluster via HelixNebula More info: 13

EGI-InSPIRE RI New EGI FedCloud Communities since launch Education – Cranfield University distributed systems course Cultural Heritage – DCH-RP management of preservation services in the cloud Hydrological Modelling – Running Hydrological models to support real time analysis Bioinformatics – ELIXIR execution of the Ensamble application in the Federated Cloud environment Systems implementations – deployment of FTK developed tools and services and data preservation Internet of Things – Smart Grid systems investigation Software Development – deployment of research PaaS RNA Sequencing – deployment of analysis engines in the cloud Physiological Modelling – Calibration, scenario mapping and development More info: 14

EGI-InSPIRE RI Strengthening the underpinning platform Technology Evolution Broader support for open standards in Cloud management frameworks –Utilisation of rOCCI for interfaces to commercial cloud frameworks New feature additions to foundational tools depending on requests –Accounting, monitoring, service discovery, Application DB Innovate deployments of opensource PaaS Large improvements on documentation for all 3 stakeholder groups 15 Value added services for User Communities

EGI-InSPIRE RI Conclusions 7 new communities have joined the fedcloud activities since launch Resources Launch capability– 5,000 cores, 225 TB storage Q3 Capability– ~6,000 cores, ~300 TB storage Q (planned) – 18,000 cores,6000 TB storage (3.6x)(26x) 16

EGI-InSPIRE RI EGI’s Cloud Vision for 2020 To support the digital European Research Area through a pan- European research infrastructure based on an open federation of reliable services that provide uniform access to computing and data resources provided by the public and private sector. 17 EGI Federated Cloud capability vision 10M cores Cloud compute 1 EB Cloud storage