1. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 Federated Cloud Status David Wallom University of Oxford Chair, EGI Federated Cloud

2. www.egi.eu EGI-InSPIRE RI-261323 Value proposition The EGI Federated Cloud, a federation of institutional private Clouds, offering Cloud Services to researchers in Europe and worldwide A single cloud system able to Scale to user needs Integrate multiple different providers to give resilience Prevent vendor lock-in Enable resource provision targeted towards the research community Standards based federation of IaaS cloud: Exposes a set of independent cloud services accessible to users utilising a common standards profile Allows deployment of services across multiple providers and capacity bursting

3. www.egi.eu EGI-InSPIRE RI-261323 EGI Cloud Infrastructure EGI Core Platform Federated AAI Service Registry MonitoringAccounting EGI Cloud Infrastructure Platform Instance Mgmt 3 Information Discovery GSI GLUE2 OCCI Storage Management CDMI Cloud Management Stacks (OpenStack, OpenNebula, Synnefo, …) SAM UR Help and Support Security Co- ordination Training and Outreach EGI Collaboration Tools EGI Application DB Image Repository EGI Cloud Service Marketplace OVF Sustainable Business Models

4. www.egi.eu EGI-InSPIRE RI-261323 EGI Cloud Infrastructure EGI Core Platform Federated AAI Service Registry MonitoringAccounting EGI Cloud Infrastructure Platform Instance Mgmt 4 Information Discovery GSI GLUE2 Cloudinit Storage Management CDMI Cloud Management Stacks (OpenStack, OpenNebula, Synnefo, …) SAM UR Help and Support Security Co- ordination Training and Outreach EGI Collaboration Tools EGI Application DB Image Repository EGI Cloud Service Marketplace OVF Sustainable Business Models OCCI

5. www.egi.eu EGI-InSPIRE RI-261323 Cloud Capabilities 1.Manage VM instances Consistent operations across multiple different CMF Inclusion of Contextualisation Integrated with OCCI V1.2 More from Boris 3.Data access/transfer interface Minimal useful data operation set available More from Christos 4.Cloud service information federation Standardise GLUE2 extensions 5.Resource consumption management Upgraded to latest SSM and rolling out Need storage accounting 6.Cloud service availability monitoring Continue probe development for new services and new capabilities 5

6. www.egi.eu EGI-InSPIRE RI-261323 Cloud Capabilities 7.Federated AAI User requirements are key, survey ongoing 8.VM Image Management Production marketplace More from Marios 9.Brokering Get user communities using these! 10.SLA & Business Models - Sustainability Part of p4u activity 11.Security Surveying technologies and resource providers as part of certification 6

7. www.egi.eu EGI-InSPIRE RI-261323 Geographical dispersion 5/14 12 countries provide 15 certified resources –Czech Republic, Germany, Greece, Hungary, Italy, Macedonia, Poland, Slovakia, Spain, Sweden, Turkey, United Kingdom 2 countries currently integrating –Croatia, Finland 5 countries interested –Bulgaria, France, Israel*, The Netherlands, Switzerland Worldwide interest –South Africa* (SAGrid) –South Korea* (KISTI) * Not shown on map

8. www.egi.eu EGI-InSPIRE RI-261323 Geographical dispersion 9/14 12 countries provide 19 certified resources –Czech Republic, Germany, Greece, Hungary, Italy, Macedonia, Poland, Slovakia, Spain, Sweden, Turkey, United Kingdom 5 countries currently integrating –Croatia, France, Finland, Portugal, South Korea* (KISTI) 5 countries interested –Bulgaria, Croatia, Israel*, The Netherlands, Switzerland, + more resources + technologies from countries already engaged Worldwide interest –Australia* (NeCTAR) –South Africa* (SAGrid) –United States* (NIST, NSF Centres TACC, Arizona) * Not shown on map

9. www.egi.eu EGI-InSPIRE RI-261323 FedCloud in Numbers 15 certified Resource Providers –100 Percent IT Ltd, United Kingdom –The University of Zaragoza’s Institute for Biocomputation and Physics of Complex Systems (BIFI), Spain –Academic Computer Centre CYFRONET AGH (Cyfronet), Poland –The Supercomputing Centre of Galicia (CESGA), Spain –The Czech Education and Scientific NETwork (CESNET), Czech Republic –Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG), Germany –Consejo Superior de Investigaciones Científicas (CSIC) / Instituto de Física de Cantabria (IFCA), Spain –Okeanos by the Greek Research and Technology Network (GRNET), Greece –Institute of Informatics, Slovak Academy of Sciences (II SAS), Slovakia –Italian National Institute of Nuclear Physics (INFN), Division of Catania, Italy –The Royal Institute of Technology, Sweden (KTH), Sweden –Italian National Institute of Nuclear Physics (INFN), Division of Bari, Italy –The Hungarian Academy of Sciences Institute for Computer Science and Control (MTA SZTAKI), Hungary –Turkish Academic Network and Information Centre (ULAKBIM), Turkey –The European Information & Innovation Centre (UKIM), Macedonia 152 individuals participating and contributing to the Federated Cloud group –130 conference calls (Sep 2011 – May 2014) –14’800 VM instantiations accounted for in the last year

10. www.egi.eu EGI-InSPIRE RI-261323 FedCloud in Numbers 19 certified Resource Resources from 17 Providers –100 Percent IT Ltd, United Kingdom –The University of Zaragoza’s Institute for Biocomputation and Physics of Complex Systems (BIFI), Spain –Barcelona Supercomputing Centre, Spain –Academic Computer Centre CYFRONET AGH (Cyfronet), Poland –The Supercomputing Centre of Galicia (CESGA), Spain –The Czech Education and Scientific NETwork (CESNET), Czech Republic –Forschungzentrum Juelich, Germany –Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG), Germany –Consejo Superior de Investigaciones Científicas (CSIC) / Instituto de Física de Cantabria (IFCA), Spain –~Okeanos by the Greek Research and Technology Network (GRNET), Greece –Institute of Informatics, Slovak Academy of Sciences (II SAS), Slovakia –Italian National Institute of Nuclear Physics (INFN), Division of Catania, Italy –The Royal Institute of Technology, Sweden (KTH), Sweden –Italian National Institute of Nuclear Physics (INFN), Division of Bari, Italy –Italian National Institute of Nuclear Physics (INFN), Division of Padova, Italy –The Hungarian Academy of Sciences Institute for Computer Science and Control (MTA SZTAKI), Hungary –Turkish Academic Network and Information Centre (ULAKBIM), Turkey –The European Information & Innovation Centre (UKIM), Macedonia –Centro Extremeño de Tecnologías Avanzadas CIEMAT (CETA-CIEMAT), Spain 158 individuals participating and contributing to the Federated Cloud group –>150 conference calls (Sep 2011 – Sept 2014) –78,422 VM instantiations accounted for in the last year –190,513 CPU Hours on certified resources Currently no storage accounting though sites have examples of heavy utilisation

11. www.egi.eu EGI-InSPIRE RI-261323 Accounting 11 156276 VMs launched across 24 clouds Current Usage Record Content Site User: local user/group, Digital Certificate DN VMUUID (VM instance identifier) Start (and End) Time Status Wall and CPU Duration VM source image Cloud Type

12. www.egi.eu EGI-InSPIRE RI-261323 Accounting 12 217352 VMs launched across 25 clouds Usage Record Content Site User: local user/group, Digital Certificate DN VMUUID (VM instance identifier) Start (and End) Time Status Wall and CPU Duration VM source image Cloud Type

13. www.egi.eu EGI-InSPIRE RI-261323 EGI FedCloud Communities 9/2014 Ecology – BioVeL: Biodiversity Virtual e-Laboratory Structural biology – WeNMR: a worldwide e-Infrastructure for NMR and structural biology Linguistics – CLARIN: ‘British National Corpus’ service (BNCWeb) Earth Observation – SSEP: European Space Agency’s Supersites Exploitation Platform for volcano and earthquakes monitoring (Collaboration with Helix Nebula) Software Engineering – SCI-BUS: simulated environments for portal testing Software Engineering – DIRAC: deploying ready-to-use distributed computing systems Software Engineering – Catania Science Gateway Framework Musicology – Peachnote: dynamic analysis of musical scores Earth Observation – ENVRI: Common Operations of Environmental Research infrastructures (collaboration with EISCAT3D) Geology – VERCE: Virtual Earthquake and seismology Research Ecology – LifeWatch: E-Science European Infrastructure for Biodiversity and Ecosystem Research High Energy Physics – CERN ATLAS: ATLAS processing cluster via HelixNebula More info: https://wiki.egi.eu/wiki/Fedcloud-tf:Users 13

14. www.egi.eu EGI-InSPIRE RI-261323 New EGI FedCloud Communities since launch Education – Cranfield University distributed systems course Cultural Heritage – DCH-RP management of preservation services in the cloud Hydrological Modelling – Running Hydrological models to support real time analysis Bioinformatics – ELIXIR execution of the Ensamble application in the Federated Cloud environment Systems implementations – deployment of FTK developed tools and services and data preservation Internet of Things – Smart Grid systems investigation Software Development – deployment of research PaaS RNA Sequencing – deployment of analysis engines in the cloud Physiological Modelling – Calibration, scenario mapping and development More info: https://wiki.egi.eu/wiki/Fedcloud-tf:Users 14

15. www.egi.eu EGI-InSPIRE RI-261323 Strengthening the underpinning platform Technology Evolution Broader support for open standards in Cloud management frameworks –Utilisation of rOCCI for interfaces to commercial cloud frameworks New feature additions to foundational tools depending on requests –Accounting, monitoring, service discovery, Application DB Innovate deployments of opensource PaaS Large improvements on documentation for all 3 stakeholder groups 15 Value added services for User Communities

16. www.egi.eu EGI-InSPIRE RI-261323 Conclusions 7 new communities have joined the fedcloud activities since launch Resources Launch capability– 5,000 cores, 225 TB storage Q3 Capability– ~6,000 cores, ~300 TB storage Q4 2014 (planned) – 18,000 cores,6000 TB storage (3.6x)(26x) 16

17. www.egi.eu EGI-InSPIRE RI-261323 EGI’s Cloud Vision for 2020 To support the digital European Research Area through a pan- European research infrastructure based on an open federation of reliable services that provide uniform access to computing and data resources provided by the public and private sector. 17 EGI Federated Cloud capability vision 10M cores Cloud compute 1 EB Cloud storage