1. connect communicate collaborate Open Cloud eXchange (OCX) as a part of High Performance Cloud Services Delivery Infrastructure (CSDI) for Big Data Applications Yuri Demchenko (UvA) On behalf of OCX development team GN3plus JRA1 Task 2 EGI-GEANT Symposium 25-26 September 2014 Amsterdam

2. 2 connect communicate collaborate GN3+ JRA1: Network Architectures for Horizon 2020 General use cases and scenarios for Cloud services delivery to NRENs and universities – Cloud and Intercloud Infrastructure and Services Geant Open Cloud eXchange (gOCX) – OCX design principles and suggestions – Background concepts and related research gOCX Demos – TNC2014 demo (UvA, Okeanos, CloudSIGMA, SURFnet, SWITCH) – SC14 demo 16-21 Nov 2014 (gOCX + Amazon AWS) Further development Additional information Agenda

3. 3 connect communicate collaborate Task 0: Activity Leadership Task 1: Future Network Architectures Task 2: Network Architectures for Cloud Services Task 3: Network Architecture for Aggregating High-Speed Mobile Networking JRA1: Network Architectures for Horizon 2020

4. 4 connect communicate collaborate Scientific application and scientific (Big) data – LHC/HEP, genomics, astronomy, climate, 4K/UHD video, etc. Streaming high-speed high volume experimental data to labs in campus location – Direct links through campus network Distributed (Big) Scientific Data processing with MPP tools on distributed facilities – Data distributed between few locations next to local datacenters CSP and campus L0-L2 (L3) network peering – Dark fiber with termination as campus network or as CSP network VoIP – approach with mobile data access – Support mobile access network (LTE) and tunnel access to campus network Use Cases for delivering Cloud services to campus based users

5. 5 connect communicate collaborate General use case for infrastructure provisioning: Workflow => Logical (Cloud) Infrastructure Enterprise/Scientific workflow Is mapped to heterogeneous cloud infrastructure containing IaaS, PaaS components Input Data Instrum. Data Data Filtering Special Proc 1 Storage Data Special Proc 2 Data Archive Visual Present Enterprise/Scientific workflow Visuali- sation User Group A Campus A User CE Visuali- sation User Group B Campus B User CE CN Cloud IaaS Provider Cloud PaaS Provider Resource/ Service Provider VR1 VR3 VR5 VR4 VR2 VR6 VR7 Enterprise/Project based Intercloud Infrastructure Cloud 1 IaaS Cloud 2 PaaS

6. 6 connect communicate collaborate Multi/Inter- cloud infrastructure provisioning: => Intercloud Federation and gOCX functions Input Data Instrum. Data Data Filtering Special Proc 1 Storage Data Special Proc 2 Data Archive Visual Present Enterprise/Scientific workflow Visuali- sation User Group A Campus A User CE Visuali- sation User Group B Campus B User CE CN Cloud IaaS Provider Cloud PaaS Provider Resource/ Service Provider VR1 VR3 VR5 VR4 VR2 VR6 VR7 Enterprise/Project based Intercloud Infrastructure Cloud 1 IaaS Cloud 2 PaaS Open Cloud eXchange (OCX) interconnects CSP and Customer (campus) infrastructures OCX assumes collocation or collapsed backbone for connecting all OCX members Solving “last mile” problem OCX as a part of GÉANT infrastructure OCX at NREN OCX at NREN Big Data and Data Intensive Tasks

7. 7 connect communicate collaborate Direct service/inter-member peering – Re-use and leverage two known concepts: Internet eXchange Point and GLIF Open Lightpath Exchange (GOLE) – Physical Point of Presence (PoP) for providers and customers – Scalability for growing number of members – Controlled network parameters (QoS) L0-L2 network interconnection facility – Associated service should allow topology information exchange between providers and customers in a secure and consistent way No value added third party (intermediary/broker) services – Transparency for cloud based services – No involvement into peering or mutual business relations Trusted Third Party (TTP) services – To support dynamic federation establishment – Trusted Introducer for dynamic trust establishment – Open Marketplace and SLA clearinghouse for member CSP’s gOCX Definition and Operational Principles

8. 8 connect communicate collaborate OCX Topological model and Connectivity OCX L0-L2/L3 topology Any-to-any Distributed or collapsed backbone Hierarchical Topology information exchange L0-L2 (+ L3?) between members SDN control over OCX switching QoS parameters Bandwidth Speed, latency Jitter, impairment CloudCom2013Open Cloud eXchange (OCX) 8 OCX Nodes: Providers and customers OCX OCX Facilities OCX OCX backbone links/connectivity (Lo-L2)

9. 9 connect communicate collaborate OCX Trusted Third Party services TTP goals and services Enable dynamic federations establishing Service Registry and Discovery – Open Marketplace SLA repository and clearinghouse – Repository of CSP policies Trusted Certificates and CA’s Repository – Similar to TACAR (TERENA Academic CA Repository) – Trusted Introducer Service CloudCom2013Open Cloud eXchange (OCX) 9 OCX TTP Pre-established trust relation with OCX as TTP Trust relations established as a part of dynamic federation between OCX members

10. 10 connect communicate collaborate OCX location options: GEANT, NREN CloudCom2013Open Cloud eXchange (OCX) 10 CSP GÉANTNREN University L0 L1 DFlow IP/L3 L2 OCX VR3 VR5 VR4 VR6 VR7 VR3 VR5 VR4 VR6 VR7 Visuali- sation User CE Visuali- sation User CE

11. 11 connect communicate collaborate OCX Hierarchical and Distributed Topology Model CloudCom2013Open Cloud eXchange (OCX) 11 CSP L0 L1 DFlow IP/L3 L2 OCX VR3 VR5 VR4 VR6 VR7 VR3 VR5 VR4 VR6 VR7 Visuali- sation User CE Visuali- sation User CE GÉANTNREN University

12. 12 connect communicate collaborate Background concepts and research OCX concept is conceptually justified and (to be) practically proven Big Data and Cloud Computing – Complete fusion + High-Performance Scalable Computing – Multi-provider services infrastructure and inter-provider communication – Moving data, tasks and workflow => Intercloud Cloud Services Delivery Infrastructure (ICSDI) Intercloud Architecture Framework (ICAF) proposed by UvA Intercloud Federation: part of ICAF and mainstream development in Cloud Computing Cloud Computing industry is striving to solve the problem of services and content delivery from cloud provider to customer – DirectConnect by AWS and ExpressConnect by Microsoft Azure – Equinix Cloud Exchange for inter-provider business and infrastructure level integration – CSP’s Content Delivery Networks: CloudFront by AWS and CDN by Microsoft Azure – Federated IDM and Access Control – Marketplace and VMs/Services/Apps directory

13. 13 connect communicate collaborate Gartner Hypercycle for Cloud Computing and Big Data Big Data Cloud Computing

14. 14 connect communicate collaborate gOCX Development – Milestones and Next Steps Design and implementation – Started November 2013 – Followed by implementation March – May 2014 – TNC2014 Demo 19-22 May 2014, Dublin – OCX Design meeting 3-4 July 2014, Amsterdam – API definition and topology visualization (+ Opt/Net BV) – SDN based control and management over OCX Update based on the public draft discussion – Whitepaper GN3plus-JRA1-task2 (published) – Security, topology, protocols, use cases Planned demos – TNC2014, Dublin (May 2014) – SC14, USA (November 2014) – GN3plus Symposium (February 2015) – TNC2015, Porto (June 2015) CloudCom2013Open Cloud eXchange (OCX) 14 Dissemination and Standardization Standardization contribution: use case, awareness – OGF, IETF, IEEE, NIST – IEEE InterCloud Testbed (ICTB) Initiative – To consider Conference and publications – CloudCom2013 and NetCloud2013 Workshop, Bristol – TNC2014, Dublin – TNC2015, Porto (TBC) – Intercloud2015 Workshop @IC2E2015

15. 15 connect communicate collaborate gOCX Demo @TNC2014 Demo Scenario: HD video editing and streaming The University of Amsterdam (UvA) has some 4K movies that need efficient transcoding. Using their local OCX (NetherLight) the UvA can easily get access to necessary compute resources at different Cloud Service Providers via high performance dedicated network links. In the demo we use Okeanos (connected via GRNET OCX) and Cloud Sigma (connected via SWITCH OCX). The UvA created scheduling software that is able to spawn virtual machines at Okeanos or Cloud Sigma. The machines are spawned inside the L2-domain of the UvA, which gives the UvA full control.

16. 16 connect communicate collaborate Planned SC14 Demo – 16-21 November 2014, Tempe, Arizona USA Scenario: large volume 4K video processing – Parallel processing, upload and download, visualization/streaming Participants – UvA – SURFnet, GRNET, SWITCH, HEAnet (TBC) – Okeanos, CloudSigma, AWS (confirmed), Microsoft Azure (TBC) New tools to be involved – Vampire (UvA): VM placement and tasks allocation based on Bag of Tasks (BoT) – Application Layer – Topology Visualisation: External contribution by Opt/Net BV New OCX development involved – OCX API: CSP-OCX-CSP, Apps-OCX, Signaling, Topology and path exchange Testing real-life scenario: VM images compatibility, Live migration, inter-cloud/inter-processes signaling

17. 17 connect communicate collaborate Future Developments Combining SDN and OCX – OCX functions control, integration with network environment and Application layer control OCX API Definition – CSP-OCX-CSP – Application Layer to OCX – Signaling, Topology and path exchange TTP to support dynamic federations creation Application layer integration: BoT Vampire (UvA) and marketplace/brokering

18. 18 connect communicate collaborate Proposal: gOCX to SDN-gOCX Service: Connectivity pCSPs through OCX into the NREN (aCSP/aICTs) and vice versa Value proposal from SDN: Configurability (policy based) and programmability of the control/data plane Automation Compatible APIs with other SDN components

19. connect communicate collaborate Thank you Questions? More information: gOCX Whitepaper http://www.geant.net/Resources/White_Papers/Documents/MS101_M J1-2-1_Network-Architectures-for-Cloud-Services.pdf http://www.geant.net/Resources/White_Papers/Documents/MS101_M J1-2-1_Network-Architectures-for-Cloud-Services.pdf TNC2014 gOCX presentation and paper https://tnc2014.terena.org/core/presentation/21 https://tnc2014.terena.org/core/presentation/21 TNC2014 Demo video (4:50) http://www.youtube.com/watch?v=q7IAAFUcTY0&feature=youtu.be http://www.youtube.com/watch?v=q7IAAFUcTY0&feature=youtu.be