NORDUnet’s views on cloud and cloud providers LHCOPN/ONE meeting Taipei, March 2016

CMS ATLAS LHC and Clouds “For example CMS is working with Amazon Web Services (AWS), via a research grant awarded to CMS earlier in 2015, as well as continuing to work on a remarkable number of projects that aim at enabling CMS to burst into extra (commercial or not) resources on demand.” “After the commissioning period, we were able to stably running at >50k cores continuously for days. AWS resources contributed to several official Monte Carlo production requests, corresponding to ~500M events to be sued for Moriond 2016. In this period, AWS contributed to more than 20% of the global CMS concurrent capacity.” ATLAS “ATLAS continues to engage proactively with possible sources of beyond-pledge CPU power. ATLAS has already run production on donated and commercial cloud resources or volunteer computing.”

Plenty of Clouds Big Players (GAFA = Google, Apple, Facebook and Amazon) Local commercial players, selling point often that they are in line with local legislation NREN working on cloud services (filling niches) Can NRENs compete with the market for generic cloud services? Might add Microsoft and Dropbox and …. DAGMAF=“Hello, Stupid” in Ducth

NRENs and Clouds Internet NREN working on cloud services NREN A Our view is that a NREN should aggregate demand from constituency, act as system integrator. Add value, for instance integrate with identity federations. Global Commercial Provider NREN A REGIONAL NETWORK Internet University Connectivity SERVICE NREN B Local Commercial Provider Might add Microsoft and Dropbox and …. DAGMAF=“Hello, Stupid” in Ducth

Clouds all over

NREN ensures connectivity for user institutions (as usual) Simplifying We can break the problem into smaller pieces using open exchange points NREN ensures connectivity for user institutions (as usual) NREN connects to one or more OPXs (and each-other) Cloud provider has NREN connectivity through: Direct connections to NREN Connections to one or more OXPs * Slide 6: First bullet: Mention NREN somehow, as we are not intending to bypass the R&E Networks. I would suggest something like: - NREN ensures connectivity for the user institutions, - NREN connects to one or more OXPs (and each other), - Cloud provider has connectivity with NRENs through:  > direct connections to NREN [Cloud Provider A & B in my picture]  > connections with one or more OXPs [Cloud Provider C]

Open exchanges

Researcher connects to OXP This one is fairly easy – under the assumption that traffic to and from universities are research traffic. Connecting a user institution to an open exchange is in most cases trivial. Most institutions are already connected to an NREN that connects to a relevant open exchange, either directly or through an aggregator network such as GÉANT, NORDUnet, or Internet2. In a few cases (such as CERN), institutions connect directly to an open exchange. In either case, network capacity and required services are in place, and are largely covered by existing costing and cost-sharing agreements. As a result, NRENs are already connecting user institutions to the mesh of inter-connected OXPs.

Cloud connects to OXP Connecting cloud providers to Open Exchanges is not as well progressed to date. For major providers, creating such connections may already be in progress (i.e., Amazon at GÉANT Open in London, Amazon and others at NetherLight, etc.). For providers not already connected, several options exist, i.e.: Large providers often have proprietary networks and will already be in locations served by an Open Exchange. Connecting such providers directly to the Open Exchange is a fairly trivial task. For a provider without the above option, it will often be possible to identify an R&E Network that can provide connectivity as a layer 2 or 2.5 with a direct link. Implementing this comes with an cost that can be directly assigned to the provider and does not involve other services. Failing the above options, it may be possible to connect a provider to the IP service of an R&E network and allowing the provider’s traffic with the open exchange traffic over shared IP, possibly using a tunnel or a VPN

Researcher connects to Cloud This one is fairly easy – under the assumption that traffic to and from universities are research traffic.

Researcher connects to Cloud The tricky bit here – as Mike was already hinting at yesterday is to know when to allow traffic to use NREN connectivity for transit between clouds. What we rely on the ability for Cloud Providers to ensure that the virtual machines allocated to research use is somehow labelled correctly, so that we can distinguish traffic and route it correctly.

Inter-exchange Inter-exchange bandwidth will have to be acquired and needs to have sufficient bandwidth for the traffic and distribution of user institutions and providers user. This is similar to the case of both LHCOPN and the trans-Atlantic links for LHCONE. Such links can be both mission specific (i.e., dedicated to WLCG), or general for all user of e.g. the European Open Science Cloud. In either case, it should be noted that major R&E Networks have already invested in ample bandwidth interconnecting the locations of the important open exchange points, and are often already using these exchanges to terminate the links. Hence, OXP-OXP links for cloud traffic can in many cases be readily provided at layer 2 or 2.5 by participating R&E networks.

Additional requirements All links and networks used must allow for Cloud-Researcher traffic, without AUP restrictions. Open Exchanges and any links used to connect commercial cloud providers must also allow for Cloud-Cloud on behalf of a researcher. Note that due to the way OXPs are used, no such requirement exist for the networks used to connect user institutions. We must resolve any cost sharing required for connecting providers to exchanges (or require providers to carry that cost), and we must resolve cost sharing for inter-exchange traffic in cases where this is not a service offered by aggregation networks. First bullet - On AUP, Sketch an example in which e.g. a scientist collects weather data from sensors, stores this on Azure and after some collection time moves the big data set from Azure to Amazon for processing. Is this commercial use? No, of course not. The NRENs need to mature themselves in the area of AUPs, and ensure that the can help this scientist getting his science done in the most optimal way, and yes that might involve carrying cloud to cloud traffic. On cost sharing, We need to approach this from different angles: User institution: Connects to the NREN NREN: Ensures great connectivity to other NRENs and OXPs Cloud provider: Connect to the nearest NREN, or if that is impossible or you want to peer with a lot of NRENs and commercial ISPs, connect to an OXPs

Clouds are already here and being used. Conclusion Clouds are already here and being used. NRENs can support the scientists in using clouds. The approach suggested here fits nicely with the way NORDUnet things of Global networking. Mention a few pts from yesterday GTS: virtualization, orchestration, in a multidomain environment. A homogeneous AAI is paramount! Identity Federations, interconnected through eduGAIN. Work in progress, contribute on topics like attributes.