Multi-Domain Virtual Private Network service A seamless infrastructure for NRENs, GEANT and NORDUnet GN3+ project SA3T3 team : Xavier Jeannin (RENATER) - presenter, Tomasz Szewczyk (PSNC), Bojan Jakovljevic (AMRES), Thomas Schmid (DFN), Dave Wilson (HEAnet), Brian Bach Mortensen (NORDUnet) TNC 15, Porto, Portugal 15-18 June 2015

What is MD-VPN? The service provides a seamless, scalable transport infrastructure A joint service provided by the GÉANT network and NRENs A seamless transport infrastructure that provides a connectivity service: Layer3 or Layer2 VPNs spanning several domains point-to-point or multipoint Multi-domain networking IPv4 IPv6 L3VPN P2P L2VPN MD - VPN MP L2VPN

MD-VPN service highly scalable, seamless transport infrastructure Configure only at the edge NREN OPEX Reduced VPN Provisioning as easy as in a single-domain Easy to deploy No CAPEX VPN multiplexed Lead-time reduced An end-to-end extensible and flexible service Configure only at the edge

A double benefit for NRENs with regional network

An innovative design with added value for end-users An original connectivity network service Multi-domain networking Facilitate an foster distributed collaboration in Europe Cover a wide scope of use cases Reduce OPEX and CAPEX for use Cost saving – VPN cheaper Cost saving – No tender for research project Safe infrastructure Security opex saved on site Reduce firewall usage

MD-VPN use cases A wide scope for MD-VPN use All scientific projects based on international collaboration LHCONE is an example of successful L3VPN multi-domain service ITER, CONFINE Distributed infrastructure Cloud provider Grid – HPC center Scientific infrastructure: Telescope, sensor network

MD-VPN use cases A wide scope for MD-VPN use Quick P2P connection Conference demonstration P2P data transport between to sites http://cuc.carnet.hr/2014?news_hk=5605&news_id=285&mshow=1105#mod _news No firewall – smaller delay – better TCP throughput Education Remote lecture E-learning

MD-VPN use cases A wide scope for MD-VPN use MD-VPN transparent data transport layer for high level network services like SDN, … and in general by future internet project No firewall – smaller delay – better TCP throughput

How does it work? Underlying principle behind this Multi-Domain VPN technology The LSP is extended from a PE up to the remote PE in another domain Signaling is split in 2 parts Signalling for multi-domain MPLS path between PE routers thanks to a BGP peering with labelled unicast SAFI (internal route) Signalling for VPN labels and prefixes exchange between PE routers (external route) thanks to an external BGP VPNv4 family peering GEANT and NORDUnet implement Carrier of Carriers (CoC) providing transparent transport of VPN traffic

MDVPN: BGP-signalling L2VPN, L3VPN Multi-hop eBGP VPNv4, VPNv6, L2VPN iBGP BGP-signaled L2VPN and L3VPN label and prefix exchange eBGP labeled-unicast iBGP Multi-domain PE to PE MPLS path

MDVPN: tLDP-signalling L2 circuit Targeted LDP -signaled L2 circuit label exchange eBGP labeled-unicast iBGP Multi-domain PE to PE MPLS path

MDVPN data plane label operations incoming packet push VPN label push transport label push LDP label outgoing packet pop VPN label swap transport label pop transport label pop LDP label swap transport label swap transport label swap transport label push CoC label pop CoC label swap CoC label MDVPN packets labels: With the courtesy of Jani Myyry (Funet) LDP label Transport label VPN label Data CoC label Transport VPN Data

Global view of the service Geographical extensibility Service extensibility

CPE-NREN-B-VPN-ASTRO GEANT CPE-NREN-A-VPN-ASTRO Peering BGP VPNv4 CPE-NREN-B-VPN-ASTRO PE-RENATER ASBR-1-GEANT ASBR-NREN-A ASBR-2-GEANT ASBR-NREN-B PE-NREN-B RR-NREN-B RR- NREN-A NREN-A NREN B Peering Multi-hop E-BGP VPNv4 (No next-hop self) Physical connections Peering labeled-unicast VRF ASTRO RT:22:30 VRF BIO - RT:22:32 VRF md-vpn1 - RT:33:10 VRF md-vpn2 - RT:13092:17 L2Circuit toward AMRES L2Circuit PE-RENATER - PE-REMOTE-NREN Standard Deployment VPN-Route-Reflector VRF CoC

CPE-NREN-B-VPN-ASTRO GEANT CPE-NREN-A-VPN-ASTRO Peering BGP VPNv4 CPE-NREN-B-VPN-ASTRO PE-RENATER ASBR-1-GEANT ASBR-NREN-A ASBR-2-GEANT ASBR-NREN-B PE-NREN-B RR- NREN-A NREN-A NREN B Peering Multi-hop E-BGP VPNv4 (No next-hop self) Physical connections Peering labeled-unicast VRF ASTRO RT:22:30 VRF BIO - RT:22:32 VRF CoC VRF md-vpn1 - RT:33:10 VRF md-vpn2 - RT:13092:17 L2Circuit toward AMRES L2Circuit PE-RENATER - PE-REMOTE-NREN Limited deployment VPN-Route-Reflector MPLS is enabled only on the AS Border Router VPN is propagated internally by any other internal means: VLAN, dedicated link, other solutions …

Where can you use MD-VPN? MD-VPN service in the GÉANT portfolio 18 NRENs connected (+ 1 NREN using MD-VPN Proxy + 1 NREN still working on) Roughly 400 PoPs available that European scientist can already use MD-VPN

Reliability demonstrated since August 2014 Pilot phase :Service reliability checking during 3 months Statistics available at https://tools.geant.net/portal/links/mdvpn/ms_avail_sum m.jsp

A monitored service https://tools.geant.net/portal/links/mdvpn/ms_st atus_dashboard.jsp Portal available at:

Security MD-VPN provides the same level of security as VPN MPLS service, There was no security concern related to users or even to MD-VPN users But it is impossible to protect the access to VPNs if the core is compromised The only threats that can occur a NREN attacking another NREN a NREN router compromised by a pirate Very difficult and slow attack (never seen so far) A “detector” of this type of attack was demonstrated and will be deployed at the end of this year MPLS firewall under test with DELL company support

A scientist project using MD-VPN for production 16 sites connected in 12 countries https://www.fi-xifi.eu/federation.html Using all types of connection: Direct connection Via VPN-Proxy Private companies not connected to any NREN http://infographic.lab.fi-ware.org/status A first scientist project FIWARE FIWARE is a project of the European Public-Private-Partnership on Future Internet (FI-PPP) programme

MD-VPN offers a new way of cooperating MD-VPN enables a new way for GÉANT and NRENs to cooperate, which significantly increases network scalability from a service point of view Operation Level Agreement VPN provisioning, debugging, … Acceptable User Policy Manage service extension (regional, metropolitan network)

MD-VPN and GEANT Porfolio MD-VPN service positioning regarding GEANT Plus service and L3VPN service MD-VPN usage should be encouraged when it can be used

User Network Interface Prospectives MD-VPN service improvement Scripting for VPN provisioning – VPN automation delivery Improve lead time and NOC quality work Optical transport MD-VPN Innovation Users User Network Interface With MD-VPN, we create a seamless transmission infrastructure using MPLS as data plane and all services compatible with MPLS could use this infrastructure NG-mVPN, EVPN New experimental service can use this infrastructure SDN, CoCo project

Summary An innovative and highly scalable design Seamless transport infrastructure A bundle of services (IPv4, IPv6, P2P L2VPN, VPLS, L3VPN) with added value for our users that was added to GÉANT portfolio An original and useful service unavailable in a commercial NSP portfolio A FI-PPP project, FIWARE, use GÉANT’s MD-VPN to provide its network infrastructure Broad European deployment 18 connected NRENs Roughly 400 PoPs already available

The dream team Tomasz Szewczyk (PSNC) Thomas Schmid (DFN) Magnus Bergroth (NORDUnet) Daniel Lete (HEAnet) Carlos Friacas (FCCN) Jani Myyry (Funet) Bojan Jakovljevic (AMRES) Miguel Angel Sotos (RedIRIS) Niall Donaghy (DANTE) Xavier Jeannin (RENATER) With the support of Brian Bach Mortensen (DiEC) A small team, very small amount of manpower … but highly motivated and skilled

Xavier.jeannin “at” renater.fr Any Questions? Xavier.jeannin “at” renater.fr

How to connect “non MD-VPN site”? VPN-proxy

VPN-Proxy implementation Implemented thanks to logical router available in Juniper ASBR-GEANT logical router GEANT NREN not MPLS-aware BGP-LU peering VPN-Route-Reflector I-BGP VPNv4 Back-to-back connection, VRF BIO, VRF ASTRO, … VPN-Proxy Play the role of ASBR + PE + route exchange VRR

Scalability Scalability MD-VPN is designed to provide thousands (and more) of service Thanks to separation between data transport in the core network and services provided at the edge. In the core network only labels and routes related to PE routers are maintained (1000 routes) The services are maintained at the network edge, on PE routers. Each PE router maintains only the set of entries (labels or routes) related to services provided by this very PE router. The number of VPNs that are active between NRENs has zero impact on the GEANT and NORDUnet infrastructure since they are completely transparent to the GEANT network.

Security MD-VPN provides the same level of security as VPN MPLS service, There was no security concern related to users or even to MD-VPN users But it is impossible to protect the access to VPNs if the core is compromised In case of MD-VPN, the core is multi-domain The only threats that can occur a NREN attacking another NREN a NREN router compromised by a pirate

Data plan attack from another NREN Label spoofing we could imagine that a pirate, first take control of a NREN router and secondly forge packet and inject into the NREN with the purpose to compromise a user VPN that is not located in remote NREN in the MD-VPN infrastructure. This case is very rare and our investigation demonstrates that such attack is very difficult to put in place and very long to lead. Counter measure label spoofing attack requires a specific step called “label scan” and this step is easily detectable thanks to NetFlow. Sudden increase of number of VPN labels is easy to detect. In this condition, the scan detection feature is capable to be our firewall

Label scan detection thanks to NetFlow With the courtesy of Julius Kriukas (LITnet) 2015/03/25 10:21:39 ALARM 193.51.178.10:29770 (#49), interface 104, label {16459 0}, threshold reached, 409 unique labels, 13 labels is allowed 2015/03/25 10:21:39 ALARM 193.51.178.10:2024 (#17), interface 104, label {16459 0}, threshold reached, 416 unique labels, 13 labels is allowed

NetFLow detection deployment