The Benefit and Need of Standard Contribution for IXPs Jan Stumpf System Engineer

Agenda Definition IXP DE-CIX Facts and Details Need and Benefit of Standard Contribution Make Route Server Aware of Data Link Failure Commonly Agreed BGP Community for Blackholing 2

Definition IXP A physical network facility operated by a separate legal entity Interconnection of more than two independent Autonomous Systems (AS) Interconnection of ASes only Primarily facilitating the exchange of Internet traffic Distinct from an Internet access network or a transit network/carrier 3

DE-CIX Facts Operates Internet exchanges (IXs or IXPs) in –Frankfurt –Hamburg –Munich –New York –Dubai –more to come … Provides services such as peering: the settlement- free exchange of Internet traffic Connects almost 700 networks worldwide Strictly carrier- and data center-neutral 4

DE-CIX Frankfurt Founded in 1995 (Arnold Nipper co-founder) World‘s largest Internet exchange (4.0 Tbps peak, 2.3 Tbps average) Serves and connects 600+ networks Keeps 65,000+ active peering sessions Has 1GE, 10GE and 100GE ports connected Total capacity of 12Tbps Available in 18 data center facilities troughout the city of Frankfurt 5

Traffic Growth DE-CIX Frankfurt 6

eco Association Owner of DE-CIX 750+ members (such as AT&T, Brocade, Cisco, CloudFlare, Telekom, …) Representing its members’ interests in politics and in international bodies Offers legal support 7

Need of Standard Contribution DE-CIX is special in size –#customers, traffic, #router in IXP LAN IXP business is a niche but especially important Standard = Compatibility with many vendors Protocols not optimized for IXP use case 8

Benefit of Standard Contribution Selected examples: –Making Route Servers aware of data link failures –Commonly agreed BGP community for blackholing 9

Make Route Server Aware of Data Link Failure 10

Typical Scenario: BGP Session 11 Peer BPeer A BGP Data The control plane is able to detect the data plane failure.

Challenge: Route Server at IXPs 12 IXP Peer B /8, IP B Peer A /8, IP A Route Server BGP Data Problem: The control plane is not able to detect data plane failure any more. Data traffic is lost!

Solution 1.Client routers must have a means of verifying connectivity amongst themselves 2.Client routers must have a means of communicating the knowledge so gained back to the route server 13  Bidirectional Forwarding Detection, RFC 5880  North-Bound Distribution of Link-State and TE Information using BGP, Draft

Solution 14 IXP Peer B /8, IP B Peer A /8, IP A Route Server BGP /8  IP B NHIB: Nodes: B 1. Route Server: Next Hop Information Base (NHIB) updated 2. Client Router: Verify connectivity BFD connections are setup automatically BFD 3. Client Router: NHIB updated BGP /8 NHIB: Nodes: B Links: A->B 4. Route Server: Route selection All routes with next hop declared unreachable are excluded

Solution Bidirectional Forwarding Detection (BFD): –Hello packets are exchanged between two client routers (comparable to BGP Hello) –Rate: 1 packet / second, detection after 3 missing packets North-Bound Distribution of Link-State and TE Information using BGP (BGP-LS): –Model IXP network –Per peer: Next-Hop Information Base (NHIB) stores reachability for all next-hops 15

Data Link Failure 16 IXP Peer B /8, IP B Peer A /8, IP A Route Server 1. Client Router: Data link fail detected BFD 2. Client Router: NHIB updated BGP /8 NHIB: Nodes: B Links: 3. Route Server: Route selection All routes with next hop declared unreachable are excluded BGP NHIB: Nodes: B

Commonly Agreed BGP Community for Blackholing 17

The Problem: Massive DDoS Attack 18 IXP DDoS IXP Port Congestion If an IXP customer is hit by a massive DDoS attack its port can get congested and impact legitimate traffic

A Solution: Blackholing #19 Blackhole server: answer ARP requests Blackhole IP = Blackhole MAC IXP DDoS ACL Preparation IXP: 1.ACL: Block Blackhole MAC 2.Blackhole server for ARP For the IP prefix for which a blackholing is triggered all traffic is discarded at the IXP. Traffic for other IP prefixes gets through without any congestion. 19 BGP: Announce IP prefix under attack: Next Hop = Blackhole IP

Customer: How to Trigger Blackholing The customer announces the IP prefix under attack with the next hop IP address set to the blackholing IP address Blackholing works with bi-lateral and multi-lateral (route server) peerings Limited acceptance of /32 IP prefixes. < /24 is preferred. Route server: policy control to whitelist/blacklist a particular ASN can be used 20

Number of Prefixes Blackholed

Well-Known BGP Community for Blackholing Currently, many IXPs provide the blackholing feature Triggering is implemented differently at various IXPs (e.g. BGP community, next hop IP address (DE-CIX) ) A commonly agreed trigger is preferred: Well-known BGP community for blackholing All IXPs offering the blackholing feature voted on a tech mailing list for: 65535:666 – is a reserved ASN – 65535:666 = 0xFFFF029A is in the well-known BGP community space but unused – 666 is often used to trigger blackholing on transit networks An Internet Draft is currently coined – support is highly appreciated #22 BGP: Announce Prefix with Next Hop = Black-Hole IP Tag: 65535:666 22

Conclusion Two examples showed need for Standard Contribution –BFD Standardization for making it possible for Hardware vendors to implement the feature –Commonly Agreed BGP Community for Blackholing Standardization for easy triggering of the feature Higher goal: for the good of the Internet 23

Questions, Comments, Feedback? 24

DE-CIX Management GmbH Lindleystr Frankfurt Germany Phone