DDoS Mitigation Using BGP Flowspec We live in a connected world and the foundation for these connections is the network. Broadband Internet traffic is doubling each and every year (according to IDC) [or] Internet traffic worldwide will grow three-fold by the year 2017. (Internet Trends, Mary Meeker (KCPB) Today we have 2.5 billion Internet users in the world – roughly one-third of the Earth’s population. In the next decade, the number of Internet users will double to 5 billion (Mary Meeker, KPCB) That means that two-thirds of the world will be connected by 2023. When you add in the big trends of cloud, mobility, video and security, the combined rate of acceleration is placing unprecedented demands on the network. [Optional stats/factoids] 100 hours of video uploaded every single minute to YouTube (YouTube)   Mobile video traffic exceeded 50 percent for the first time in 2012. (Cisco VNI) Mobile network connection speeds more than doubled in 2012. (Cisco VNI) In 2012, a fourth-generation (4G) connection generated 19 times more traffic on average than a non-4G connection. Although 4G connections represent only 0.9 percent of mobile connections today, they already account for 14 percent of mobile data traffic. (Cisco VNI) [NOTE: Consider finding alternate source for above stats to avoid siting Cisco] As you just described (refer to pain points from previous slide), you are living in this world and feeling the pressure every day. Pradeep Sindhu founded Juniper 17 years ago on the belief that we should solve technology problems that matter most to our customers and that make a difference in the world. He recognized the importance of the network and the impact it would have on our world. Our mission is simple, but powerful; to connect everything and empower everyone. In today’s connected world, this mission is more relevant than ever. Here at Juniper we are focused on helping alleviate those pain points through our portfolio of high performance networking products. [T] And we do this by listening to our customers and helping them address their challenges and capitalize on their opportunities. DDoS Mitigation Using BGP Flowspec Justin Ryburn Senior System Engineer jryburn@juniper.net

Agenda Problem Statement Legacy DDoS Mitigation BGP Flowspec Overview Use Case Examples State of the Union

Problem Statement

Is DDoS Really an Issue? “…taking down a site or preventing transactions is only the tip of the iceberg. A DDoS attack can lead to reputational losses or legal claims over undelivered services.” Kaspersky Lab [1] Quick snapshot of Juniper today: Our Innovation: Innovation is in our DNA. We invest 17-19% of our net revenue in R&D- more than any other company in the industry. We have more than 1700 patents (working with the legal team to phrase this is a more powerful way i.e. # of patents per distinguished engineer or some other metric). The result? Powering XX% of the world’s video traffic and XX% of mobile traffic runs across our routers and switches. [Stats being updated/sourced by Barbara and team] Our flagship platforms are typically 1.2 to 1.9x more energy efficient than competitive designs. (Daniel Kharitonov)   Our own global corporate network includes over 1,400 Juniper Networks products. (Bask Iyer). Our Talent: more than 9,513 employees globally in 123 offices and 47 countries; we partner with more than 12,000 channel partners globally; 16 around-the-clock technical support centers; ~45% of our employees are dedicated to R&D with almost 80% of those software engineers; and we have received recognition as one of the world’s most ethical companies for the past 3 years. Our Revenue: Financially stable with $4.4B in annual revenue in 2012; geographically split 53% in America, 30% in EMEA and 17% in APAC; market cap of ~$9.5B; strong balance sheet with $3.8B in cash and investments; we fuel innovation through our Junos innovation fund which allows us to invest in early- and growth-stage software companies that expand the Junos ecosystem. We also have a strategic investment program that allows us to invest in hardware and components that are key to growing our business. Finally, we have been rated “investment grade” by both Moody’s and S&P credit agencies. [Transition] Now let me take just a moment to conclude with a story about the power of the “and”… Verisign [2] “Attacks in the 10 Gbps and above category grew by 38% from Q2 … Q3.” NBC News [3] “…more than 40 percent estimated DDoS losses at more than $1 million per day.” Tech Times [4] “DDoS attack cripples Sony PSN while Microsoft deals with Xbox Live woes”

Legacy DDoS Mitigation

Blocking DDoS in the “Old” Days “HELP” I’m being attacked. Service Provider Enterprise or DC 203.0.113.0/24 203.0.113.0/24 Internet 203.0.113.0/24 203.0.113.0/24 203.0.113.0/24 x 203.0.113.1 SP NOC NOC might connect to each router and add filter Ease of implementation and uses well understood constructs Requires high degree of co-ordination between customer and provider Cumbersome to scale in a large network perimeter Mis-configuration possible and expensive

Destination Remotely Triggered Black Hole (D/RTBH) BGP Prefix with next-hop set to discard route. Victim initiates RTBH announcement Service Provider Enterprise or DC 203.0.113.1/32 Internet 203.0.113.1/32 203.0.113.1/32 x 203.0.113.1 RFC 3882 circa 2004 Requires pre-configuration of discard route on all edge routers Victim’s destination address is completely unreachable but attack (and collateral damage) is stopped.

Source Remotely Triggered Black Hole (S/RTBH) BGP prefix with next-hop pointed at discard and uRPF enabled. “HELP” I’m being attacked. Service Provider Enterprise or DC Internet x 203.0.113.1 SP NOC NOC configures S/RTBH on route server RFC 5635 circa 2009 Requires pre-configuration of discard route and uRPF on all edge routers Victim’s destination address is still useable Only works for single (or small number) source.

BGP FlowSpec Overview

BGP Flow Specification Specific information about a flow can now be distributed using a BGP NLRI defined in RFC 5575 [5] circa 2009 AFI/SAFI = 1/133: Unicast Traffic Filtering Applications AFI/SAFI = 1/134: VPN Traffic Filtering Applications Flow routes are automatically validated against unicast routing information or via routing policy framework. Must belong to the longest match unicast prefix. Once validated, firewall filter is created based on match and action criteria.

BGP Flow Specification BGP Flowspec can include the following information: Type 1 - Destination Prefix Type 2 - Source Prefix Type 3 - IP Protocol Type 4 – Source or Destination Port Type 5 – Destination Port Type 6 - Source Port Type 7 – ICMP Type Type 8 – ICMP Code Type 9 - TCP flags Type 10 - Packet length Type 11 – DSCP Type 12 - Fragment Encoding

BGP Flow Specification Actions are defined using BGP Extended Communities: 0x8006 – traffic-rate (set to 0 to drop all traffic) 0x8007 – traffic-action (sampling) 0x8008 – redirect to VRF (route target) 0x8009 – traffic-marking (DSCP value)

Vendor Support DDoS Detection Vendors: Router Vendors: Arbor Peakflow SP 3.5 Accumuli DDoS Secure Router Vendors: Alcatel-Lucent SR OS 9.0R1 Juniper JUNOS 7.3 Cisco 5.2.0 for ASR and CRS [6] OpenSource BGP Software: ExaBGP

What Makes BGP Flowspec Better? Same granularity as ACLs Based on n-tuple matching Same automation as RTBH Much easier to propagate filters to all edge routers in large networks Leverages BGP best practices and policy controls Same filtering and best practices used for RTBH can be applied to BGP Flowspec

Caveats Forwarding Plane resources Not a replacement technology Creating dynamic firewall filters that use these resources More complex FS routes/filters will use more resources Need to test your vendors limits and what happens when it is hit Usually ways to limit the number and complexity of filters to avoid issues Not a replacement technology Should be ADDED to existing mitigation methods and not replace them When it goes wrong (bugs) it goes wrong fast Cloudflare outage: https://blog.cloudflare.com/todays-outage-post-mortem- 82515/

Use Case Examples

Inter-domain DDoS Mitigation Using Flowspec BGP Prefix installed with action set to rate 0. Victim initiates Flowspec announcement for 53/UDP only Service Provider Enterprise or DC Internet 203.0.113.1/32,*,17,53 x 203.0.113.1 Allows ISP customer to initiate the filter. Requires sane filtering at customer edge.

Edge Router Configuration Alcatel-Lucent Cisco [7] Juniper router autonomous-system 64496 bgp group "CUST-FLOWSPEC" neighbor 192.0.2.1 family ipv4 flow-ipv4 peer-as 64511 no flowspec-validate exit no shutdown Exit router bgp 64496 ! Initializes the global address family address-family ipv4 flowspec ! remote-as 64511 ! Ties it to a neighbor configuration protocols { bgp { group CUST-FLOWSPEC { peer-as 64511; neighbor 192.0.2.1 { family inet { flow; } routing-options { flow { term-order standard;

Intra-domain DDoS Mitigation Using Flowspec BGP Prefix installed with action set to rate 0. “HELP” I’m being attacked. Service Provider Enterprise or DC Internet x 203.0.113.1 SP NOC NOC configures Flowpec route on route server Could be initiated by phone call, detection in SP network, or a web portal for the customer. Requires co-ordination between customer and provider.

Edge Router Configuration Alcatel-Lucent Cisco [7] Juniper router autonomous-system 64496 bgp group "RR-CLIENT-FLOWSPEC" neighbor 198.51.100.1 family ipv4 flow-ipv4 peer-as 64496 exit no shutdown router bgp 64496 ! Initializes the global address family address-family ipv4 flowspec ! remote-as 64496 ! Ties it to a neighbor configuration protocols { bgp { group RR-CLIENT-FLOWSPEC { type internal; neighbor 198.51.100.1 { family inet { flow; } routing-options { flow { term-order standard;

Route Server Configuration Alcatel-Lucent Cisco [7] Juniper router autonomous-system 64496 bgp group "RR-CLIENT-FLOWSPEC" neighbor 198.51.100.2 family ipv4 flow-ipv4 peer-as 64496 exit no shutdown router bgp 64496 ! Initializes the global address family address-family ipv4 flowspec ! remote-as 64496 ! Ties it to a neighbor configuration protocols { bgp { group RR-CLIENT-FLOWSPEC { type internal; neighbor 198.51.100.2 { family inet { flow; } export FLOWROUTES_OUT;

Route Server Configuration Cisco [7] Juniper class-map type traffic match-all attack_fs match destination-address ipv4 203.0.113.1/32 match protocol 17 match destination-port 53 end-class-map ! policy-map type pbr attack_pbr class type traffic attack_fs drop class class-default end-policy-map flowspec address-family ipv4 service-policy type pbr attack_pbr exit routing-options { flow { term-order standard; route attack_fs { match { destination 203.0.113.1/32 protocol udp; destination-port 53; } then discard; policy-options { policy-statement FLOWROUTES_OUT { from { rib inetflow.0; then accept;

DDoS Mitigation Using Scrubbing Center Attack traffic is scrubbed by DPI appliance. BGP Prefix installed with action set to redirect. Legitimate traffic sent to customer via GRE or VRF tunnel. Scrubbing Center Service Provider “HELP” I’m being attacked. Enterprise or DC Internet x 203.0.113.1 SP NOC NOC configures Flowpec route on route server Could be initiated by phone call, detection in SP network, or a web portal for the customer. Allows for mitigating application layer attacks without completing the attack.

Edge Router Configuration Alcatel-Lucent Cisco [7] Juniper router autonomous-system 64496 bgp group "RR-CLIENT-FLOWSPEC" neighbor 198.51.100.1 family ipv4 flow-ipv4 peer-as 64496 exit no shutdown router bgp 64496 ! Initializes the global address family address-family ipv4 flowspec ! remote-as 64496 ! Ties it to a neighbor configuration protocols { bgp { group RR-CLIENT-FLOWSPEC { type internal; neighbor 198.51.100.1 { family inet { flow; } routing-options { flow { term-order standard;

Route Server Configuration Alcatel-Lucent Cisco [7] Juniper router autonomous-system 64496 bgp group "RR-CLIENT-FLOWSPEC" neighbor 198.51.100.2 family ipv4 flow-ipv4 peer-as 64496 exit no shutdown router bgp 64496 ! Initializes the global address family address-family ipv4 flowspec ! remote-as 64496 ! Ties it to a neighbor configuration protocols { bgp { group RR-CLIENT-FLOWSPEC { type internal; neighbor 198.51.100.2 { family inet { flow; } export FLOWROUTES_OUT;

Route Server Configuration Cisco [7] Juniper class-map type traffic match-all attack_fs match destination-address ipv4 203.0.113.1/32 match protocol 17 match destination-port 53 end-class-map ! policy-map type pbr attack_pbr class type traffic attack_fs redirect nexthop 192.0.2.7 class class-default end-policy-map flowspec address-family ipv4 service-policy type pbr attack_pbr exit routing-options { flow { term-order standard; route attack_fs { match { destination 203.0.113.1/32 protocol udp; destination-port 53; } then discard; policy-options { policy-statement FLOWROUTES_OUT { from { rib inetflow.0; then { next-hop 192.0.2.7; accept;

How Do I Know It Is Working? Alcatel-Lucent Cisco [7] Juniper show router bgp routes flow-ipv4 show router bgp routes flow-ipv6 show filter ip fSpec-0 show filter ip fSpec-0 associations show filter ip fSpec-0 counters show filter ip fSpec-0 entry <entry-id> show processes flowspec_mgr location all show flowspec summary show flowspec vrf all show bgp ipv4 flowspec show bgp neighbor <neighbor> | match inet-flow show route table inetflow.0 extensive show firewall filter __flowspec_default_inet__

Real World Example "Where I think FlowSpec excels, is for protection of our mobile platform. 2 /24s are shared among a million mobile devices with NAT in a firewall. The link capacity (and in part the firewall itself) is overloaded by a simple DDoS attack against just one of these adresses. The system detects a DoS attack against an address on the firewall. It will identify total traffic, UDP, fragments, TCP SYN, ICMP, whatever, and depending on what kind of attack it is, a policer is added for the specific protocol/attack on individual peering routers. Protocols are policed with individual policers, so that for instance UDP and TCP SYN can be policed to different throughputs. Basically, an attack against a single IP on UDP will not affect other customers being NAT'ed to the same address, using anything but UDP - and link capacity is protected."

Real World Example Attack on 1/13/16

Where Are We Going? IPv6 Support Relaxing Validation http://tools.ietf.org/html/draft-ietf-idr-flow-spec-v6-06 Relaxing Validation http://tools.ietf.org/html/draft-ietf-idr-bgp-flowspec-oid-02 Redirect to IP Action https://tools.ietf.org/html/draft-ietf-idr-flowspec-redirect-ip-02

State of the Union

Summary of Survey Great idea and would love to see it take off but… Enterprises and Content Providers are waiting for ISPs to accept their Flowspec routes. Some would even be willing to switch to an ISP that did this. ISPs are waiting for vendors to support it. More vendors supporting it Specific features they need for their environment Better scale or stability

References [1] Kaspersky Lab – Every Third Public Facing Company Encounters DDoS Attacks http://tinyurl.com/neu4zzr [2] Verisign – 2014 DDoS Attack Trends http://tinyurl.com/oujgx94 [3] NBC News – Internet Speeds are Rising Sharply, But So Are Hack Attacks http://tinyurl.com/q4u2b7m [4] Tech Times – DDoS Attack Cripples Sony PSN While Microsoft Deals with Xbox Live Woes http://tinyurl.com/kkdczjx [5] RFC 5575 - Dissemination of Flow Specification Rules http://www.ietf.org/rfc/rfc5575.txt [6] Cisco - Implementing BGP Flowspec http://tinyurl.com/mm5w7mo [7] Cisco – Understanding BGP Flowspec http://tinyurl.com/l4kwb3b

Thank You!