Low-Rate TCP-Targeted DoS Attack Disrupts Internet Routing Ying Zhang Z. Morley Mao Jia Wang Good morning. today I will tell you about a DoS attack we discovered targeting the Internet routing infrastructure. This attack is stealthy and it exploits known vulnerabilities of the TCP protocol. My name is Ying Zhang, I am a Ph.D. student from U of M. This is the joint work with my advisor Professor Mao, and Dr. Jia Wang from AT&T Labs Research.

Attacks on the Internet Attacks targeting end hosts Denial of Service attacks, worms, spam Attacks targeting the routing infrastructure Compromised routers Stealthy denial of service attacks Internet C BR C BR Today various attacks exist on the Internet. So far, we have mostly focused on attacks targeting the end hosts, such as denial of service attacks, worms, spam. Another arguably more important type of attack is targeting the critical infrastructure such as the internet routing system. Such attacks have severe impact, as traffic can be disrupted and even blackholed. How can the routing system be attacked? One possibility is by breaking into routers, as routers can become compromised just like end-hosts due to software vulnerabilities. routing protocols may have also be security holes. The focus of this talk however is on a new type of attack, which is stealthy and does not require direct access to the router, nor the ability to send traffic destined to the router. This new type of denial of service targeting the routing system only requires the ability to identify the target link and then choose traffic flows intelligently to traverse the target link. Such attack traffic flows can come from any source and go to any destination, as long as they go through the target link. To be more precise, this attack targets the routing protocol running on the target link and it is mainly exploiting the problem that by default routing traffic is not protected from congestion due to data traffic. Bots Target link C BR Attackers Target Destination

Border Gateway Protocol Border Gateway Protocol De facto standard inter-domain routing protocol BGP session reset Keepalive Keepalive confirm peer liveliness; determine peer reachability BGP HoldTimer expired BGP session Let me now describe more details about the interdomain routing protocol of the current Internet – BGP which standards for the Border Gateway Protocol. Two routers involved in a BGP session to disseminate interdomain routing information establish a long-lived TCP connection. To ensure liveness of the neighbor in a BGP session, routers periodically exchange keepalive messages. If several consecutive keepalive messages are not received (possibly due to congestion or router problems) from a neighbor, the other router in the BGP session will send out a “BGP HoldTimer expired” message and then close the session. This is called BGP session reset. This maximum waiting time period is defined by the “Hold Timer” value, which is configurable. When BGP session is reset, all the routes previously exchanged are withdrawn, this effect can be propagated globally to the rest of the Internet, causing routing instability. Thus, frequent BGP session reset and session reset involving core Internet routers carrying a large number of routes is devastating to the stability of the Internet! Given that BGP uses TCP as its transport protocol, it is natural that BGP may be affected TCP’s vulnerability. Moreover, because in current Internet, routing traffic is not always protected from data traffic congestion, attacks against TCP can also disrupt BGP. Therefore, we examine a recently discovered attack against TCP. C BR AS 1 C BR AS 2 Transport: TCP connection

Low-rate TCP-targeted DoS attacks [Kuzmanovic03] Exploiting TCP’s deterministic retransmission behavior No packet loss ACKs received packet loss No ACK received TCP Congestion Window Size (packets) First, let me briefly review this recently identified attack against TCP called the low-rate TCP-targeted DoS attack which appeared in Sigcomm 2003. This attack is exploiting TCP’s deterministic retransmission timeout behavior. TCP congestion control restricts the sending rate. It specifies the growth behavior of the congestion window which is the number of outstanding unacknowledged packets. After connection establishment, under no packet loss, the congestion window size increases exponentially during slow start until reaching some threshold. If there is sufficient packet loss triggering retransmission timeout specified by the minRTO value, the congestion window is reset to be 1 MSS, entering into slow start again. minRTO is by default 1 second in most of the Unix system and increases exponentially. (what about windows????) Initial window size minRTO 2 x minRTO 4 x minRTO Time

Low-rate TCP-targeted DoS attacks Attack flow period approximates minRTO of TCP flows TCP congestion window size (segments) TCP retransmission timeout value is doubled upon each successive timeout occurrence. The attacker exploits this mechanism in the following way. First, it tries to introduce congestion to cause sufficient packet drop to trigger retransmission timeout. Then it introduces another congestion after minRTO to conincide with sender’s retransmission, creating the second burst to drop the retransmitted packets. In other words, the attack flow synchronizes with the retransmission flow and cause all the consecutive retransmitted packets to be dropped. Initial window size minRTO 2 x minRTO 4 x minRTO Time

Impact of low-rate TCP DoS attacks Impact on any TCP connections TCP continuously experiences loss TCP obtains near zero throughput Difficult to detect due to low-rate property Our finding: Low-rate TCP DoS attacks can disrupt BGP (with default configurations) This type of attack causes TCP to continuously experience packet loss, stay in the timeout state, and results in near zero throughput. The attack flows have a low average sending rate and this makes detection very challenging. In our work, we find that this low rate attack can disrupt BGP routing for routers with default configuration settings.

Impact of routing disruption Reduced sending rate Increasing convergence delay BGP session reset Routing instability Unreachable destinations Traffic performance degradation The effect of this attack on BGP has two aspects. First, it’s easy to see that attack traffic lowers the sending rate of the TCP connection carrying BGP traffic. This will result in increased convergence delays. Second, the more severe effect on the BGP session is the possibility of BGP session reset caused by all packets dropped within a time interval exceeding the hold timer value. Session reset can cause global routing instability, make some destinations unreachable, and introduce performance degradation due to transient effects. Given its severe consequence, our work studies in detail how likely BGP session reset can happen due to this attack against TCP.

Outline Description of a potential attack against Internet routing Attack demonstration using testbed experiments Increased attack sophistication Using multi-host coordination Defense solutions through prevention I’ve describe this potential attack targeting the Internet routing. In the remainder of the talk, I will first introduce our lab-controlled testbed experiments using commercial routers. Then I introduce a more sophisticated low-rate attack using multiple hosts. Finally we propose solutions to prevent the attack.

Testbed experiments Using high-end commercial routers Demonstrating the attack feasibility Receiver B Sender A Gigabit Ethernet Gigabit Ethernet Our testbed consists of seven types of routers from two vendors, Cisco and Juniper. The results presented in this talk are based on the high-end Cisco router GSR. It is widely used in the core of the Internet and is very powerful, thus expected to be more resistant to attacks than less powerful routers. Note that we use real commercial routers in our study because the TCP stack implementation on commercial routers is not publicly known so is thus difficult to simulate. The goal of the testbed experiments is to understand whether the attack can succeed on real commercial routers. The testbed is set up in such a way that the sender can easily overload the link carrying BGP traffic without any background traffic. This is not realistic in practice, but we relax the requirement in later experiments. Note also that in reality, attackers may not be directly connected to the routers. The attacker also does not need to control the receiver, as attack traffic can be sent be any destination as long as it traverses the target link. OC3 155Mbps C BR C BR Router R1 (Cisco GSR) Router R2 (Cisco GSR)

The attack to bring down a BGP session UDP-based attack flow Attacker A Packet is dropped due to congestion Receiver B BGP Keepalive message Let’s take a closer look at the attack. The BGP session is between two routers R1 and R2. The attacker A sends traffic going through the target link to any destination Receiver B. The UDP based attack flow creates the first burst to congest the target link and force the BGP Keepalive and update packets to be dropped. Router R2 C BR Router R1 C BR

The attack to bring down a BGP session UDP-based attack flow Attacker A Receiver B Retransmitted BGP Keepalive message Using the minRTO as the inter burst period, the attack flow creates the second burst and drop the first retransmitted BGP message. minRTO Router R2 C BR Router R1 C BR

The attack to bring down a BGP session UDP-based attack flow Attacker A 2nd Retransmitted BGP Keepalive message Receiver B Subsequently all retransmitted packets are dropped due to synchronized attack flow with the packet retransmission. minRTO 2*minRTO Router R2 C BR Router R1 C BR

The attack to bring down a BGP session UDP-based attack flow Attacker A 7th retransmitted BGP Keepalive message Receiver B In the default Cisco setting, after the 7th retransmitted packet is dropped, router R2’s BGP hold timer expires and causing the BGP session to reset. The total attack duration required is usually less than 3 minutes. The actual attack duration and required packet drop depends on value of the hold timer and the built-in TCP minRTO timer. minRTO BGP Session Reset 2*minRTO Router R2 C BR Router R1 C BR Hold Timer expired!

Basic attack flow properties Burst length L Magnitude of the peak R We’ve verified our initial hypothesis: the attacker can indeed bring down the BGP session using low-rate TCP targeted DoS attack. What are the factors that affect the attack’s effectiveness? There are three factors: Burst Length L needs to be long enough to cause congestion, Peak magnitude R also needs to be large to cause congestion; Inter-burst period T needs to be minRTO to cause session reset; We conducted experiments to analyze how each factor affect the attack’s effectiveness. Inter-burst period T

How likely is BGP session reset? R:185Mbps T: 600msec Min duration:216 sec 30% session reset probability with 42% capacity usage Let’s examine the relation between the burst length L and the probability of session reset as an example. Other factors are elaborated in the paper. X axis is the burst length. The right Y axis is the normalized average rate. The left Y axis is the probability of session reset. Session reset does not occur deterministically because the attack flow burst may not cause sufficient packet drop to trigger the initial retransmission timeout. We conduct the experiments 100 times for each burst length, with fixed inter-burst period 600 msec, matching the router minRTO, and the peak magnitude of 185 Mbps to congest the link. We limit the maximum attack duration to be 2 hours and we found that the minimum duration can be 216 seconds. As expected that the session reset probability and the average attack rate increase with the burst length value. We observe that with the burst length of 225 msec, the attacker has around 30% probability to reset the session with 42% available bandwidth.

Router implementation diversity Router Type Router OS Version minRTO (msec) Keepalive (sec) HoldTimer Cisco 3600 IOS 12.2(25a) 300 60 180 Cisco 7200 IOS 12.2(28)S3 600 Cisco 7300 IOS 12.3(3b) Cisco 12000 IOS 12.0(23)S Juniper M10 JUNOS[6.0R1.3] 1000 30 90 Although we focus on GSR, we also reproduced the attack on various routers with different IOS versions including both Cisco and Juniper. We show that this is not just a vendor specific problem, instead it is protocol specific. Since the TCP stack implementation on routers is not public, we develop a tool to infer parameters such as minRTO. We found that it is easier to attack Juniper M10 router because its smaller HoldTimer value, reducing the required attack duration, and its larger minRTO, allowing attack flow of lower average rate. Hilight the last two rows General problem.

Explanation of packet drops BGP packet drop locations: Ingress or egress line card buffer queues Resource sharing across interfaces Interfaces share buffers and processing time Router Interface 1 BGP pkt BGP pkt Egress line card We further analyze where packet drop occurs inside the router to help design mitigation solutions. Interestingly, packets can be dropped in both the router ingress and egress line card buffer queue. When BGP packet is locally generated, it can be dropped in the egress line card due to congestion. BGP packets can also be routed due to multihop BGP session, in such cases, it can encounter loss in the ingress line card. This means that protection for BGP traffic needs to be carried out at both ingress and egress line cards. We have another noteworthy observation related to router resource sharing. Each line card usually has multiple interfaces, and the line card buffer queue is in fact shared by all the interfaces. It means congestion on one interface will affect other interfaces. Thus traffic that does not share the same interface as the BGP traffic can also introduce congestion affecting BGP. Ingress line card Interface 2 Interface 3 Interface 4

Buffer allocation in line cards Line card memory is divided into buckets of different packet sizes Packets cannot utilize buckets of a different size Line card buffer queues Switch fabric Full! Packet size (0,80Byte] BGP pkt Drop! Another related observation is related to buffer allocation in line cards. The linecard buffer queue is usually divided into buckets of different packet sizes to ensure smaller packets are guaranteed certain buffer space. The allocation based on packet size is strictly enforced, meaning that packets are only allowed to placed to queue matching its size. For example, the typically small BGP packet can only be placed in the first queue, which may already be filled up by the small attack packets. So the BGP packet can be be dropped despite empty spaces in other queues. The implication here is that attack flows do not need to fill up the entire line card buffer, and only needs to saturate the queue shared by BGP packets. [81Byte,270Byte] Empty [271Byte, 502Byte] [503Byte, 908Byte] [909Byte,1500Byte]

Necessary conditions for session reset Inter-burst period approximates minRTO The attack flow’s path traverses at least one link of the BGP session Attack flow’s bottleneck link is the target link Receiver Attack flow’s path C BR C BR Attacker Now, after going through the details of this attack, let me summarize the necessary conditions for the attack to succeed in resetting the BGP session using a single attack flow. First, the inter-burst period of the attack flow needs to be the same as the minRTO in the router to repeatedly introduce packet loss. Second, the attack flow needs to traverse at least one link of the BGP session. The third condition is that the attack flow’s bottleneck link is the target link carrying BGP traffic, otherwise, attack flow won’t have enough strength to cause congestion. The third condition is usually difficult to satisfy for end-hosts due to prevalent bottleneck links at the edge networks. However, analogous to DDos attack, attackers may exploit multiple hosts to launch such an attack. Next, let’s examine the possibility of a coordinated low-rate attack. Bottleneck link C BR C BR C BR C BR Router R2 Router R1 Multi-hop BGP Session

Outline Description of a potential attack against Internet routing Attack demonstration using testbed experiments Increased attack sophistication Using multi-host coordination Defense solutions through prevention

Coordinated low-rate DoS attacks Attack host A Destination C Router R1 C BR Router R2 C BR Target BGP session Assume the attacker has control over host A and B, the target link is between R1 and R2, each attacker transmits a square-wave of attack flow to destinations C, and D respectively. Attack host and destination is not directly connected to the router. When the individual attack flow arrives at the target link, they can be aggregated to be a square-wave flow with larger strength. Destination D Attack host B

Coordinated low-rate DoS attacks Attack Host A Destination C Router R1 C BR Router R2 C BR Target BGP session In fact, such synchronization is not really necessary. Attack flows can be simply aggregated to be a near constant flow with enough strength to cause congestion. Destination D Attack Host B

Coordinated low-rate DoS attacks BR C BR C Target BGP session From a network-wide view, it is challenging to detect such an attack due to its distributed nature. In particular, these flows originate from different sources, going to apparently unrelated destinations. Even monitoring at the target link cannot easily reveal that such traffic flows are correlated. Attackers today can accomplish this using a botnet, which can easily have up to several hundreds of hosts.

Host selection for coordinated attacks Selecting attack host-destination pairs to traverse target link Identify the target link’s geographic location and ASes Identify prefixes with AS-level path through the target link Identify IP-level paths Now let’s examine how feasible it is for attackers to identify hosts to launch such kind of distributed attack targeting a network link. Many Internet mapping projects such as rocketfuel demonstrate that it is not difficult to discover router-level network topology. Given the network topology, we show that it is actually quite feasible to select attack source host and corresponding destinations. We sketch the process here. First attackers can identify the target link’s geographic location and its ASes. Second, they may select the prefixes whose route contains the target link at the AS level. Finally, probing can be used to confirm that traffic goes through the IP-level link.

Wide-area experiments Internet bottleneck link available bandwidth measurement 160 peering links 330 customer and provider links Attack host selection PlanetLab hosts as potential attack hosts Attack hosts geographically close to the target link Attacks targeting a local BGP session To understand the feasibility of such an attack on Internet, we conduct three sets of experiments. We measure the available bandwidth on 160 peering links and 330 customer provider links to understand how easy it is to congest such links very likely carrying BGP traffic. We use planetlab as potential attack hosts to show that it is possible to select a subset of these hosts to send traffic traversing the target links. And finally, we launch the attack to bring down a local BGP session set up using software routers.

Wide-area coordinated attacks against a local BGP session R=5Mbps L=300msec T=1s Average Rate = 1.5Mbps UW1 (US) 10Mbps 100Mbps Targeted UW2 WAN BGP session We set up a BGP session using software routers in our local network, we use Planetlab in different places to launch the attack. To avoid sending too much traffic from each node, we perform time synchronization designed to create synchronized low-rate attack flow patterns. And we were able to bring down the BGP session successfully. Software router 1 Software router 2 THU1(China) THU2

Conditions for a single attack flow Coordinated attacks 1. Inter-burst period approximates minRTO 1’. Sufficiently strong combined attack flows to cause congestion 2. The attack flow’s path traverses the BGP session 3. Attack flow’s bottleneck link is the target link 3’. Identify the target link location Let’s step back again to summarize the required conditions for this attack The three conditions we have discussed before for single attack flow has been changed given the possibility of coordinated mult-host attack. First, instead of requiring the knowledge of the minRTO, with coordinated attack, we only require the combined attack flow to be sufficiently strong to cause congestion. The second condition remains. The third condition is also relaxed. It only requires the attacker to be able to identify the target link location, and select sufficient number of attack hosts. Understanding these condition clearly is needed for devising defense solutions against the attack.

Outline Description of a potential attack against Internet routing Attack demonstration using testbed experiments Increased attack sophistication Using multi-host coordination Defense solutions through prevention Because of its stealthy nature, attack detection is inherently difficult. we focus on prevention, which is a better solution to deal with this attack.

Attack prevention: hiding information Randomize minRTO [Kuzmanovic03] minRTO is any value within range [a,b] Does not eliminate BGP session reset Hide network topology from end-hosts Disabling ICMP TTL Time Exceeded replies at routers The first approach is to hide the information to prevent the attack. To prevent single host based attack, we can randomize the minRTO to a certain range instead of using a fixed value. However, this is not sufficient to eliminate BGP session reset, especially under coordinated attacks. To prevent coordinated attacks, we need to hide the topology from the attacker to thwart the construction of attack flows required to traverse the target the link. Disabling ICMP reply is one option, as ICMP responses are needed to discover network topology. But it will make Internet measurement and troubleshooting more difficult. So, these solutions do not fundamentally address the problem.

Attack prevention: prioritize routing traffic Weighted Random Early Detection (WRED) Prevent TCP synchronization Selectively drop packets Drop low-priority packets first when the queue size exceeds defined thresholds Assumption of WRED The IP precedence field is not spoofed We need to police the IP precedence markings The fundamental weakness enabling the attack is the lack of protection of the routing traffic from data traffic, So, to fundamentally solve the problem, we need to provide guaranteed bandwidth for routing packets. WRED is one existing feature that can provide such functionality. It is a mechanism already available in today’s commercial router. It selectively drops the low-priority first before dropping any high priority packet. It helps provide resource isolation for BGP routing traffic. However, WRED relies on the IP precedence field in the packet header to differentiate high priority packets. This means that IP precedence should not be spoofed. Therefore, we need to police the IP precedence marking to prevent spoofing by attack traffic.

Support from existing commercial routers Router supported policing features Committed Access Rate (CAR) Class-based policing Traffic marking Reset the incoming packets to be low priority Class-based queuing Drop the packets with low priority when the traffic burst is high Effective in isolating BGP packets from attack traffic! We have studied two types of traffic marking mechanisms current router supports. The committed access rate and class-based policing. They are equivalent in functionality. Using these router supported features, we propose a defense solution consisting of edge routers policing the marking to prevent spoofing of high priority packets and all other routers carrying BGP traffic to enable WRED to provide bandwidth guarantee to routing packets. Our experiments found this approach to be effective in isolating BGP from attack traffic.

Conclusion Feasibility of attacks against Internet routing infrastructure Lack of protection of routing traffic Prevention solution using existing router configurations Ubiquitous deployment is challenging Difficulties in detecting and defending against coordinated attacks may affect any network infrastructure We now conclude with the following observations. 1. I have shown that attack against internet routing infrastructure is possible One of the fundamental reason is that BGP traffic is not protected from data traffic 2. We use router’s existing features to prevent the attack. But it’s difficult to ensure these configurations are adopted globally. As long as some networks do not deploy these best common practices, Internet is still susceptible to potential resulting routing instability. Finally, it is still an open issue to detect and defend against multi-host, coordinated attack, which can also be used to attack any network infrastructure.

Thank you!

Backup slides

Attack flow notations Periodic, on-off square-wave flow Burst period length L Inter-burst period T Burst magnitude of the peak R Burst Length L Magnitude of the peak R Inter-burst period T

Attack inter-burst period’s impact on table transfer duration (R=185Mbps,L=200msec)

Attack peak magnitude’s impact on session reset and table transfer duration (Top:T=600msec,L=200msec) (Bottom:T=1.2s,L=200msec) Normalized avg rate 0.48 Normalized avg rate 0.24

Synchronization accuracy

BGP table transfer with WRED enabled under attack