Flash Crowds And Denial of Service Attacks: Characterization and Implications for CDNs and Web Sites Aaron Beach Cs395 network security

OVERVIEW What is a “Flash Event?” (FE) What is a “Denial of Service Attack?” What is the difference? How can we distinguish between them? What is/are the solution(s)? – Adaptive Content Distribution Networks? –Others? –Do you have any ideas??? Think about it

Flash Events A flash event (FE) is a large surge in traffic to a particular Web site causing a dramatic increase in server load and putting severe strain on the network links leading to the server, which results in considerable increase in packet loss and congestion “Flash crowds”

Denial of Service Attack (DoS) An explicit attempt by attackers to prevent legitimate users of a service from using that service Their definition: –any attempt to undermine a Web site What do you think?

The Major Differences Flash Events represent legitimate traffic to a website. This often means the website wants to service these requests as well as possible, while DoS attacks our unwanted and should not be serviced, but ignored or controlled.

Distinguishing Between Them 3 main characteristics –Traffic patterns –Client characteristics –File reference characteristics

Traffic Patterns Overall traffic volume determines how much a server should provision resources to keep the site operational Servers can shut down from over use Studying these patterns allows us to articulate the period when an unusually large number of clients can overwhelm a site We also can understand how and in what time pattern the server must defend against these rises in traffic

How substantial can an FE be? 88.2% of traffic in 11% of time 71% of traffic in 7% of time

You can see the spikes in traffic They look indistinguishable?

Now do they look the same? Quite different… however

Behavior of traffic First fifteen minutes They both rise, one over a period of One over 70 minutes One over 40 seconds

Client Characteristics and clustering They use a network-aware clustering technique to determine the topological distribution of clients in FE and DoS. Client clustering allows one to aggregate individual clients into groups belonging to the same administrative domain. Clustering uses a large collection of unique network prefixes assembled from a wide set of BGP routing tables. The various client IP addresses are grouped into clusters based on longest prefix matching.

Clusters and Clients trends Spikes in request volumes during an FE correspond closely with the spikes in the number of clients accessing the site. Thus, the number of clients in a flash event follows the same increase patterns as the overall request rate.

No large change in average per-client request rate

“Old” clusters during an FE Clusters that have already visited the site VS new clusters during an FE During the two FEs we are studying there was 42.7% in the Play-along trace and 82.9% in the Chile trace that were “old” clusters demonstrating that in these FEs a large percentage had made previous requests

File Reference Characteristics Locality of reference enables a reduction of server load through caching. They use these characteristics in designing an “adaptive CDN.” We consider: –aggregate file references –reference patterns of individual clients –reference patterns of client clusters.

What files are accessed in FE 60% (61% and 82% for Play-along and Chile, respectively) of documents are accessed only during flash events. So, CDN’s will not cache and not be prepared for the FE Indeed, most CDN caches will not have these documents at the beginning of the FE So there will be many misses at the beginning of an FE

Popularity of files

Also about clusters and file popularity Requests for documents come from many different Clusters… This means that current CDNs will result in many different serves getting requests for the same file… resulting in more misses for the files popular only during FEs

Password cracking Much like DoS attacks We must detect early and stop them Detect “401 unauthorized” messages

Trends during attacks During attacks most clients making requests were new… never had made requests before Only 0.6% of the clusters seen at one site during the attack had been seen before, and the percentage of these clusters drops to 0.1% for another site.

Trends in DoS requests (Code Red)

Rise in Clusters vs Clients FE DoS

Overlap of clusters during DoS Calculated overlap for DoS was: 0.6% in the creighton site 0% in the fullnote site 1.8% in the spccctxus site 14.3% rellim site. Compare this to: 42.7% and 82.9% in the FEs studied

Comparing the two: DoS vs FE

SOLUTION TIME!!! What should the server do when it is being overwhelmed?? –Discard “more malicious” requests How? –Monitor users and average request rate –Periodically “cluster” addresses –When overwhelmed… drop malicious addresses (must belong to old clusters and continue “normal” request rates - Solution not too taxing on processes and you can implement it in an filtering accept() function

Will this always work?? Sometimes DoS attacks are able to flood links… and the server can do nothing… Since attacker does not know who is using site they cannot know which clusters to send with (the author thinks this is a way to avoid letting this information prepare attackers… what do you think??

What about FEs? If we know how to deal with DoS attacks… we still have the problem of what to do when flash events happen Solution ： Adaptive CDN

Adaptive CDN “Dynamic Delegation” The more caches the more requests, so make less caches with more space Have primaries and delegates… When a FE is detected the DNS servers sends requests to delegates first and they go to primaries… Only primaries can make requests to origin server, clustering caches

Algorithm for Dynamic Delegation When a node “P” is overloaded it redirects packets to another node that has a low load, using it as a “delegate” When a node goes low it stops using delegates Tests show this lowered load on origin server by: a factor of 50 in one test and 30 in the other… without too high load distribution in the caches.

Review Flash Event (Flash Crowd) FE vs DoS Difference and Detection Detecting and stopping Dealing with FE using adaptive CDN