Network-Based Spam Filtering Anirudh Ramachandran Nick Feamster Georgia Tech

2 Spam 75-90% of all traffic –PDF Spam: ~11% and growing –Content filters cannot catch! Late 2006: there was a significant rise in spammers use of botnets, armies of PCs taken over by malware and turned into spam servers without their owners realizing it. August 2007: Botnet-based spam caused volumes to increase 53% from previous day Source: NetworkWorld, August 2007

3 More Than Just a Nuisance As of August 2007, one in every 87 s constituted a phishing attack Targeted attacks on the rise –20k-30k unique phishing attacks per month –Spam targeted at CEOs, social networks on the rise

4 One Approach: Filtering Prevent traffic from reaching users inboxes by distinguishing spam from ham Key question: What features best differentiate spam from legitimate mail? –Content –IP address of sender –Behavioral features

5 Content-Based Filtering is Malleable Low cost to evasion: Spammers can easily alter features of an s content can be easily adjusted and changed Customized s are easy to generate: Content- based filters need fuzzy hashes over content, etc. High cost to filter maintainers: Filters must be continually updated as content-changing techniques become more sophistocated

6 This Talk: Network-Based Filtering Filter based on how it is sent, in addition to simply what is sent. Network-level properties are more fixed –Hosting or upstream ISP (AS number) –Botnet membership –Location in the network –IP address block Challenge: Which properties are most useful for distinguishing spam traffic from legitimate ? Very little (if anything) is known about these characteristics

7 Talk Outline Study current sending and mitigation techniques –Network-level behavior of spammers –The effectiveness of IP-based blacklists Design behavioral based filtering techniques –Behavioral blacklisting General idea First trial of system on basic set of features –Joint work with Santosh Vempala Deploy distributed monitoring system to –learn distinguishing features on the fly

8 Studying Sending Patterns Where is the spam coming from? –What IP address space? –ASes? –What are the OSes of the senders? What techniques? –Botnets –Short-lived route announcements –Shady ISPs Capabilities and limitations? –Bandwidth –Size of botnet army

9 BGP Spectrum Agility Log IP addresses of SMTP relays Join with BGP route advertisements seen at network where spam trap is co-located. A small club of persistent players appears to be using this technique. Common short-lived prefixes and ASes / / / ~ 10 minutes Somewhere between 1-10% of all spam (some clearly intentional, others might be flapping)

10 Why Such Big Prefixes? Flexibility: Client IPs can be scattered throughout dark space within a large /8 –Same sender usually returns with different IP addresses Visibility: Route typically wont be filtered (nice and short)

11 Characteristics of IP-Agile Senders IP addresses are widely distributed across the /8 space IP addresses typically appear only once at our sinkhole Depending on which /8, 60-80% of these IP addresses were not reachable by traceroute when we spot- checked Some IP addresses were in allocated, albeit unannounced space Some AS paths associated with the routes contained reserved AS numbers

12 Lessons for Improving Spam Filters IP-Based Blacklists are Becoming Less Effective Effective spam filtering requires –A better notion of end-host identity –Filtering based on features that are more persistent Some features may require network-wide monitoring capabilities

13 Two Parts Study the network-level behavior of spammers –Majority of spam comes from a very small portion of the Internet address space –Most coming from Windows hosts –Most senders low volume to our domain –Conventional blacklists somewhat ineffective Develop behavioral based filtering techniques –Behavioral blacklisting

14 Two Metrics for Evaluating Blacklists Completeness: The fraction of spamming IP addresses that are listed in the blacklist Responsiveness: The time for the blacklist to list the IP address after the first occurrence of spam

15 Completeness of IP Blacklists ~80% listed on average ~95% of bots listed in one or more blacklists Number of DNSBLs listing this spammer Only about half of the IPs spamming from short-lived BGP are listed in any blacklist Fraction of all spam received Spam from IP-agile senders tend to be listed in fewer blacklists

16 Completeness and Responsiveness 10-35% of spam is unlisted at the time of receipt % of these IP addresses remain unlisted even after one month

17 Problems with Existing Blacklists Based on ephemeral identifier (IP address) –More than 10% of all spam comes from IP addresses not seen within the past two months Dynamic renumbering of IP addresses Stealing of IP addresses and IP address space Compromised machines IP addresses of senders have considerable churn Requires a human to first notice the behavior –Spamming is compartmentalized by domain and not analyzed across domains

18 Problem: Changing IP Addresses Fraction of IP Addresses

19 Problem: Low Volume to Each Domain Lifetime (seconds) Amount of Spam Most bot IP addresses send very little spam, regardless of how long they have been spamming. Single-domain observation cannot detect.

20 SpamTracker: Main Idea and Intuition Idea: Blacklist sending behavior (Behavioral Blacklisting) –Identify sending patterns that are commonly used by spammers Intuition: Much more difficult for a spammer to change the technique by which mail is sent than it is to change the content

21 SpamTracker Design For each sender, construct a behavioral fingerprint Cluster senders with similar fingerprints Filter new senders that map to existing clusters Approach Cluster Classify IP x domain x time Collapse LookupScore

22 Building the Classifier: Clustering Feature: Distribution of sending volumes across recipient domains Clustering Approach –Build initial seed list of bad IP addresses –For each IP address, compute feature vector: volume per domain per time interval –Collapse into a single IP x domain matrix: –Compute clusters

23 Clustering: Output and Fingerprint For each cluster, compute fingerprint vector: New IPs will be compared to this fingerprint IP x IP Matrix: Intensity indicates pairwise similarity

24 Classifying IP Addresses Given new IP address, build a feature vector based on its sending pattern across domains Compute the similarity of this sending pattern to that of each known spam cluster –Normalized dot product of the two feature vectors –Spam score is maximum similarity to any cluster

25 Evaluation Emulate the performance of a system that could observe sending patterns across many domains –Data: Postfix logs –Build clusters / train on given time interval –Evaluate classification with subsequent data in trace Evaluate classification –Relative to labeled logs –Relative to IP addresses that were eventually listed

26 Dataset: Summary and Issues 30 days of Postfix logs from large provider –Time, remote IP, receiving domain, accept/reject –Allows us to observe sending behavior over a large number of domains –Problem: About 15% of accepted mail is also spam Creates problems with validating SpamTracker 30 days of SpamHaus database in the month following the Postfix logs –Allows us to determine whether SpamTracker detects some sending IPs earlier than SpamHaus

27 Initial Results Many single- domain senders Large volumes to just a few domains SpamTracker Score Ham Spam Problems

28 Rejected Mails Have Higher Scores Ham Spam SpamTracker Score

29 SpamTracker and Early Detection Compare SpamTracker scores on accepted mail to the SpamHaus database –About 15% of accepted mail was later determined to be spam –Can SpamTracker catch this? Of 620 s that were accepted, but sent from IPs that were blacklisted within one month –65 s had a score larger than 5 (85 th percentile)

30 Deployment Integration with existing infrastructure –Deploy SpamTracker as yet another DNSBL –Existing spam filters use SpamTracker score as an additional feature –Advantage: easy deployment On the wire –Infer connections/ from traffic flow records in individual domains –Advantage: Stop mail closer to the source

31 Improving Classification Use additional features, and combining for more robust classification –Temporal: interarrival times, diurnal patterns, etc. –Spatial: sending patterns of groups of senders Improved similarity computation –Better similarity metrics –Better metrics for detecting early onset

32 Evasion Problem: Malicious senders could add noise to a large feature vector –Possibility: Use smaller number of trusted domains Problem: Malicious senders could change sending behavior to emulate normal senders –In doing so, they may limit their own effectiveness

33 Other Questions and Challenges Reactivity: Can the features be observed quickly enough to construct the fingerprints? Scalability: How can the data be aggregated and collected without imposing too much overhead? Reliability: How can SpamTracker be replicated to better defend against attack or failure? Sensor placement: From where should we watch spam to ensure that the clusters can be distinguished? Symbiosis between botnet detection and spam filtering

34 Summary Spam is on the rise and becoming more clever –12% of spam now PDF spam. Content filters are falling behind –Also becoming more targeted IP-Based blacklists are evadable –Up to 30% of spam not listed in common blacklists at receipt. ~20% remains unlisted after a month –Spammers commonly steal IP addresses New approach: Behavioral blacklisting –Blacklist how the mail was sent, not what was sent