1. Filtering Spam Under Attack: some notes from the field
Aleksander Kołcz
Microsoft Live Labs
(Slides stolen from lots of people,
especially Josh Goodman and Geoff Hulten)

2. Source: Pew Internet & American Life Project

3. Email addiction 41% check email first thing in the morning
Source: AOL Addiction Survey
41% check first thing in the morning
23% have checked in bed in their pajamas
40% of users have checked their in the middle of the night.
26% say they haven't gone more than two to three days without checking their .

4. Too bad we have spam SPAM is the number one problem for email systems
Estimates from about 71% to 87% of mail is spam
At 71%, if you stop 90% of the spam, 1/5 of your mail will be spam
Over a billion spam a day will get past filters worldwide.

5. Overview Email Spam An important application
Lots of great research problems
Spam
Definitional problems
Challenges in applying standard ML/TM procedures
Techniques spammers use
Other solutions to Spam
Other kinds of communication abuse (SPIM)

6. Email : some interesting problems
Finding what’s important
Priorities
Task Flags
Organizing mail
Auto foldering
Auto tagging
Finding what’s interesting
Automatic search
Contact finding

7. Other interesting email research
Not all research is language oriented
Social Network Analysis
Calendar Research
HCI
Visualization
Storage
Next generation protocols

8. Solutions to Spam Filtering Postage (Disposable Email Addresses)
Machine Learning
Matching/Fuzzy Hashing
(Blackhole Lists (IP addresses))
Postage
Turing Tests, Money, Computation
(Disposable Addresses)
Smart Proof

9. Machine Learning/Text Mining
A labeled iid sample from the spam/non-spam distribution
Well defined performance criteria: false positive/negative rates, misclassification costs, processing/storage costs
Word features (eg, breaking by whitespace) possibly enhanced with specific ones (eg, header features)
The favorite linear model (Naïve Bayes, SVM, Logistic Regression)

10. What is spam ? Unsolicited commercial mail?
A user signs up for ebay and signs an agreement to receive communications from various parts of ebay
An arrives from half.com
The user classifies is as spam
Yet half.com is truly part of ebay
Users are not likely to play detective and trace back affiliations and business relationships of senders

11. What is spam ? The emails individual users don’t like?
RE: how much stuff can I safely load on my rear rack?
After receiving the 15th posting to this mailing list thread (cycling) I was ready to consider further follow ups as spam
I like some offers from Amazon.com, but not others
What I like right may depend on what I bought recently
s with certain content or from certain senders can sometimes be considered spam, but not always

12. What is spam ? Emails people agree are spam?
There are a lot of s users are split about – should we just let them go (graymail)?
Agree on what?
Personalization and per-message randomization make it hard to figure out who gets the same campaign
Thanks to botnets, a single spam campaign comes from a lot of different senders
s with certain content or from certain senders can sometimes be considered spam, but not always

13. Spam campaign randomization

14. What is spam ? Emails people agree are spam?
There are a lot of s where users are split about – should we just let them go (graymail)?
Agree on what?
Personalization and per-message randomization make it hard to figure out who gets the same campaign
Thanks to botnets, a single spam campaign comes from a lot of different senders
s with certain content or from certain senders can sometimes be considered spam, but not always

15. What is spam ? Emails caught in “honeypot” accounts?
Long dead accounts and never used accounts should not receive legitimate
Everything they get is most likely spam, but...
Mistyping and old subscriptions could deliver legitimate mail occasionally
Users who engage in real communication may be getting spam that is different
Honeypots are great but ones has to consider false positives and sample selection bias

16. Examples of good-mail
Even if noisy, labeled examples of non-spam are relatively easy to acquire (people are eager to help)
Examples of good mail are much tougher to come buy
Nobody wants to share sensitive material, but such messages are arguably the most expensive to misclassify
Good- datasets often suffer from sample-selection bias, where certain important content areas are under-represented

17. Label noise
People make old fashioned mistakes when donating samples
The error rates can run as high as 1-5%
This complicates ensuring a low enough FP rate
With sampling bias, misclassifying a few legitimate messages as spam may translate into a very large FP rate
Setting a very low FP rate over noisy data may lead to an unacceptably low recall

18. Operating point worries
Most machine learning focuses on accuracy
Assumes all errors equally bad
For spam (and most other problems) cost of deleting good mail much higher than cost of spam in inbox
(No missed spam)
Some research on optimizing area under the curve – so you get good performance everywhere
Almost no research on how to optimize for a specific point.
(All spam missed) 1
(No good caught) 1
(All good caught)

19. The cost formula and its problems
Classic cost-sensitive decision making
The prevalence of spam, π , is highly time and user dependent
The misclassification costs tend to be hard to quantify
Setting the FP rate < th is often more sensible

21. Adversarial attacks and why size matters
Spammers respond to new filter defenses, but not uniformly
Attacking my personal filter is a waste of time/money
Attacking a corporate filter may lead to a few thousand successful deliveries
A successful attack against Hotmail, AOL, Yahoo! or Gmail can lead to millions of eyeballs!

22. Adversarial attacks: agility counts
Spammers can respond very fast with new tactics
Large systems are often hampered by lengthy deployment procedures
Solutions need to be naturally adaptive or allow for easy manual intervention
Constant monitoring and rapid response are paramount

23. What Happened When we Shipped an Adaptive Spam Filter
The first spam filter we shipped was adaptive
If user corrected mistakes, we improved the filter.
What to do if the user does not correct mistakes?
We assumed the filter was correct
For users who rarely fixed mistakes, this lead to catastrophically bad results – the filter got worse and worse and worse

24. Threshold Drift Conservative Threshold Setting
Separator: 50/50 mark
We are conservative in our filtering.
For instance, maybe we need to be 96% certain that mail is spam before we classify as spam
Conservative Threshold: 96% sure

25. Threshold Drift Lots of Spam Classified as Good
Separator: 50/50 mark
Conservative Threshold: 96% sure

26. Threshold Drift Old Conservative Threshold: 96% sure Old Separator:
New Separator:
New Conservative Threshold: 96% sure

27. Threshold Drift Old Conservative Threshold: 96% sure Old Separator:
New Separator:
New Conservative Threshold: 96% sure

28. Adaptation with partial user feedback is hard
Users may correct all errors, or only all spam, all good, 50% spam, 10% spam, no errors, etc.
Need to work no matter what the user correction rate is
Great problem that you find when you try to build a real system

29. Attack vectors/techniques
Reputation based attacks:
Sending (or pretending) to send from sources (IPs, domains accounts) with good or unknown reputation
Content based attacks:
Chaff
Invisible ink
Encoding
Image spam
Spelling

30. The Hitchhiker Chaffer
Content Chaff
Random passages from the Hitchhiker’s Guide
Footers from valid mail
“This must be Thursday,” said Arthur to himself, sinking low over his beer, “I never could get the hang of Thursdays.”
Express yourself with MSN Messenger 6.0…

31. Hitchhiker Chaffer’s Later Work: invisible ink
Can use hidden text, e.g. white on white or many other tricks
User sees only spammy text
Spam filter sees everything, including good words.

32. Hitchhiker Chaffer’s Later Work: invisible ink
Can use hidden text, e.g. white on white or many other tricks
Also included a number of unusual statements made by candidates during, ‘On display? I eventually had to go down to the cellar to find them.’

33. Weather Report Guy Content in Image Good Word Chaff
Weather, Sunny, High 82, Low 81, Favorites…

34. Secret Decoder Ring Looks easy Is it?
Viagra – Proven sexual aid to enhance performance…

35. Secret Decoder Ring Dude
Character Encoding
HTML word breaking
Pharmacy
Produc<!LZJ>t<!LG>s

36. Diploma Guy
Word Obscuring
Dplmoia Pragorm
Caerte a mroe prosoeprus

37. Diploma Guy
Word Obscuring
Dipmloa Paogrrm
Cterae a more presporous

38. Diploma Guy
Word Obscuring
Dimlpoa Pgorram
Cearte a more poosperrus

39. Diploma Guy
Word Obscuring
Dpmloia Pragorm
Caetre a more prorpeosus

40. Diploma Guy
Word Obscuring
Dplmoia Pragorm
Carete a mroe prorpseous

41. More of Diploma Guy
Diploma Guy is good at what he does

42. Trends in Spam Exploits (Hulten et al.)
2003
Spam
2004
Delta
(Absolute %)
Description
Word Obscuring 4%
20%
16%
Misspelling words, putting words into images, etc.
URL Spamming 0%
10%
Adding URLs to non-spam sites (e.g. msn.com).
Domain Spoofing
41%
50% 9%
Using an invalid or fake domain in the from line.
Token Breaking 7%
15% 8%
Breaking words with punctuation, space, etc.
MIME Attacks 5%
11% 6%
Putting non-spam content in one body part and spam content in another.
Text Chaff
52%
56%
Random strings of characters, random series or words, or unrelated sentences.
URL Obscuring
22%
17%
-5%
Encoding a URL in hexadecimal, hiding the true URL with sign, etc.
Character
Encoding
Pharmacy renders into Pharmacy.

43. Economy considerations
Building complex systems (eg combinations of system wide and personal filters) can improve filtering accuracy
The implementation costs can be substantial though
E.g., a fully personalized service-side spam filtering complex:
Per user model storage
Per user message processing (important for messages with multiple recipients)

44. Solutions to Spam Filtering Postage (Disposable Email Addresses)
Machine Learning
Matching/Fuzzy Hashing
(Blackhole Lists (IP addresses))
Postage
Turing Tests, Money, Computation
(Disposable Addresses)
Smart Proof

45. Matching/Fuzzy Hashing
Use “Honeypots” – addresses that should never get mail
All mail sent to them is spam
Look for similar messages that arrive in real mailboxes
Exact match easily defeated
Use fuzzy hashes
How effective?
Dedicated attacks can defeat near-duplicate detection
Make
Earn
thousands of dollars
lots of money
working at home
in the comfort of your own house
!!! .

46. MSN blocks e-mail from rival ISPs
Blackhole Lists
MSN blocks from rival ISPs
By Stefanie Olsen Staff Writer, CNET News.com February 28, 2003, 2:34 PM PT
Microsoft's MSN said its services had blocked some incoming messages from rival Internet service providers earlier this week, after their networks were mistakenly banned as sources of junk mail.
The Redmond, Wash., company, which has nearly 120 million customers through its Hotmail and MSN Internet services, confirmed Friday it had wrongly placed a group of Internet protocol addresses from AOL Time Warner's RoadRunner broadband service and EarthLink on its "blocklist" of known spammers whose mail should be barred from customer in-boxes.
Once notified of the error by the two ISPs, MSN moved the IP addresses "over to a safe list immediately," according to a Microsoft spokeswoman.
Lists of IP addresses that send spam
Open relays, Open proxies, DSL/Cable lines, etc…
Easy to make mistakes
Open relays, DSL, Cable send good and spam…
Who makes the lists?
Some list-makers very aggressive
Some list-makers too slow
                                                                                        

47. Postage Basic problem with email is that it is free Multiple kinds of
Force everyone to pay (especially spammers) and spam goes away
Send payment pre-emptively, with each outbound message, or wait for challenge
Multiple kinds of
payment:
Turing Test,
Computation,
Money

48. Turing Tests (HIP, CAPTCHA) (Naor ’96)
You send me mail; I don’t know you
I send you a challenge: type these letters
Your response is sent to my computer
Your message is moved to my inbox, where I read it

49. Computational Challenge (Dwork and Naor ’92)
Sender must perform time consuming computation
Example: find a hash collision
Easy for recipient to verify, hard for sender to find collision
Requires say 10 seconds (or 5 minutes?) of sender CPU time (in background)
Can be done preemptively, or in response to challenge

50. $$$ Money Pay actual money (1 cent?) to send a message
Interesting variation: take money only when user hits “Report Spam” button
Otherwise, refund to sender
Free for non-spammers to send mail, but expensive for spammers
Requires multiple monetary transactions for every message sent – expensive
Who pays for infrastructure?

51. SmartProof: Most challenge-response approaches challenge every message
Use machine learning. Challenge only suspicious messages (avoids annoying challenges)
Can auto-respond with computation
Least annoying to sender – may never see challenge
Can respond by solving a Turing Test
Works for people with old or incompatible computers
Can respond with micro-payment

52. Other kinds of abuse Email spam Chat rooms (SPAT)
Instant Messenger (SPIM)
Blog spam
Web spam
SMS spam
IP phone spam

53. Chat Room Spam MSN closed its free chat rooms
Spambots come in and pretend to chat
But really just advertising porn sites
Some spambots trivial
Don’t talk at all, but take up space
Link to porn spam in their profile
Some spambots very sophisticated
You can have a short conversation with them before they try to convince you to go to their website
Randomized conversations so hard for users to spot

54. Chat Bot
joshuagood9: hi there superchristina: hey there how u doin? joshuagood9: doing fine, and you? superchristina: hey there how u doin? joshuagood9: are you a bot? superchristina: im not a bot are u? lol joshuagood9: are you a bot? superchristina: i hate bots lol joshuagood9: how old are you? superchristina: whats up? joshuagood9: asl? superchristina: im 21 f usa and u? joshuagood9: I am fine, thank you superchristina: right on asl?...  im 20 f usa joshuagood9: 74/M, WA superchristina: nice age joshuagood9: thank you superchristina: yw sweety..could u do me a favor..check out my homepage and my profile see if my cam works? brb

56. Instant Messenger Spam “SPIM”
Send messages to people via IM
Microsoft solved this by requiring people to get permission before IMing
Spammers put spam in their “name” – so permission request message now has spam!

57. Blog Spam Post comments with links in blogs
The links used to be used by search engines as part of rankings
Most search engines now completely ignore these links (throwing away valuable information)
Spammer posts links from his blog to victim blog
Trackback software shows victim that there is a link to his blog
Victim uses trackback to see who linked Many providers disabling trackbacks

58. SPIM, etc. are great NLP problems
Tons of ways to obfuscate spam, because you can send pictures and arbitrary HTML
IM, chat rooms, blog comments all basically restricted to plain text
NLP techniques may be more appropriate for these domains than for spam
Other kinds of abuse in chat rooms
Pedophiles, phishing, etc.
MSN and Yahoo have both closed off large parts of their chat room systems because of pedophiles

59. Conclusion: lots of exciting research
Priorities, Task Flags, Auto-folder, Auto-Tag, Automatic Search
Spam
Still haven’t solved it – can keep improving
New problems like phishing
Apply to other domains (SPIM, etc)
The conference on and Anti-Spam
CEAS <