Antispam GARR Michele Michelotto Hepix Karlsruhe, 11 May 2005

HEPIX, Karlsruhe 11/5/05 1 Antispam activities at GARR WG sec mail Enrico Ardizzoni (Università di Ferrara) Enrico Ardizzoni (Università di Ferrara) Alberto D’Ambrosio (INFN, Torino) Alberto D’Ambrosio (INFN, Torino) Roberto Cecchini (INFN, Firenze) Roberto Cecchini (INFN, Firenze) Fulvia Costa (INFN, Padova) Fulvia Costa (INFN, Padova) Giacomo Fazio (INAF, Palermo) Giacomo Fazio (INAF, Palermo) Antonio Forte (INFN, Roma 1) Antonio Forte (INFN, Roma 1) Matteo Genghini (IASF, Bologna) Matteo Genghini (IASF, Bologna) Michele Michelotto (INFN, Padova) Michele Michelotto (INFN, Padova) Ombretta Pinazza (INFN, Bologna) Ombretta Pinazza (INFN, Bologna) Alessandro Spanu (INFN, Roma 1) Alessandro Spanu (INFN, Roma 1) Alfonso Sparano (Università di Salerno) Alfonso Sparano (Università di Salerno)

HEPIX, Karlsruhe 11/5/05 2 Antispam activities at GARR Goals anti-spam and anti-virus anti-spam and anti-virus Stop them or at least reduce to a reasonable level Stop them or at least reduce to a reasonable level “best practices” “best practices” mail services configuration and mail server protection mail services configuration and mail server protection Sender authentication Sender authentication SPF, domain keys SPF, domain keys Dissemination Dissemination mailto: mailto:

HEPIX, Karlsruhe 11/5/05 3 Antispam activities at GARR anti-spam SpamAssassin (SA) analysis and efficiency improvement: SpamAssassin (SA) analysis and efficiency improvement: Monitoring; Monitoring; Bayesian filter; Bayesian filter; Real Time Block List (RBL); Real Time Block List (RBL); Network distributed “cooperative” systems. Network distributed “cooperative” systems.

HEPIX, Karlsruhe 11/5/05 4 Antispam activities at GARR anti-spam Alternative tools tests: Alternative tools tests: Bogofilter: Bogofilter: DSPAM: DSPAM:

HEPIX, Karlsruhe 11/5/05 5 Antispam activities at GARR SpamAssassin Rule based Rule based Each rule adds a score (positive or negative) Each rule adds a score (positive or negative) Mail over threshold can be deleted, marked, moved to a quarantine folder Mail over threshold can be deleted, marked, moved to a quarantine folder Choice of threshold is difficult Choice of threshold is difficult Some spam have a score lower than legitimate mail (ham) Some spam have a score lower than legitimate mail (ham)

HEPIX, Karlsruhe 11/5/05 6 Antispam activities at GARR Dove metto la soglia? Threshold too high – Many FALSE NEGATIVES Two weeks s spams (75.7%)

HEPIX, Karlsruhe 11/5/05 7 Antispam activities at GARR Dove metto la soglia? Threshold too low – Some FALSE POSITIVES (Dangerous) Two weeks s spams (75.7%)

HEPIX, Karlsruhe 11/5/05 8 Antispam activities at GARR Indipendent methods Improve the spam/ham identification Improve the spam/ham identification I can’t move the threshold I can’t move the threshold If I lower it I get too many False Negatives If I lower it I get too many False Negatives If I raises is even worse because I can get some False Positives If I raises is even worse because I can get some False Positives Look for “indipendent methods” Look for “indipendent methods” Bayesian Filters Bayesian Filters Cooperative methods Cooperative methods RBL RBL

HEPIX, Karlsruhe 11/5/05 9 Antispam activities at GARR Bayesian Filters Based on Bayesian statistics Based on Bayesian statistics The filters “learn” which words (actually tokens) are more probable in ham and spam The filters “learn” which words (actually tokens) are more probable in ham and spam Bayesian filters ageing Bayesian filters ageing Learning by manually submitting ham spam sample is time consuming Learning by manually submitting ham spam sample is time consuming Auto Learning is dangerous. Spammers send mail designed to “poison” the filters Auto Learning is dangerous. Spammers send mail designed to “poison” the filters Best performance with frequents update submitted by the users Best performance with frequents update submitted by the users Even better: different databases for each user Even better: different databases for each user

HEPIX, Karlsruhe 11/5/05 10 Antispam activities at GARR Bayesian Filters Filters “ageing”: must keep them up to date. Filters “ageing”: must keep them up to date. Manual update is time expensive Manual update is time expensive Frequents update from selected samples chosen by users, best with individual db for each user. Frequents update from selected samples chosen by users, best with individual db for each user. Automatic update is dangerous Automatic update is dangerous Some mail sent only for bayesing filter “poisoning”. Some mail sent only for bayesing filter “poisoning”.

HEPIX, Karlsruhe 11/5/05 11 Antispam activities at GARRageing AGEING NEW TRAINING

HEPIX, Karlsruhe 11/5/05 12 Antispam activities at GARR Real-Time Block List For each a DNS query is issued to see if the sender is present in a list of known spammer For each a DNS query is issued to see if the sender is present in a list of known spammer Good method to add score Good method to add score Don’t use to reject mail Don’t use to reject mail Spoofing of sender Spoofing of sender Some RBL not very accurate in checking if sender is a real spammer or in removing those who fixed the problem Some RBL not very accurate in checking if sender is a real spammer or in removing those who fixed the problem URIRBL: Very good because the check is done against the URL in the mail body URIRBL: Very good because the check is done against the URL in the mail body The spammer will not spoof the URL in the body !!! The spammer will not spoof the URL in the body !!!

HEPIX, Karlsruhe 11/5/05 13 Antispam activities at GARR Cooperative methods UBE: Unsolicited Bulk UBE: Unsolicited Bulk Based on the Mass Diffusion of spam Based on the Mass Diffusion of spam Razor: Razor: Users submit spam to a network of Razor server. Users submit spam to a network of Razor server. Mail with many submission tagged as spam Mail with many submission tagged as spam Users rating Users rating Closed protocol and closed server network Closed protocol and closed server network Pyzor: Pyzor: Similar to Razor but protocol and sw is open source and you can became a server Similar to Razor but protocol and sw is open source and you can became a server

HEPIX, Karlsruhe 11/5/05 14 Antispam activities at GARR DCC Mail with similar signature are counted in several sites Mail with similar signature are counted in several sites If a mail is seen by many DCC server is tagged as suspect If a mail is seen by many DCC server is tagged as suspect Open Network Open Network Our group now has 3 DCC Servers Our group now has 3 DCC Servers Each server can provide anonymous access or high priority access to registered user Each server can provide anonymous access or high priority access to registered user

HEPIX, Karlsruhe 11/5/05 15 Antispam activities at GARR Dcc stats

HEPIX, Karlsruhe 11/5/05 16 Antispam activities at GARR DCC: our stats A tipical day at the DCC server at IASF in Palermo A tipical day at the DCC server at IASF in Palermo 800k checksum request (70k from registered clients) 800k checksum request (70k from registered clients) 1.2M report from clients 1.2M report from clients Average response time 5ms Average response time 5ms

HEPIX, Karlsruhe 11/5/05 17 Antispam activities at GARR Spam in September spam received in my mailbox during the CHEP week 12% False Negatives

HEPIX, Karlsruhe 11/5/05 18 Antispam activities at GARR Spam in September 04 From 12% at the end of September to 1.7% False Negatives at end of November

HEPIX, Karlsruhe 11/5/05 19 Antispam activities at GARR Monitoring trend

HEPIX, Karlsruhe 11/5/05 20 Antispam activities at GARR Top plugin

HEPIX, Karlsruhe 11/5/05 21 Antispam activities at GARR Sender Authentication Sender Policy Framework (SPF): Sender Policy Framework (SPF): Each DSN server should publish a “reverse MX record” DNS listing the smtp server autorized to send for that domain Each DSN server should publish a “reverse MX record” DNS listing the smtp server autorized to send for that domain The receiver can use this information to reject mail or to increase SA score The receiver can use this information to reject mail or to increase SA score This means that the roaming users should always use his own SMTP server (after authentication) This means that the roaming users should always use his own SMTP server (after authentication)

HEPIX, Karlsruhe 11/5/05 22 Antispam activities at GARR

HEPIX, Karlsruhe 11/5/05 23 Antispam activities at GARR SPF tests Salerno University Salerno University One month One month 650 · 10 3 mail 650 · 10 3 mail 32% from SPF compliant domain 32% from SPF compliant domain 12% esternal 12% esternal 20% internal (useful to cut all the spam with faked internal sender, mostly virus or phishing) 20% internal (useful to cut all the spam with faked internal sender, mostly virus or phishing)

HEPIX, Karlsruhe 11/5/05 24 Antispam activities at GARR Best practices Open port 25 only to your site server Open port 25 only to your site server Open ports 587 and 468 for external authenticated users Open ports 587 and 468 for external authenticated users Force external users authentication (necessary to implement SPF) Force external users authentication (necessary to implement SPF) Antivirus configuration to avoid sender notification (since is almost always spoofed) Antivirus configuration to avoid sender notification (since is almost always spoofed) “greet pause” on sendmail (≥ 8.13) “greet pause” on sendmail (≥ 8.13)

HEPIX, Karlsruhe 11/5/05 25 Antispam activities at GARR Open item “unofficial” plugin test “unofficial” plugin test Sender Authentication Sender Authentication Bogofilter and dspam tests Bogofilter and dspam tests More DCC or Pyzor server? More DCC or Pyzor server? Online filter (spam rejection)? Online filter (spam rejection)? Close group and buy commercial “turnkey” sw ? Close group and buy commercial “turnkey” sw ? Like we do with A/V Like we do with A/V (e.g. Sophos PureMessage) (e.g. Sophos PureMessage)

HEPIX, Karlsruhe 11/5/05 26 Antispam activities at GARR Questions?