Filtering Mail with Mail::Audit and Mail::SpamAssassin Creede Lambard penguinsinthenight.com 20 August 2002

General Outline:

● How UNIX handles mail

General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial

General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial ● Why mail filtering is a Good Thing tm

General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial ● Why mail filtering is a Good Thing tm ● If you use Windows...

General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial ● Why mail filtering is a Good Thing tm ● If you use Windows... ● Using Mail::Audit

General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial ● Why mail filtering is a Good Thing tm ● If you use Windows... ● Using Mail::Audit ● Using Mail::SpamAssassin

How Unix handles your mail

.forward to another mail address:

How Unix handles your mail Piping to another program: | vacation

Does this look familiar?

spam

● Unsolicited commercial

spam ● Unsolicited commercial – Sent in bulk

spam ● Unsolicited commercial – Sent in bulk – Directly or indirectly advertises a product or service

spam ● Unsolicited commercial – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient

spam ● Unsolicited commercial – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient ● Not necessarily mail you don't want...

spam ● Unsolicited commercial – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient ● Not necessarily mail you don't want... – Although for purposes of this presentation we'll treat them the same.

When Spam tm is acceptable

spam is a Bad Thing tm

● It shifts the burden of costs to the recipient

spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net

spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time

spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● Items/services advertised through spamming tend to be of questionable value

spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● Items/services advertised through spamming tend to be of questionable value ● The vast majority of it is fraudulent

Dealing with spam

● Ignore it

Dealing with spam ● Ignore it... and hope it goes away

Dealing with spam

● Ignore it... not an option

Dealing with spam ● Ignore it... not an option ● Just hit Delete...

Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done

Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done ● Filter it as early as possible in its life cycle

Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done ● Filter it as early as possible in its life cycle ● Filter it as it's trying to enter your machine

If you use Windows...

Mail filtering

| /home/you/mailfilter

Mail filtering apart from spam filtering

● Separating mailing lists into their own folders

Mail filtering apart from spam filtering ● Separating mailing lists into their own folders ● News-to-mail gateways

procmail

● Advantages:

procmail ● Advantages: – Well-established

procmail ● Advantages: – Well-established – Lots of sample scripts

procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages:

procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax

procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax – Like learning a new language...

procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax – Like learning a new language... – And it's not Perl!

Mail::Audit

● Written by Simon Cozens

Mail::Audit ● Written by Simon Cozens procmail is nasty. It has a tortuous and complicated recipe format, and I don't like it. I wanted something flexible whereby I could filter my mail using Perl tests. - Simon Cozens, from the Mail::Audit perldoc

Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen

Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen ● It's Perl!!!!!!!!!!!!!!!

Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen ● It's Perl!!!!!!!!!!!!!!! ● A module, not a standalone program

How Mail::Audit Works

Parsing mail

● Mail::Internet object

Parsing mail ● Mail::Internet object ● Parse by:

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text ● Anything can be parsed

Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text ● Anything can be parsed – Using Mail::Internet::as_string

Installation

● Download and install Mail::Audit from CPAN

Installation # perl -MCPAN -e shell cpan> install Mail::Audit

Installation ● Download and install Mail::Audit from CPAN ● Create.forward file

Installation | /home/creede/mailfilter

Installation ● Download and install Mail::Audit from CPAN ● Create.forward file ● Create filter file

Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit;

Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject;

Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body =

Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body = my $xloop = $mail->get('X-Loop');

Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body = my $xloop = $mail->get('X-Loop'); my $message = $mail->{obj}->as_string;

Installation ● Download and install Mail::Audit from CPAN ● Create.forward file ● Create filter file ● Remember to chmod 0755!

Mail disposition ● $mail->accept – Accepts mail into default inbox

Mail disposition (continued) if ($mail->from =~ { $mail->accept; }

Mail disposition (continued) ● $mail->accept(“/path/to/alternate/mailbox”) – Accepts mail into a non-default mailbox

Mail disposition (continued) my $maildir = “/home/me/mail”; if ($mail->subject =~ /spug/i) { $mail->accept(“$maildir/spug-list”); }

Mail disposition (continued) ● $mail->pipe(“/path/to/external/program”) – Pipes mail through the specified program

Mail disposition (continued) if ($mail->subject =~ /keplerian/i) { $mail->pipe(“/home/creede/parse_kepler”); }

Mail disposition (continued) ● – Sends the mail in its entirety to another address

Mail disposition (continued) if (is_419($message)) { $mail->{noexit} = 1; $mail->put_header('X-Loop', $mail->put_header('To', "$to (forwarded -- no monetary loss -- for your files)"); $mail->{noexit} = 0; $mail->ignore; }

Mail disposition (continued) ● $mail->reject($reason) – Rejects the mail, returning it to the sender with the (optional) reason specified

Mail disposition (continued) if (is_murky($mail)) { $mail->put_header('X-Loop', $mail->reject("I don't like spam."); }

Mail disposition (continued) ● $mail->ignore – Consigns the mail to the bit bucket

Mail disposition (continued) # kill off Korean spam if ($body =~ /ks.c/i) { $mail->ignore; }

Mail::SpamAssassin

● Header analysis

Mail::SpamAssassin ● Header analysis ● Text analysis

Mail::SpamAssassin ● Header analysis ● Text analysis ● Blacklists

Mail::SpamAssassin ● Header analysis ● Text analysis ● Blacklists ● Vipul's Razor

Mail::SpamAssassin – Installation ● Download and install Mail::SpamAssassin from CPAN

Mail::SpamAssassin – Installation # perl -MCPAN -e shell cpan> install Mail::SpamAssassin

Mail::SpamAssassin – Installation #!/usr/bin/perl use Mail::Audit; use Mail::SpamAssassin; my $mail = new Mail::Audit; my $spamtest = new Mail::SpamAssassin; my $status = $spamtest->check($mail); if ($status->is_spam()) { $mail>accept(“/home/you/spamtrap”); }

Mail::SpamAssassin – Configuration ● Load configuration from /etc/mail/spamassasin.conf or /home/you/.spamassassin/user_prefs

Mail::SpamAssassin – Configuration # SpamAssassin user preference file # required_hits4 # #default is 5 # whitelist_from blacklist_from score USER_AGENT_AOL1.00

Paul Graham's Plan for Spam

madam 0.99 promotion 0.99 republic 0.99 shortest mandatory standardization sorry supported

URLs for more information

● Internet Mail

URLs for more information ● Internet Mail ● Mail::Audit

URLs for more information ● Internet Mail ● Mail::Audit ● Mail::SpamAssassin (Outlook)

URLs for more information ● Internet Mail ● Mail::Audit ● Mail::SpamAssassin (Outlook) ● Paul Graham's Plan for Spam

URLs for more information ● Internet Mail ● Mail::Audit ● Mail::SpamAssassin (Outlook) ● Paul Graham's Plan for Spam ● And of course Google.com!

Questions?

Thank you!