1. Filtering Mail with Mail::Audit and Mail::SpamAssassin Creede Lambard penguinsinthenight.com 20 August 2002

2. General Outline:

3. ● How UNIX handles mail

4. General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial email

5. General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thing tm

6. General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thing tm ● If you use Windows...

7. General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thing tm ● If you use Windows... ● Using Mail::Audit

8. General Outline: ● How UNIX handles mail ● A simple understated diatribe against unsolicited commercial email ● Why mail filtering is a Good Thing tm ● If you use Windows... ● Using Mail::Audit ● Using Mail::SpamAssassin

9. How Unix handles your mail

19. .forward to another mail address: me@myotherisp.com

20. How Unix handles your mail Piping to another program: | vacation

21. Does this look familiar?

22. spam

23. ● Unsolicited commercial email

24. spam ● Unsolicited commercial email – Sent in bulk

25. spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service

26. spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient

27. spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient ● Not necessarily mail you don't want...

28. spam ● Unsolicited commercial email – Sent in bulk – Directly or indirectly advertises a product or service – Not requested by recipient ● Not necessarily mail you don't want... – Although for purposes of this presentation we'll treat them the same.

29. When Spam tm is acceptable

31. spam is a Bad Thing tm

32. ● It shifts the burden of costs to the recipient

33. spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net

34. spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time

35. spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● Items/services advertised through spamming tend to be of questionable value

36. spam is a Bad Thing tm ● It shifts the burden of costs to the recipient ● It clogs the Net ● It wastes your time ● Items/services advertised through spamming tend to be of questionable value ● The vast majority of it is fraudulent

37. Dealing with spam

38. ● Ignore it

39. Dealing with spam ● Ignore it... and hope it goes away

40. Dealing with spam

41. ● Ignore it... not an option

42. Dealing with spam ● Ignore it... not an option ● Just hit Delete...

43. Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done

44. Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done ● Filter it as early as possible in its life cycle

45. Dealing with spam ● Ignore it... not an option ● Just hit Delete... The damage is already done ● Filter it as early as possible in its life cycle ● Filter it as it's trying to enter your machine

46. If you use Windows...

47. Mail filtering

48. | /home/you/mailfilter

49. Mail filtering apart from spam filtering

50. ● Separating mailing lists into their own folders

51. Mail filtering apart from spam filtering ● Separating mailing lists into their own folders ● News-to-mail gateways

52. procmail

53. ● Advantages:

54. procmail ● Advantages: – Well-established

55. procmail ● Advantages: – Well-established – Lots of sample scripts

56. procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages:

57. procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax

58. procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax – Like learning a new language...

59. procmail ● Advantages: – Well-established – Lots of sample scripts ● Disadvantages: – Arcane syntax – Like learning a new language... – And it's not Perl!

60. Mail::Audit

61. ● Written by Simon Cozens

62. Mail::Audit ● Written by Simon Cozens procmail is nasty. It has a tortuous and complicated recipe format, and I don't like it. I wanted something flexible whereby I could filter my mail using Perl tests. - Simon Cozens, from the Mail::Audit perldoc

63. Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen

64. Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen ● It's Perl!!!!!!!!!!!!!!!

65. Mail::Audit ● Written by Simon Cozens ● Based on audit_mail and deliverlib by Tom Christiansen ● It's Perl!!!!!!!!!!!!!!! ● A module, not a standalone program

66. How Mail::Audit Works

67. Parsing mail

68. ● Mail::Internet object

69. Parsing mail ● Mail::Internet object ● Parse by:

70. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines

71. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject

72. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers

73. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text

74. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text ● Anything can be parsed

75. Parsing mail ● Mail::Internet object ● Parse by: – From, To or CC lines – Subject – Absence, presence or content of headers – Body text ● Anything can be parsed – Using Mail::Internet::as_string

76. Installation

77. ● Download and install Mail::Audit from CPAN

78. Installation # perl -MCPAN -e shell cpan> install Mail::Audit

79. Installation ● Download and install Mail::Audit from CPAN ● Create.forward file

80. Installation | /home/creede/mailfilter

81. Installation ● Download and install Mail::Audit from CPAN ● Create.forward file ● Create filter file

82. Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit;

83. Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject;

84. Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body = join(“\n”, @$body);

85. Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body = join(“\n”, @$body); my $xloop = $mail->get('X-Loop');

86. Installation #!/usr/bin/perl use Mail::Audit; my $mail = new Mail::Audit; my $from = $mail->from; my $to = $mail->to; my $cc = $mail->cc; my $subject = $mail->subject; my $_body = $mail->body; my $body = join(“\n”, @$body); my $xloop = $mail->get('X-Loop'); my $message = $mail->{obj}->as_string;

87. Installation ● Download and install Mail::Audit from CPAN ● Create.forward file ● Create filter file ● Remember to chmod 0755!

88. Mail disposition ● $mail->accept – Accepts mail into default inbox

89. Mail disposition (continued) if ($mail->from =~ /mom@applepie.com/) { $mail->accept; }

90. Mail disposition (continued) ● $mail->accept(“/path/to/alternate/mailbox”) – Accepts mail into a non-default mailbox

91. Mail disposition (continued) my $maildir = “/home/me/mail”; if ($mail->subject =~ /spug/i) { $mail->accept(“$maildir/spug-list”); }

92. Mail disposition (continued) ● $mail->pipe(“/path/to/external/program”) – Pipes mail through the specified program

93. Mail disposition (continued) if ($mail->subject =~ /keplerian/i) { $mail->pipe(“/home/creede/parse_kepler”); }

94. Mail disposition (continued) ● $mail->resend(“someguy\@otherisp.com”) – Sends the mail in its entirety to another address

95. Mail disposition (continued) if (is_419($message)) { $mail->{noexit} = 1; $mail->put_header('X-Loop', 'creede@penguinsinthenight.com'); $mail->put_header('To', "$to (forwarded -- no monetary loss -- for your files)"); $mail->resend("uce\@ftc.gov"); $mail->resend("419.fcd\@usss.treas.gov"); $mail->{noexit} = 0; $mail->ignore; }

96. Mail disposition (continued) ● $mail->reject($reason) – Rejects the mail, returning it to the sender with the (optional) reason specified

97. Mail disposition (continued) if (is_murky($mail)) { $mail->put_header('X-Loop', 'creede@penguinsinthenight.com'); $mail->reject("I don't like spam."); }

98. Mail disposition (continued) ● $mail->ignore – Consigns the mail to the bit bucket

99. Mail disposition (continued) # kill off Korean spam if ($body =~ /ks.c/i) { $mail->ignore; }

100. Mail::SpamAssassin

101. ● Header analysis

102. Mail::SpamAssassin ● Header analysis ● Text analysis

103. Mail::SpamAssassin ● Header analysis ● Text analysis ● Blacklists

104. Mail::SpamAssassin ● Header analysis ● Text analysis ● Blacklists ● Vipul's Razor

105. Mail::SpamAssassin – Installation ● Download and install Mail::SpamAssassin from CPAN

106. Mail::SpamAssassin – Installation # perl -MCPAN -e shell cpan> install Mail::SpamAssassin

107. Mail::SpamAssassin – Installation #!/usr/bin/perl use Mail::Audit; use Mail::SpamAssassin; my $mail = new Mail::Audit; my $spamtest = new Mail::SpamAssassin; my $status = $spamtest->check($mail); if ($status->is_spam()) { $mail>accept(“/home/you/spamtrap”); }

108. Mail::SpamAssassin – Configuration ● Load configuration from /etc/mail/spamassasin.conf or /home/you/.spamassassin/user_prefs

109. Mail::SpamAssassin – Configuration # SpamAssassin user preference file # required_hits4 # #default is 5 # whitelist_from mom@applepie.com blacklist_from scuzzball@spamspewer.com score USER_AGENT_AOL1.00

110. Paul Graham's Plan for Spam

111. madam 0.99 promotion 0.99 republic 0.99 shortest 0.047225013 mandatory 0.047225013 standardization 0.07347802 2600 0.0813768 sorry 0.08221981 supported 0.09019077

112. URLs for more information

113. ● Internet Mail http://www.imc.org/rfcs.html

114. URLs for more information ● Internet Mail http://www.imc.org/rfcs.html ● Mail::Audit http://simon-cozens.org/writings/mail-audit.html

115. URLs for more information ● Internet Mail http://www.imc.org/rfcs.html ● Mail::Audit http://simon-cozens.org/writings/mail-audit.html ● Mail::SpamAssassin http://www.spamassassin.org/ http://www.deersoft.com (Outlook)

116. URLs for more information ● Internet Mail http://www.imc.org/rfcs.html ● Mail::Audit http://simon-cozens.org/writings/mail-audit.html ● Mail::SpamAssassin http://www.spamassassin.org/ http://www.deersoft.com (Outlook) ● Paul Graham's Plan for Spam http://www.paulgraham.com/spam.html

117. URLs for more information ● Internet Mail http://www.imc.org/rfcs.html ● Mail::Audit http://simon-cozens.org/writings/mail-audit.html ● Mail::SpamAssassin http://www.spamassassin.org/ http://www.deersoft.com (Outlook) ● Paul Graham's Plan for Spam http://www.paulgraham.com/spam.html ● And of course Google.com!

118. Questions?

119. Thank you! creede@penguinsinthenight.com http://www.penguinsinthenight.com/spamtalk