Introduction to Unix SMTP & Sendmail

Overview Some SPAM Statistics Introduction to SMTP and Email Message Breakdown Sample Messages Extensions (MIME) MTA’s and Mailbox Protocols

Email Spam Statistics Email considered Spam 40% of all email Daily Spam emails sent 12.4 billion Daily Spam received per person 6 Annual Spam received per person 2,200 Spam cost to all non-corp Internet users $255 million Spam cost to all U.S. Corporations in 2002 $8.9 billion States with Anti-Spam Laws 26 Email address changes due to Spam 16% Estimated Spam increase by 2007 63% Annual Spam in 1,000 employee company 2.1 million Users who reply to Spam email 28% Users who purchased from Spam email 8% Corporate email that is considered Spam 15-20% Wasted corporate time per Spam email 4-5 seconds

SMTP Originated in 1982 (rfc0821, Jon Postel) Goal: To transfer mail reliably and efficiently

SMTP SMTP clients and servers have two main components User Agents – Prepares the message, encloses it in an envelope. (Eudora for example) Mail Transfer Agent (MTA) – Transfers the mail across the internet User Agent Mail Transfer Agents

SMTP SMTP also allows the use of Relays allowing other MTAs to relay the mail Mail Gateways are used to relay mail prepared by a protocol other then SMTP and convert it to SMTP

What is Mail? Mail is a text file Envelope – Message – sender address receiver address other information Message – Mail Header – defines the sender, the receiver, the subject of the message, and some other information Mail Body – Contains the actual information in the message

Post Office Mailbox Return-Path: <Jwatson@cis.udel.edu> Delivered-To: jwatson@cis.udel.edu Received: by mail.eecis.udel.edu (Postfix, from userid 62) id 17FBD328DE; Wed, 5 Nov 2003 11:27:02 Received: from mail.acad.ece.udel.edu (devil-rays.acad.ece.udel.edu [128.4.60.10]) by mail.eecis.udel.edu (Postfix) with ESMTP id 5F41832893 for <Jwatson@cis.udel.edu>; Wed, 5 Nov 2003 11:27:01 Received: by mail.acad.ece.udel.edu (Postfix, from userid 62)id 47509456C; Wed, 5 Nov 2003 11:27:01 Received: from stimpy.eecis.udel.edu(stimpy.eecis.udel.edu [128.4.40.17])by mail.acad.ece.udel.edu (Postfix) with SMTP id 7C2943D79 for <Jwatson@cis.udel.edu>; Wed, 5 Nov 2003 11:26:34 Message-Id: <20031105162634.7C2943D79@mail.acad.ece.udel.edu> Date: Wed, 5 Nov 2003 11:26:34 From: Jwatson@cis.udel.edu To: undisclosed-recipients: ; MIME-Version: 1.0 This is a test. Post office and mail route Receivers Mailbox

How SMTP works The Essentials How about a Demo? Keyword Arguments HELO Sender’s Host Domain Name MAIL FROM: Email Address of sender RCPT TO: Email of Intended recipient DATA Body of the message QUIT

How SMTP works The Extras Keyword Arguments RSET VRFY Name to be verified NOOP TURN EXPN Mailing list to expand HELP Command Name

Status Codes The Server responds with a 3 digit code that may be followed by text info 2## - Success 3## - Command can be accepted with more information 4## - Command was rejected, but error condition is temporary 5## - Command rejected, Bad User!

Connection Establishment TCP Connection Establishment

Message Progress

Connection Termination TCP Connection Termination

Problems with SMTP No inherent security Authentication Encryption Only uses NVT (Network Virtual Terminal) 7-bit ASCII format

E-mails can be forged….. HELO mail.rose.edu MAIL FROM: carberry@rose.edu RCPT TO: wrichards@rose.edu DATA From: Dr. Art Zenner To: Professor Richards Subject: CIT 2243 Professor Richards, By department decree all students in your “Introduction to Unix” class are hereby to be granted automatic A’s. Thank you, Dr. Art Zenner . QUIT

Extensions to SMTP MIME – Multipurpose Internet Mail Extensions Transforms non-ASCII data to NVT (Network Virtual Terminal) ASCII data Text Application Image Audio Video

MIME Headers Goes between the Email Header and Body MIME-Version: 1.1 Content-Type Content-Transfer-Encoding Content-Id Content-Description

MIME Headers Content-Type – Type of data used in the body of the message Text – plain, unformatted text; HTML Multipart – Body contains multiple independent parts Message – The body is whole mail message, part of a message, or a pointer to a message

MIME Headers Image – The message is a stationary image (JPEG or GIF) Video – The message is an animation (Mpeg) Audio – The message is 8 kHz standard audio data Application – The message is a type of data not previously defined

MIME Headers Content-Transfer-Encoding – The method used to encode the messages 7 bit – no encoding needed 8 bit – Non-ASCII, short lines Binary – Non-ASCII, unlimited length lines Base64 – 6 bit blocks encoded into 8-bit ASCII Quoted-printable – send non-ASCII characters as 3 ASCII characters, =##, ## is the hex representation of the byte

Base64 Encoding Divides binary data into 24 bit blocks Each block is then divided into 6 bit chunks Each 6-bit section is interpreted as one character incurs a 25% overhead 11001100 10000001 00111001 110011 001000 000100 111001 (51) (8) (4) (57) (z) (I) (E) (5) 01111010 01001001 01000101 00110101

Quoted-Printable Encoding Used when the data has a small non-ASCII portion Non-ASCII characters are sent as 3 characters First is ‘=‘, second and third are the hex representation of the byte 01001100 10011101 00111001 (=) (9) (D) 00111101 00111001 01000100

MIME Headers Content-Id – Uniquely identifies the whole message in a multiple message environment Content-Description – defines whether the body is image, audio, or video

A Multipart, Encoded MIME Message From: joe_luthier@plucknplay.com To: lchae@mfi.com Subject: Info on Gibson guitar MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=17 - 17 Content-Type: text/enriched; charset="us-ascii" Content-Transfer-Encoding: 8bit Content-Description: Greetings As promised, I'm getting back to you about the Gibson Southern Jumbo guitar you were Interested in. I've enclosed a spec sheet on the guitar, which is in Microsoft Word. I guarantee that you'll love it! - 17 Content-Type: application/octet-stream Content-Transfer-Encoding: base64 Content-Description: Spec sheet saved as MS Word file - 17 -

MIME Example Date: Wed, 04 Apr 2001 00:11:37 -0400 From: Meghna Naik <mnaik@UDel.Edu> MIME-Version: 1.0 To: stoweg@hotmail.com Subject: =?gb2312?B?1tDOxA==?= title Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: 7bit a body text, blah, blah

Mail Transfer Agents (MTA) MTAs do the actual mail transfers MTAs are not meant to be directly accessed by users. Other MTA’s are: Postfix Qmail MS Exchange CC:Mail Lotus Notes ….etc.

O’Reilly’s “Bat Book” 3rd Edition

This is what happens if you try to learn to much about Sendmail…

Sendmail It's been said that you aren't a real Unix system administrator until you've edited a sendmail.cf file. It's also been said that you're crazy if you've done it twice.

What is Sendmail? Sendmail is the most widely used Mail Transport Agent (MTA) on the internet MTAs send mail from one machine to another. Sendmail is not a client program, which you use to read your email. Sendmail is one of the behind-the-scenes programs which move email over the Internet. Normally it runs as a background daemon Can even be run out of the super daemon (xinetd)

Implementations SMTP Gateway An SMTP gateway allows users on your network to communicate with others on the Internet without concern as to which local mail software package exists on your network. All incoming mail for your network will pass through this gateway which converts the message into the appropriate format specific to your local mail software. Similarly, all mail destined for the Internet from your network will pass through this gateway to be sent across the Internet via SMTP

Implemetations SMTP Relay ”Warning Will Rogers” An SMTP relay is a machine that actually sends the mail across the Internet. A common misconception is that SMTP gateways are the same as SMTP relays. This is not always the case. There are SMTP gateways that act as relays themselves, but there are also many that do not. If the latter is the case on your network, you'll need to bounce your mail off one of the relays.

The Pieces The binary: The configuration file: Supporting files: /sbin/sendmail The configuration file: /etc/mail/sendmail.cf Supporting files: /etc/mail/access /etc/mail/aliases …and many more

More Pieces Email messages are stored in the directory: /var/spool/mail There is a separate file for each user Email waiting to be sent ./var/spool/mqueue A log of Email sent and received: /var/log/mail

Sendmail Features Sendmail uses DNS (Domain Naming System) But not 100% dependent: [Joe@[192.168.1.1] DNS provides Mail Exchange (MX) Info Sendmail can do a DNS double-tap Lookup up who the client says they are Sendmail default is mail relay off Realtime Blackhole Lists (RBL) Mail Relay checkers - - Open Mail Relay Db http://www.ordb.org/submit/

Sendmail Anti-Spam Enhancements Mailscanner Minimal anti-spam Anti-virus integration (scan in/outbound) http://www.sng.ecs.soton.ac.uk/mailscanner/ Or http://www.mailscanner.info Spam Assassin Rule based heuristic Header and text analysis Blacklist (RBL) Vipul's Razor (http://razor.sf.net) http://www.spamassassin.org

Mail Access Protocols The MTAs place the email in the user’s mailbox The Mail Access Protocols are used by the users to retrieve the email from the mailbox POP3 (Post Office Protocol) IMAP (Internet Message Access Protocol )

POP vs. IMAP POP3: IMAP: …. Whole message headers All Messages Mr Smith Friends …. headers

Post Office Protocol v3 Simple Allows the user to obtain a list of their Emails Users can retrieve their emails Users can either delete or keep the email on their system Minimizes server resources http://www.washington.edu/computing/windows/issue13/imap_pop.html

Internet Mail Access Protocol v4 Has more features then POP3 User can check the email header before downloading Emails can be accessed from any location Can search the email for a specific string of characters before downloading User can download parts of an email User can create, delete, or rename mailboxes on a server

References RFC’s: E-mail Explained RFC 821 - Simple Mail Transfer Protocol RFC0822 - Standard for the Format of ARPA Internet Text Messages RFC 1521 - MIME (Multipurpose Internet Mail Extensions) E-mail Explained http://www.sendmail.org/email-explained.html