Electronic Mail

Most heavily used application on any network Uses client-server architecture –Electronic mail client accepts mail from user and delivers to server on destination computer –Many variations and styles of delivery Simple Mail Transfer Protocol (SMTP) –TCP/IP –Delivery of simple text messages Multi-purpose Internet Mail Extension (MIME) –Delivery of other types of data –Voice, images, video clips

SMTP Concept

User Agents and Mail Transfer Agents

Relay MTAs

Mail Gateways

The Entire System

SMTP Simple Mail Transfer Protocol is standard application protocol for delivery of mail from source to destination Built on top of TCP: provides reliable delivery of data SMTP does not normally use intermediate mail servers for sending mail SMTP uses persistent connections Other functions: – address lookup – address verification Example protocol exchange

Sample SMTP interaction C: c:\Telnet hamburger.edu 25 S: 220 hamburger.edu C: HELO ce.sharif.edu S: 250 Hello ce.sharif.edu, pleased to meet you C: MAIL FROM: S: 250 Sender ok C: RCPT TO: S: 250 Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C:. S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection

SMTP RFC 821 Not concerned with format of messages or data –Covered in RFC 822 (see later) SMTP uses info written on envelope of mail –Message header Does not look at contents –Message body Except: –Standardize message character set to 7 bit ASCII

Basic Operation Mail created by user agent program (mail client) –Message consists of: Header containing recipient’s address and other info Body containing user data Messages queued and sent as input to SMTP sender program –Typically a server process (daemon on UNIX)

Mail Message Contents Each queued message has: –Message text RFC 822 header with message envelope and list of recipients Message body, composed by user –A list of mail destinations Derived by user agent from header May be listed in header May require expansion of mailing lists May need replacement of mnemonic names with mailbox names If BCCs indicated, user agent needs to prepare correct message format

SMTP Sender Takes message from queue Transmits to proper destination host –Via SMTP transaction –Over one or more TCP connections to port 25 Host may have multiple senders active When delivery complete, sender deletes destination from list for that message When all destinations processed, message is deleted

Optimization If message destined for multiple users on a given host, it is sent only once –Delivery to users handled at destination host If multiple messages ready for given host, a single TCP connection can be used –Saves overhead of setting up and dropping connection

Possible Errors Host unreachable Host out of operation TCP connection fail during transfer Sender can re-queue mail –Give up after a period Faulty destination address –User error –Target user changed address –Redirect if possible –Inform user if not

SMTP Protocol - Reliability Used to transfer messages from sender to receiver over TCP connection Attempts to provide reliable service No guarantee to recover lost messages No end to end acknowledgement to originator Error indication delivery not guaranteed Generally considered reliable

SMTP Receiver Accepts arriving message Places in user mailbox or copies to outgoing queue for forwarding Receiver must: –Verify local mail destinations –Deal with errors Transmission Lack of disk space Sender responsible for message until receiver confirm complete transfer –Indicates mail has arrived at host, not user

SMTP Forwarding Mostly direct transfer from sender host to receiver host May go through intermediate machine via forwarding capability –Sender can specify route –Target user may have moved

Conversation SMTP limited to conversation between sender and receiver Main function is to transfer messages Rest of mail handling beyond scope of SMTP –May differ between systems

Figure 3.9 SMTP Mail Flow

SMTP System Overview Commands and responses between sender and receiver Initiative with sender –Establishes TCP connection Sender sends commands to receiver e.g. HELO Each command generates exactly one reply e.g. 250 requested mail action ok; completed

SMTP Replies Leading digit indicates category –Positive completion reply (2xx) –Positive intermediate reply (3xx) –Transient negative completion reply (4xx) –Permanent negative completion reply (5xx)

Operation Phases Connection setup Exchange of command-response pairs Connection termination

Connection Setup Sender opens TCP connection with receiver Once connected, receiver identifies itself –220 service ready Sender identifies itself –HELO Receiver accepts sender’s identification –250 OK If mail service not available, step 2 above becomes: –421 service not available

Mail Transfer Sender may send one or more messages to receiver MAIL command identifies originator –Gives reverse path to used for error reporting –Receiver returns 250 OK or appropriate fail/error message One or more RCPT commands identifies recipients for the message –Separate reply for each recipient DATA command transfers message text –End of message indicated by line containing just period (.)

Closing Connection Two steps Sender sends QUIT and waits for reply Then initiate TCP close operation Receiver initiates TCP close after sending reply to QUIT

Electronic Mailboxes and Addresses users have an electronic mailbox into which mail is deposited Usually associated with computer account; one user may have different electronic mailboxes User accesses mail with a mail reader program Electronic mailbox is identified by an address

Internet Mail Addressing User portion or mailbox portion is site specific –Joe –Seyyed_hamid_1221 – Host portion or computer portion is domain name Source mail client –Resolves destination name using DNS –Contacts mail delivery server at destination –Copies mail to server Destination mail server –Interprets user name according to local mailbox addresses –Places mail in appropriate mailbox

message format Simple two-part format –Header includes delivery information –Body carries text of message Header and body separated by blank line

Headers Lines of text in format keyword: information keyword identifies information; information can appear in any order Essential information: –To: list of recipients –From: sender –Cc: list of copy recipients Useful information –Reply-to: different address than From: –Received-by: for debugging Frivolous information: –Favorite drink: lemonade –Phase-of-the-moon: gibbous

Data in Original Internet mail carried only 7-bit ASCII data Couldn’t contain arbitrary binary values; e.g., executable program Techniques for encoding binary data allowed transport of binary data Uuencode: three 8-bit binary values as four ASCII characters (6 bits each) –Also carries file name and protection information –Incurs 33% overhead –Requires manual intervention

Multipurpose Internet Mail Extension (MIME) Extension to RFC822 SMTP can not transmit executables –Uuencode and other schemes are available Not standardized Can not transmit text including international characters (e.g. â, å, ä, è, é, ê, ë) –Need 8 bit ASCII Servers may reject mail over certain size Translation between ASCII and EBCDIC not standard SMTP gateways to X.400 can not handle none text data in X.400 messages Some SMTP implementations do not adhere to standard –CRLF, truncate or wrap long lines, removal of white space, etc.

MIME Multipart Internet Mail Extensions - Extends and automates encoding mechanisms Allows inclusion of separate components – programs, pictures, audio clips – in a single mail message Sending program identifies the components so receiving program can automatically extract and inform mail recipient –Header includes MIME-Version: 1.0 Content-Type: Multipart/Mixed; Boundary=Mime_separator –Separator line gives information about specific encoding Content-Type: text/plain

MIME (cont.) MIME is extensible – sender and receiver agree on encoding scheme MIME is compatible with existing mail systems –Everything encoded as ASCII –Headers and separators ignored by non-MIME mail systems MIME encapsulates binary data in ASCII mail envelope

MIME

Overview of MIME Five new message header fields –MIME version –Content type –Content transfer encoding –Content Id –Content Description Number of content formats defines Transfer encoding defined

Content Types Text body Multipart –Mixed, Parallel, Alternative, Digest Message –RFC 822, Partial, External-body Image –jpeg, gif Video –mpeg Audio –Basic Application –Postscript –octet stream

MIME Transfer Encodings Reliable delivery across wide largest range of environments Content transfer encoding field –Six values –Three (7bit, 8bit, binary) no encoding done Provide info about nature of data Quoted-printable –Data largely printable ASCII characters –Non-printing characters represented by hex code Base64 –Maps arbitrary binary input onto printable output X-token –Named nonstandard encoding

Figure 3.10 Printable Encoding of Binary Data into Radix-64 Format

Programs as mail recipients Can arrange for box to be associated with a program rather than a user’s mail reader Incoming mail automatically processed as input to program Example – mailing list subscription administration Can be used to implement client-server processing –Client request in incoming mail message –Server response in returned mail reply

Mailing Lists and Forwarders addresses can be attached to programs as well as electronic mailboxes Mail exploder or mail forwarder resends copies of message to addresses in mailing list –UNIX mail program sendmail provides mail aliases –Mailing list processor, e.g. listserv, can also interpret subscription management commands

Mail Gateways Mailing list processing may take significant resources in large organizations May be segregated to a dedicated server computer: mail gateway –Provides single mail destination point for all incoming mail –Can use MX records in DNS to cause all mail to be delivered to gateway

Mail gateways and forwarding Users within an organization may want to read mail on local or departmental computer Can arrange to have mail forwarded from mail gateway Message now makes multiple hops for delivery Hops may be recorded in header Forwarded mail may use proprietary (non-SMTP) mail system

Mail Gateways and Addresses Organization may want to use uniform naming for external mail Internally, may be delivered to many different systems with different naming conventions Mail gateways can translate addresses

Mailbox Access Where should mailbox be located? Users want to access mail from most commonly used computer Cannot always use desktop computer as mail server –Not always running –Requires multitasking OS –Requires local disk storage Can TELNET to remote computer with mail server

Mail Access Protocols Typically users do NOT run a server on their local PC User agent accesses a mailbox from a shared mail server Mail access protocol transfers mail from server to local PC 2 popular protocols: –POP3 (Post Office Protocol v3) –IMAP (Internet Mail Access Protocol) Can’t use SMTP because getting messages is a pull operation & SMTP is a push protocol.

Mail Access Protocol Instead of TELNEt, use protocol that accesses mail on remote computer directly TCP/IP suite includes Post Office Protocol (POP) for remote mailbox access –Computer with mailboxes runs POP server –User runs POP client on local computer –POP client can access and retrieve messages from mailbox –Requires authentication (password) –Local computer uses SMTP for outgoing mail

Protocols

POP3 and SMTP

POP3 Three phases –Authorization –Transaction –Update During a POP3 session, the server maintains state information POP3 downloads mail messages to the local machine – not good for “nomadic user”

Dialup Access and POP POP useful for dialup connection –User’s computer not always connected –Can download all mail at once and read off-line –Can compose mail off-line and mail in one connection

POP3 protocol authorization phase client commands: –user: declare username –pass: password server responses –+OK –-ERR transaction phase, client: list: list message numbers retr: retrieve message by number dele: delete quit C: list S: S: S:. C: retr 1 S: S:. C: dele 1 C: retr 2 S: S:. C: dele 2 C: quit S: +OK POP3 server signing off S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on

POP3 Stands for Post Office Protocol – Version 3 RFC in 1993 POP3 “is intended to permit a workstation to dynamically access a maildrop on a server host in a useful fashion. Usually, this means that the POP3 protocol is used to allow a workstation to retrieve mail that the server is holding for it.“

POP3 Doesn’t provide extensive mail operations on the server Normally mail is downloaded, then deleted Deletion doesn’t really occur until QUIT Uses port 110

Minimal POP3 Commands USER name – user’s name PASS string – user’s password QUIT – quits, and updates the mailbox STAT – status LIST [msg] – lists the mail messages (#’s only) RETR msg – retrieves a mail message DELE msg – deletes a message NOOP – No operation RSET – resets the mail to its original state

Example POP3 Transaction Server

Establish the Socket Server Client Socket (110) Primary Thread Primary Thread

Server is OK Server Client Socket (110) Primary Thread Primary Thread +OK POP3 server ready

Establish the Socket Server Client Socket (110) Primary Thread Primary Thread USER bob

This user is OK Server Client Socket (110) Primary Thread Primary Thread +OK

Send the Password Server Client Socket (110) Primary Thread Primary Thread PASS ITSD4304

The Password is OK Server Client Socket (110) Primary Thread Primary Thread +OK

Ask how the Server is Feeling… Server Client Socket (110) Primary Thread Primary Thread STAT

I’m Fine, Thanks! Server Client Socket (110) Primary Thread Primary Thread +OK 2 320

Got Any Mail? Server Client Socket (110) Primary Thread Primary Thread LIST

Yup… Server Client Socket (110) Primary Thread Primary Thread +OK 2 messages (320 octets)

Message One Server Client Socket (110) Primary Thread Primary Thread 1 120

Message Two Server Client Socket (110) Primary Thread Primary Thread 2 200

End Server Client Socket (110) Primary Thread Primary Thread.

Get Message One Server Client Socket (110) Primary Thread Primary Thread RETR 1

Message One Coming! Server Client Socket (110) Primary Thread Primary Thread +OK 120 octets

Message One Coming! Server Client Socket (110) Primary Thread Primary Thread

End of Message 1 Server Client Socket (110) Primary Thread Primary Thread.

Get Message One Server Client Socket (110) Primary Thread Primary Thread DELE 1

Message 1 Gone! Server Client Socket (110) Primary Thread Primary Thread +OK message 1 deleted Note: it’s not really gone…

Get Message One Server Client Socket (110) Primary Thread Primary Thread QUIT

QUIT – Commits/Updates Server Client Socket (110) Primary Thread Primary Thread +OK bob POP3 server signing off Note: now it’s gone

Socket Disposed Server Client Primary Thread Primary Thread

IMAP Internet Mail Access Protocol Allows users to manipulate remote mailboxes Must maintain info about user (e.g., folder hierarchy) even after the user quits IMAP server is always in 1 of 4 states: –Nonauthenticated –Authenticated –Selected –Logout

POP3 (more) and IMAP More about POP3 Previous example uses “download and delete” mode. Bob cannot re-read if he changes client “Download-and-keep”: copies of messages on different clients POP3 is stateless across sessions IMAP Keep all messages in one place: the server Allows user to organize messages in folders IMAP keeps user state across sessions: –names of folders and mappings between message IDs and folder name

Security SMTP, POP3 and IMAP are plain text protocols Secured version available to protect the user –SMTPS –POP3s –Need to be configured both on server and client

Stunnel Name of an application/service –Even if the mail server application does not implement secure version What does it do? –gathers the encrypted message from client –Decrypts the data –Sends plain text data using a local connection to insecure port Server application considers it a normal client

STunneling Server Client Stunnel (on server) Listening to port 995 Send decrypted data to port 25 of the server Encrypted data (Command/mail body)

Web interface Nowadays there are many web interface user agents They allow accessing messages from any client with only a web browser However the interface is not as sophisticated as special mail agents But it is very useful if you need to check your s from different computers

Summary for Electronic mail allows quick, asynchronous communication across entire Internet Can attach addresses to programs for processing –Mailing lists –Other client-server applications Simple Mail Transfer Protocol (SMTP) is Internet standard for mail delivery Mail gateways –Provide uniform user addressing outside organizations –Translate from Internet mail (e.g. SMTP) to proprietary systems Post Office Protocol (POP) and Internet Mail Access Protocol (IMAP) allow remote access to electronic mailboxes.