McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Chapter 26 SMTP and FTP

McGraw-Hill©The McGraw-Hill Companies, Inc., is used for sending a single message that includes text, voice, video, or graphics to one or more recipients. Simple Mail Transfer Protocol (SMTP). Sending Mail User creates mail that looks very similar to postal mail. Envelope: sender and receiver address Message: Header: Define the sender, receiver, subject of the message,... Body:contains the actual information to be read by the recipient. Receiving mail system periodically checks the mailboxes. If a user has mail, it informs the user with a notice If the user is ready to read the mail, a list is displayed in which each line contains a summary of the information about a particular message in the mailbox. Summary contains the sender mail address, the subject, and time the mail was sent or received.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.1 Format of an

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure address Mail handling system must use an addressing system with unique addresses. Local part: Name of a special file, called the user mailbox, where all the mail received for a user is stored for retrieval by the user agent. Domain Name: Organization usually selects one or more hosts to receive and send ; they are sometimes called mail exchangers. Domain name assigned to each mail exchanger either comes from the DNS database or is a logical name (e.g., the name of the organization).

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.3 User agent First component of an electronic mail system is the user agent (UA); Mail Reader. Service provided by a user agent User agent is a software package (program) that composes, reads, replies to, and forward messages. It also handles mailboxes. Composing messages: Responsible for composing the msg to be sent out. Some of them provide a template and some provide with built- in editor; OR else user can type in an editor and paste into the user agent.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Reading messages User agent is used to read incoming messages. When a user invokes a user agent, it first checks the mail in the incoming mailbox. A one-line summary [number, flag field indicating if the mail is new OR already read but not replied to OR read and replied to OR …, size of message, sender, subject if subject line in msg is not empty] of each received mail is shown. Replying to messages After the reading a message, user can use the user agent to reply to a message. User agent allows the user to reply to the original sender or to reply to all recipients of the message. Reply contains the original message (for quick reference) and new message. Forwarding messages: Can forward with or without extra comments. Handling Mailboxes: Inbox: Keeps all received s until they are deleted by the user. Outbox: Keeps all sent s until the user deletes them.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 User Agent Types Command-Driven Normally accepts a one-character command from the keyboard to perform its task. mail, pine, elm, … GUI-Based Contain GUI components that allow the user to interact with the software by using both the keyboard and the mouse. Eudora, Microsoft’s Outlook, Netscape,… SMTP is a simple mail transfer protocol Can send messages only in 7-bit ASCII format. Cannot be used to send binary files or video or audio. Cannot support few languages that are not supported by ASCII

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.4 MIME Multipurpose Internet Mail Extensions (MIME) Supplementary protocol that allows non-ASCII data to be sent through SMTP. Only an extension to SMTP. MIME transforms non-ASCII data at the sender side to ASCII data and delivers them to client SMTP to sent through the Internet. Reverse is done at the receiver side.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.5 MIME header MIME defines five headers that can be added to original SMTP header: MIME-Version Content-Type: Type of data used in the body; has subtypes separated with slashes. Content-Transfer-Encoding Content-Id: Uniquely identifies the whole message in a multiple- message environment. Content-Description: Whether the body is image, audio, or video.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Content Types Text 7-bit ASCII; no transformation by MIME Multipart Multiple, independent parts. Needs to define the boundary between each part, as a parameter. Boundary is terminated using two hyphens. Mixed: Present the parts in exact order as they are in message. Parallel: Order of parts is unimportant. Digest: Same as mixed but default type/subtype is message/RFC822 Alternative: Same message is repeated using different formats.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Message: Body is itself a whole mail message, a part of a mail message, or a pointer to a message. Subtypes: RFC822: Used if the body is encapsulating another message (including header and the body). Partial: Used if the original message has been fragmented into different mail messages and this message is one of the fragments. Fragments must be reassembled at the destination by MIME. Three parameters must be added: id, number, total Id: identifies the message and is present in all the fragments. Number: sequence order of the fragment Total: number of fragments that comprise the original message External-body. Body does not contain the actual message but is only a reference (pointer) to the original message.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Image Original message is a stationary image, indicating that there is no animation. Joint Photographic Experts Group (JPEG) Graphics Interchange Format (GIF) Video Original message is a time-varying image (animation). Motion Picture Experts Group (MPEG) If the animated image contains sounds, it must be sent separately using the audio content type. Audio Original message is a sound. Only subtype is basic, which uses 8-KHz standard audio data. Application Original message is a type of data not previously defined. Octet-stream is used when the data must be interpreted as a sequence of 8-bit bytes (binary file). PostScript is used when the data are in Adobe PostScript format.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Table 26.1Data types and subtypes in MIME TypeSubtypeDescription TextPlainUnformatted text; 7-bit ASCII Multipart MixedBody contains ordered parts of different data types ParallelSame as above, but no order DigestSimilar to mixed, but the default is message/RFC822 AlternativeParts are different versions of the same message Message RFC822Body is an encapsulated message PartialBody is a fragment of a bigger message Ext. BodyBody is a reference to another message Image JPEGImage is in JPEG GIFVideo is in GIF format VideoMPEGVideo is in MPEG format AudioBasicSingle-channel encoding of voice at 8 KHz Application PostScriptAdobe PostScript Octet-StreamGeneral binary data (8-bit bytes)

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Content-Transfer-Encoding Method to encode the messages into 0s and 1s for transport. 7bit: 7-bit ASCII encoding Length of line should not exceed 1000 characters 8bit: [NOT Recommended] 8-bit encoding; can send non-ASCII characters. Length of line should not exceed 1000 characters MIME does not do any encoding here SMTP must be able to transfer 8-bit non-ASCII characters. Binary: [NOT Recommended] 8-bit encoding Non-ASCII characters can be sent, and the length of the line can exceed 1000 characters. MIME does not do any encoding here: SMTP must be able to transfer binary data. Base64: Solution for sending data made of bytes when the highest bit is not necessarily zero. Base64 transforms this type of data to printable characters, which can then be sent as ASCII characters or any type of character set supported by underlying mail transfer mechanism.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Table 26.2 Content-transfer encoding CategoryDescription Type ASCII characters and short lines 7bitNon-ASCII characters and short lines 8bitNon-ASCII characters with unlimited-length lines Binary6-bit blocks of data are encoded into 8-bit ASCII characters Base64 Non-ASCII characters are encoded as an equal sign followed by an ASCII code

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure Base64

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Table 26.3 Base64 encoding table ValueCodeValueCodeValueCodeValueCodeValueCodeValueCode 0A11L22W33h44s B12M23X34i45t C13N24Y35j46u D14O25Z36k47v E15P26a37l48w F16Q27b38m49x G17R28c39n50y H18S29d40o51z I19T30e41p / 9J20U31f42q531 10K21V32g43r542

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.7 Quoted-printable Quoted-Printable Base64 is a redundant encoding scheme; that is, 24 bits becomes four characters and eventually is sent as 32 bits. If the data consist mostly of ASCII characters with a small non-ASCII portion, we can use quoted-printable encoding. If a character is: ASCII, it is sent as it is. Non-ASCII, sent as three characters. First character is the equals sign (=). Next two characters are the hex of the byte.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.8 MTA client and server A mail transfer agent or MTA (also called a mail server, or a mail exchange server is a computer program or software agent which transfers electronic mail messages from one computer to another. The MTA works behind the scenes, while the user usually interacts with another program, the mail user agent (MUA), which contacts an MTA for actual delivery of the mail.computer programagentelectronic mail user agent

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Typical MTAs:  Sendmail  Postfix  Exim  QMail

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure 26.9 Commands and responses SMTP uses commands and responses to transfer messages between an MTA client and an MTA server. Each command or reply is terminated by a two-character (carriage return and line feed) end-of-line token. Commands Commands are sent from the client to the server. Consists of a keyword followed by zero and more arguments. SMTP defines 14 commands. Responses Responses are sent from server to client. Response is a three-digit code that may be followed by additional textual information.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Mail Transfer Phases Process of transferring a mail message occurs in three phases Connection establishment After a client has made a TCP connection to the well-known port 25, the SMTP server starts the connection phase. Message transfer After connection establishment, a single message between a sender and one or more recipients can be exchanged. Connection termination After the message is transferred successfully, the client terminates the connection.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Mail Delivery Delivery of from sender to receiver consists of three stages 1 st stage goes from user agent to local server Mail does not go directly to remote server because the remote server may not be available at all times. Mail is stored in the local server until it can be sent. 2 nd stage is relayed by local server, which now acts as the SMTP client, to the remote server, which is the SMTP server. is delivered to remote server and not to the remote user agent; is stored in mailbox of the user for later retrieval.. server runs the SMTP server program. 3 rd stage Remote user agent uses a mail access protocol such as POP3 or IMAP4 to access the mailbox and obtain the mail.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure delivery

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Mail access protocols SMTP is a push protocol; it pushes the message from the sender to the receiver even if the receiver does not want it. 3 rd stage needs a pull protocol; the operation must start with the recipient. Mail must stay in the mail server mailbox until the recipient retrieves it. POP3 [Post Office Protocol] Client POP3 software is installed on the recipient computer. Server POP3 software is installed on the mail server. Client (user agent) opens a connection with the server on TCP port 110. It then sends its user name and password to access the mailbox. User can then list and receive mail messages, one by one.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 POP3 modes Delete mode: Mail is deleted from the mailbox after each retrieval. Keep mode: Mail remains in the mailbox after retrieval. IMAP4 (Internet Mail Access Protocol) POP3 assumes that each time a client accesses the server, the whole mailbox will be cleared out (transferred). POP3 does not allow user to organize mail on the server; user cannot have different folders on the server; POP3 does not allow the user to partially check the contents of the mail before downloading. User can check the header prior to downloading User can search the contents of for a specific string of characters prior to downloading User can partially download . User can create, delete, or rename mailboxes on mail server User can create a hierarchy of mailboxes in a folder for storage. Web-Based Hotmail or Yahoo Mail transfer from client browser to mail server is via HTTP SMTP is used for mail transfer from sending mail server to receiving mail server. Message from receiving mail server to browser is via HTTP

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure POP3

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 File Transfer Protocol Copying a file from one host to another Problems in FTP Two systems may use different file name conventions Two systems may have different ways to represent text and data. Two systems may have different directory structures. FTP establishes two connections between client and server. One connection is used for data transfer [Port 20] Other connection for control information (commands and responses) [Port 21] Separation of commands and data transfer makes FTP more efficient. Commands are very simple.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure FTP Client has three components User interface, Client control process, client data transfer process. Server has two components Server control process, server data transfer process Control connection is made between control processes; Control connection is maintained during entire interactive FTP session. Data connection is made between data transfer processes; Data connection is opened & then closed for each file transferred.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure Using the control connection Control Connection Created in same way as other application programs. Connection remains open during the entire process. Service type used by the IP is minimize delay, because this is an interactive connection between a user (human) and a server. User types commands and expects to receive responses without significant delay. Data connection Opened when data are ready to transfer. Closed when it is not needed. Service type used by IP is maximize throughput.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure Using the data connection Communication over control connection Uses ASCII character set. Each command or response is only one short line, so we need not worry about file format or file structure. Each line is terminated with a two-character (carriage return and line feed) end-of-line token. Communication over data connection Client must define the type to be transferred, the structure of the data, and the transmission mode. Heterogeneity is solved by: file type, data structure, transmission mode.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 File type ASCII file: By default for transferring text files. Sender transforms the file from its own representation to ASCII characters Receiver transforms the ASCII characters to its own representation. EBCDIC file If one or both ends of connection use EBCDIC encoding (used in IBM computers) the file can be transferred using EBCDIC encoding. Image file Default format for transferring binary files File is sent as continuous streams of bits without any interpretation or encoding Used for compiled programs or images.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 File is encoded in ASCII or EBCDIC, another attribute must be added to define the printability of the file. Nonprint Default format for transferring a text file. File contains no vertical specifications for printing. File cannot be printed without further processing because there are no characters to be interpreted for vertical movement of the print head. This format is used for files that will be stored and processed later. TELNET File contains ASCII vertical characters such as CR (carriage return), LF (line feed), NL (new line) and VT (vertical tab). File is printable after transfer.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Data structure FTP can transfer a file across the data connection using one of the following interpretations about structure of data: File structure (default) File has no structure. Continuous stream of bytes Record structure File is divided into records (or structs in C). Used only with text files. Page structure File is divided into pages, with each page having a page number and a page header. Pages can be stored or accessed randomly or sequentially.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Transmission mode: FTP can transfer a file across the data connection by using one of following three transmission modes: Stream mode Default mode Data is delivered from FTP to TCP as a continuous stream of bytes; TCP is responsible for chopping data into segments of appropriate size. End-of-file is closing of data connection by sender. If data is divided into records, each record will have 1-byte end-of- record (EOR) character, and the end of the file will have a 1-byte end- of-file (EOF) character. Block mode Data can be delivered from FTP to TCP in blocks. Block is preceded by 3-byte header. 1 st byte is called the block descriptor; next 2 bytes defines the size of block in bytes. Compressed mode If file is big, data can be compressed. Commonly used compression method is run-length encoding. Consecutive appearance of a data unit are replaced by one occurrence and number of repetitions. Usually, Blanks in text file and null characters in binary file are compressed.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure File transfer Means of File Transfer Retrieving a file: Copy file from server to client. Storing a file: copy file from client to server. List of directory or file names is to be sent from the server to the client. Note that FTP treats a list of directory or file names as a file. It is sent over the data connection.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Example 1 Figure (next slide) shows an example of how a file is stored. 1.The control connection is created, and several control commands and responses are exchanged. 2.Data are transferred record by record. 3.A few commands and responses are exchanged to close the connection.

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Figure Example 1

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Table 26.4 List of FTP commands in UNIX Commands !, $, account, append, ascii, bell, binary, bye, case, cd, cdup, close, cr, delete, debug, dir, discount, form, get, glob, hash, help, lcd, ls, macdef, mdelete, mdir, mget, mkdir, mls, mode, mput, nmap, ntrans, open, prompt, proxy, sendport, put, pwd, quit, quote, recv, remotehelp, rename, reset, rmdir, runique, send, status, struct, sunique, tenex, trace, type, user, verbose,?

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Example 2 We show some of the user interface commands that accomplish the same task as in Example 1. The user input is shown in boldface. As shown below, some of the commands are provided automatically by the interface. The user receives a prompt and provides only the arguments. $ ftp challenger.atc.fhda.edu Connected to challenger.atc.fhda.edu 220 Server ready Name: forouzan Password: xxxxxxx ftp > ls /usr/user/report 200 OK 150 Opening ASCII mode transfer complete ftp > close 221 Goodbye ftp > quit

McGraw-Hill©The McGraw-Hill Companies, Inc., 2004 Example 3 We show an example of using anonymous [public access OR without an account] FTP. We connect to internic.net, where we assume there are some public data available. Sometimes, anonymous as user name and guest as password $ ftp internic.net Connected to internic.net 220 Server ready Name: anonymous 331 Guest login OK, send "guest" as password Password: guest ftp > pwd 257 '/' is current directory ftp > ls 200 OK 150 Opening ASCII mode bin... ftp > close 221 Goodbye ftp > quit