The Application Layer – HTTP and FTP Tahir Azim

Application Layer Protocols QoS lectures postponed to next week This week: Application Layer Protocols –FTP, HTTP, Bittorrent Materials reused from Philip Levis, David Mazieres (Stanford) and Vern Paxson (Berkeley)

File Transfer Protocol (FTP) Allows a user to copy files to/from remote hosts –Client program connects to FTP server –… provides a login id and password –… allows the user to explore the directories –… and download and upload files with the server A predecessor of the Web (RFC 959 in 1985) –Requires user to know the name of the server machine –… and have an account on the machine –… and find the directory where the files are stored –… and know whether the file is text or binary –… and know what tool to run to render and edit the file That is, no URL, hypertext, and helper applications

How Do You Transfer Files Today? HTTP - the usual Web transfer mechanism ( FTP –ftp:// links in web pages (e.g.: in sftp and scp –E.g.: to upload your project files to cms.niit.edu.pk BitTorrent and other file-sharing software Any others?

Why Study FTP? Helps cement familiarity with text/status-code based protocols Illustrates use of multiple concurrent connections –One for control (commands & replies) –Depending on command, can be additional one for data Illustrates reversal of roles –For data connection, FTP user’s process can play the server role, FTP server can play the client role

Example commands Authentication –USER: specify the user name to log in as –PASS: specify the user’s password Exploring the files –LIST: list the files for the given file specification –CWD: change to the given directory Downloading and uploading files –TYPE: set type to ASCII (A) or binary image (I) –RETR: retrieve the given file –STOR: upload the given file Closing the connection –QUIT: close the FTP connection

Server Response Codes 1xx: positive preliminary reply –The action is being started, but expect another reply before sending the next command. 2xx: positive completion reply –The action succeeded and a new command can be sent. 3xx: positive intermediate reply –The command was accepted but another command is now required. 4xx: transient negative completion reply –The command failed and should be retried later. 5xx: permanent negative completion reply –The command failed and should not be retried.

FTP Data Transfer Separate data connection –To send lists of files (LIST) –To retrieve a file (RETR) –To upload a file (STOR) control data

Creating the Data Connection Client acts like a server –Creates a socket Assigned an ephemeral port number by the kernel –Listens on socket –Waits to hear from FTP server socket

Creating Data Connection (cont.) But, the server doesn’t know the port number –So after starting to listen, client tells it to the server –Using the PORT command on the control connection –Server can tell the client a port to connect to using PASV or EPSV PORT

Why Out-of-Band Control? Avoids need to mark the end of the data transfer –Data transfer ends by closing of data connection –Yet, the control connection stays up Aborting a data transfer –Can abort a transfer without killing the control connection –… which avoids requiring the user to log in again –Done with an ABOR on the control connection Third-party file transfer between two hosts –Data connection could go to a different host –… by sending a different client IP address to the server –e.g., a user can coordinate a transfer between two servers –But: this is rarely needed, and presents security issues

Example Collected using: –tcpdump -s 0 -w ftp.trace host –ftp –Ethereal

Example blade1:~ # ftp Connected to Server sends back the following: Welcome to Pure-FTPd [privsep] [TLS] You are user number 1 of 50 allowed. 220-Local time is now 18:54. Server port: This is a private system - No anonymous login 220 You will be disconnected after 15 minutes of inactivity.

Example continued… Name ( tahir USER tahir 331 User tahir OK. Password required Password: PASS tahirazim 230-User tahir has group access to: tahir 230 OK. Current restricted directory is / SYST 215 UNIX Type: L8 FEAT 211-Extensions supported: EPRT IDLE MDTM SIZE REST STREAM MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*; MLSD ESTP PASV EPSV SPSV ESTA AUTH TLS PBSZ PROT 211 End.

Example continued… ftp> pwd PWD 257 "/" is your current location ftp> cd public_html/tinyos CWD public_html/tinyos 250 OK. Current directory is /public_html/tinyos

Example continued… ftp> ls EPSV 229 Extended Passive mode OK (|||22710|) LIST 150 Accepted data connection 226-Options: -l matches total Received from port of the server: -rw-r--r-- 1 tahir tahir Feb 1 14:23 BVR-T2-0.1.tar.gz -rw-r--r-- 1 tahir tahir 535 Jan 18 20:03 Makefile -rw-r--r-- 1 tahir tahir 2788 Jan 18 20:03 RadioCountToLedsAppC.nc -rw-r--r-- 1 tahir tahir 5219 Jan 18 20:03 RadioCountToLedsC.nc -rw-r--r-- 1 tahir tahir Jan 18 20:03 nesC-wksp.ppt

Example continued… ftp> get Makefile TYPE I 200 TYPE is now 8-bit binary SIZE Makefile EPSV 229 Extended Passive mode OK (|||9442|) RETR Makefile 150 Accepted data connection 226-File successfully transferred seconds (measured here), 1.11 Mbytes per second MDTM Makefile From port 9442 of the server: COMPONENT=RadioCountToLedsAppC BUILD_EXTRA_DEPS = RadioCountMsg.py RadioCountMsg.class RadioCountMsg.py: RadioCountToLeds.h.mig python -target=$(PLATFORM) $(CFLAGS) -python-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o RadioCountMsg.class: RadioCountMsg.java.javac RadioCountMsg.java RadioCountMsg.java: RadioCountToLeds.h.mig java -target=$(PLATFORM) $(CFLAGS) -java-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o #CFLAGS += -I/home/Tahir/T2SerialT1. include $(MAKERULES)

Example continued… ftp> exit QUIT 221-Goodbye. You uploaded 0 and downloaded 1 kbytes. 221 Logout.

HTTP Server listens on a port (by default, 80) On connection, waits for a request Protocol (but not data) is in ASCII Sends response, maybe closes connection (client can ask it to stay open)

Parsing a URL Protocol Host File path on host

HTTP Request Format Request types: GET, PUT, POST, HEAD, DELETE A trivial browser request:

A trivial browser request GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto... Accept: text/xml,application/xm... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO ,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

Other useful header fields Range: Request a partial range of data Authorization: Present authorization credentials to a server (not HTTPS) Proxy-Authorization: Present proxy credentials to a proxy server Referer: URL of the web page the user was on, when the HTTP request was made

HTTP Response Format 1xx codes: Informational 2xx codes: Successes 3xx codes: Redirection 4xx codes: Client Error, 5xx codes: Server Error

Response from thenews.com.pk HTTP/ OK Date: Fri, 14 Mar :13:05 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET Content-Length: Content-Type: text/html Set-Cookie: ASPSESSIONIDAADTQQAR=PFKAADCBBDGBDDJLMKLFJNMI; path=/ Cache-Control: private X-Cache: MISS from micronet-proxy.niit.edu.pk X-Cache-Lookup: MISS from micronet-proxy.niit.edu.pk:8080 X-Cache: MISS from proxy-ptcl.niit.edu.pk X-Cache-Lookup: MISS from proxy-ptcl.niit.edu.pk:8080 Via: 1.0 micronet-proxy.niit.edu.pk:8080 (squid/2.6.STABLE5), 1.0 proxy- ptcl.niit.edu.pk:8080 (squid/2.6.STABLE5) Connection: keep-alive The News - International - Friday, March 14, 2008 …

HTTP Performance What matters most? Different kinds of requests –Lots of small requests (loading a web page) –Big request (fetching a download) Require different solutions

Small requests Latency matters Governed by RTT between hosts Two major causes of delay: –Opening a TCP connection –Data response-request Solutions: –Persistent connections –Pre-fetching –Others??

Browser request, revisited GET / HTTP/1.1 Host: localhost:8000 User-Agent: Mozilla/5.0 (Macinto... Accept: text/xml,application/xm... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO ,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

Big requests Problem is throughput on bottleneck links (usually edge links) Use an HTTP proxy cache or mirror –Can also improve latency!

Stale caches Items in the cache can go stale (you don’t want to read yesterday’s paper) Cache needs a way to conditionally ask for a document Cache can issue a conditional GET (with an If- modified-since header) –Server can reply with a 304 Not Modified GET / HTTP/1.1 Host: If-modified-since: Wed, 2 April :00:00

Client-server vs Peer-to-peer Server can be a bottleneck –Download time can scale down O(n) with n clients –Scaling up server bandwidth can be expensive (CDNs) –Slashdotting/flash crowds Peer-to-peer: get a bunch of end-hosts to collaboratively distribute content A common peer-to-peer challenge is finding whom to collaborate with

Bittorrent