Network Applications: The Web and High-Performance Web Servers Y. Richard Yang 9/24/2013

2 Outline  Admin and recap r HTTP details r High-performance Web server

3 Admin r Assignment One solution posted on the class homepage r Assignment Two due one week from today

4 Recap: FTP r Two types of TCP connections opened: m a control connection; m data connections two approaches to open a data connection: PASV or PORT r FTP is called a stateful protocol m state established by commands such as USER/PASS CWD TYPE

5 Recap: HTTP Message Flow HTTP client HTTP server Server sends file on same connection GET /home/index.html FTP client FTP server Server initiates TCP data connection server:20 clientip:cport PORT clientip:cport RETR index.html USER xxx PASS xxx CWD home

6 Recap: HTTP Req. Msg Format r ASCII (human-readable format)

7 Examples r Access BasicWebServer using common browsers and observe the request headers

8 Recap: Dynamic Content Pages r There are multiple approaches to make dynamic web pages: m Embedding code into pages (server side include) http server includes an interpreter for the type of pages m Invoke external programs (http server is agnostic to the external program execution)

9 Example SSI r See index.shtml, header.shtml, …

10 CGI: Invoking External Programs r Two issues m Input: Pass HTTP request parameters to the external program m Output: Redirect external program output to socket

11 Example: Typical CGI Implementation r Starts the executable as a child process m Passes HTTP request as environment variables CGI standard: m Redirects input/output of the child process to the socket

12 Example: CGI r Example: m GET /search?q=Yale&sourceid=chrome HTTP/1.0 m setup environment variables, in particular $QUERY_STRING= q=Yale&sourceid=chrome m start search and redirect its input/output

13 Example r Exec m #!/usr/bin/perl -w $company = $ENV{'QUERY_STRING'}; print "Content-Type: text/html\r\n"; print "\r\n"; print " "; print " Hello! The price is "; if ($company =~ /appl/) { my $var_rand = rand(); print * $var_rand; } else { print "150"; } print " ";

14 Client Using Dynamic Pages r See ajax.html for client code example

15 Discussions r What features are missing in HTTP that we have covered so far?

HTTP: POST r If an HTML page contains forms or parameter too large, they are sent using POST and encoded in message body 16

HTTP: POST Example 17 POST /path/script.cgi HTTP/1.0 User-Agent: MyAgent Content-Type: application/x-www-form-urlencoded Content-Length: 15 item1=A&item2=B Example using nc

18 Stateful User-server Interaction: Cookies Goal: no explicit application level session r Server sends “cookie” to client in response msg Set-cookie: r Client presents cookie in later requests Cookie: r Server matches presented-cookie with server-stored info m authentication m remembering user preferences, previous choices client server usual http request msg usual http response + Set-cookie: # usual http request msg Cookie: # usual http response msg usual http request msg Cookie: # usual http response msg cookie- specific action cookie- specific action

Cookie Example r Modify BasicHTTPSwever.java to set Cookie 19

20 Authentication of Client Request Authentication goal: control access to server documents r stateless: client must present authorization in each request r authorization: typically name, password  Authorization: header line in request m if no authorization presented, server refuses access, sends WWW-authenticate: header line in response client server usual http request msg 401: authorization req. WWW-authenticate: usual http request msg + Authorization:line usual http response msg usual http request msg + Authorization:line usual http response msg time Browser caches name & password so that user does not have to repeatedly enter it.

21 Example: Amazon S3 r Amazon S3 API m /APIRest.html

22 HTTP/1.0 Delay r >= 2 RTTs per object: m TCP handshake RTT m client request and server responds --- at least 1 RTT (if object can be contained in one packet) r Discussion: how to reduce delay? TCP SYN TCP/ACK; HTTP GET TCP ACK base page TCP SYN TCP/ACK; HTTP GET TCP ACK image 1

23 HTTP Message Flow: Persistent HTTP r Default for HTTP/1.1 r On same TCP connection: server parses request, responds, parses new request, … r Client sends requests for all referenced objects as soon as it receives base HTML r Fewer RTTs

24 Browser Cache and Conditional GET r Goal: don’t send object if client has up-to-date stored (cached) version r client: specify date of cached copy in http request If-modified-since: r server: response contains no object if cached copy up- to-date: HTTP/ Not Modified client server http request msg If-modified-since: http response HTTP/ Not Modified object not modified http request msg If-modified-since: http response HTTP/ OK … object modified

25 Web Caches (Proxy) r User sets browser: Web accesses via web cache r Client sends all http requests to web cache m if object at web cache, web cache immediately returns object in http response m else requests object from origin server, then returns http response to client Goal: satisfy client request without involving origin server client Proxy server client http request http response http request http response http request http response origin server origin server

26 Two Types of Proxies

27 Summary: HTTP r HTTP message format m ASCII (human-readable format) requests, header lines, entity body, and responses line r HTTP message flow m stateless server each request is self-contained; thus cookie and authentication, are needed in each message m reducing latency persistent HTTP –the problem is introduced by layering ! conditional GET reduces server/network workload and latency cache and proxy reduce traffic and/or latency - Is the application extensible, scalable, robust, secure?

WebServer Implementation TCP socket space state: listening address: {*.6789, *.*} completed connection queue: sendbuf: recvbuf: state: listening address: {*.25, *.*} completed connection queue: sendbuf: recvbuf: state: established address: { :6789, } sendbuf: recvbuf: connSocket = accept() Create ServerSocket(6789) write file to connSocket close connSocket Discussion: what does each step do and how long does it take? read request from connSocket read local file

Writing High Performance Servers: Major Issues r Many socket/IO operations can cause a process to block, e.g.,  accept : waiting for new connection;  read a socket waiting for data or close;  write a socket waiting for buffer space;  I/O read/write for disk to finish

Recap: Server Processing Steps Accept Client Connection Read Request Find File Send Response Header Read File Send Data may block waiting on disk I/O Want to be able to process requests concurrently. may block waiting on network

Goal: Limited Only by the Bottleneck CPU DISK Before NET CPU DISK NET After

Using Multi-Threads for Servers r A thread is a sequence of instructions which may execute in parallel with other threads r A multi-thread server is a concurrent program as it has multiple threads that are active at the same time, e.g., m we can have one thread for each client connection m thus, only the flow (thread) processing a particular request is blocked

Multi-Threaded Server r A multithreaded server might run on one CPU m The CPU alternates between running different threads m The scheduler takes care of the details m Switching between threads might happen at any time r Might run in parallel on a multiprocessor machine 33

Java Thread Model r Every Java application has at least one thread m The “main” thread, started by the JVM to run the application’s main() method m Most JVM’s use POSIX threads to implement Java threads r main() can create other threads m Explicitly, using the Thread class m Implicitly, by calling libraries that create threads as a consequence (RMI, AWT/Swing, Applets, etc.) 34

Thread vs Process 35

Java Thread Class  Concurrency is introduced through objects of the class Thread m Provides a ‘handle’ to an underlying thread of control r Threads are organized into thread groups  A thread group represents a set of threads activeGroupCount (); m A thread group can also include other thread groups to form a tree m Why thread group? 36

Some Main Java Thread Methods  Thread(Runnable target) Allocates a new Thread object. ThreadRunnable  Thread(String name) Allocates a new Thread object. ThreadString  Thread(ThreadGroup group, Runnable target) Allocates a new Thread object. ThreadThreadGroup Runnable r start() Start the processing of a thread; JVM calls the run method 37

Creating Java Thread r Two ways to implement Java thread  Extend the Thread class Overwrite the run() method of the Thread class  Create a class C implementing the Runnable interface, and create an object of type C, then use a Thread object to wrap up C  A thread starts execution after its start() method is called, which will start executing the thread’s (or the Runnable object’s) run() method  A thread terminates when the run() method returns 38

Option 1: Extending Java Thread 39 class PrimeThread extends Thread { long minPrime; PrimeThread(long minPrime) { this.minPrime = minPrime; } public void run() { // compute primes larger than minPrime... } } PrimeThread p = new PrimeThread(143); p.start();

Option 1: Extending Java Thread 40 class RequestHandler extends Thread { RequestHandler(Socket connSocket) { // … } public void run() { // process request } … } Thread t = new RequestHandler(connSocket); t.start();

Option 2: Implement the Runnable Interface 41 class PrimeRun implements Runnable { long minPrime; PrimeRun(long minPrime) { this.minPrime = minPrime; } public void run() { // compute primes larger than minPrime... } } PrimeRun p = new PrimeRun(143); new Thread(p).start();

Option 2: Implement the Runnable Interface 42 class RequestHandler implements Runnable { RequestHandler(Socket connSocket) { … } public void run() { // } … } RequestHandler rh = new RequestHandler(connSocket); Thread t = new Thread(rh); t.start();

Backup Slides 43

44 Benefits of Web Caching Assume: cache is “close” to client (e.g., in same network) r smaller response time: cache “closer” to client r decrease traffic to distant servers m link out of institutional/local ISP network often bottleneck origin servers public Internet institutional network 10 Mbps LAN 1.5 Mbps access link institutional cache

45 Proxy Caches Users Regional Network Rest of Internet Bottleneck... Cache Sharing: Internet Cache Protocol (ICP)

46 Cache Sharing via ICP  When one proxy has a cache miss, send queries to all siblings (and parents): “do you have the URL?”  Whoever responds first with “Yes”, send a request to fetch the file  If no “Yes” response within certain time limit, send request to Web server Parent Cache (optional) Discussion: where is the performance bottleneck of ICP?

47 Summary Cache r Basic idea: m let each proxy keep a directory of what URLs are cached in every other proxy, and use the directory as a filter to reduce number of queries r Problem: storage requirement m solution: compress the directory => imprecise, but inclusive directory

48 The Problem abc.com/index.html xyz.edu/ Proxy A Proxy B Compact Representation ?

49 Bloom Filters r Support membership test for a set of keys r To check if URL x is at B, compute H 1 (x), H 2 (x), H 3 (x), H 4 (x), and check V B URL u Bit Vector V B k hash functions H (u) = P m bits

50 No Free Lunch: Problems of Web Caching r The major issue of web caching is how to maintain consistency r Two ways m pull Web caches periodically pull the web server to see if a document is modified m push whenever a server gives a copy of a web page to a web cache, they sign a lease with an expiration time; if the web page is modified before the lease, the server notifies the cache