1. Lecture 19 Overview

2. Hyper Text Transfer Protocol HTTP is the protocol that supports communication between web browsers and web servers. – A “Web Server” is a HTTP server Most clients/servers today speak version 1.1, but 1.0 is also in use. “HTTP is an application-level protocol with the lightness and speed necessary for distributed, hypermedia information systems.” HTTP 2

3. HTTP overview Web’s application layer protocol client/server model – client: browser that requests, receives, “displays” Web objects – server: Web server sends objects in response to requests 3 PC running Explorer Server running Apache Web server Mac running Navigator HTTP request HTTP response

4. Request - Response HTTP has a simple structure: – client sends a request – server returns a reply HTTP can support multiple request-reply exchanges over a single TCP connection The “well known” TCP port for HTTP servers is port 80 – Other ports can be used as well HTTP 4

5. HTTP connections HTTP is “stateless” – server maintains no information about past client requests Nonpersistent HTTP – At most one object is sent over a TCP connection Persistent HTTP – Multiple objects can be sent over single TCP connection between client and server 5

6. Request Line Method URI HTTP-Version\r\n The request line contains 3 tokens (words) space characters “ ” separate the tokens HTTP 6 Request-Line Headers. Content... blank line

7. URI: Universal Resource Identifier URIs defined in RFC 2396 Absolute URI: – scheme://hostname[:port]/path – http://www.cse.unr.edu:80/~mgunes/cpe401 Relative URI: – /path – /blah/foo HTTP 7 No server mentioned

8. Request Method The Request Method can be: GETHEADDELETE PUT POSTTRACE OPTIONS future expansion is supported GET, HEAD and POST are supported everywhere HTTP 1.1 servers often support PUT, DELETE, OPTIONS & TRACE HTTP 8

9. Methods GET: retrieve information identified by the URI – Typically used to retrieve an HTML document HEAD: retrieve meta-information about the URI – used to find out if a document has changed POST: send information to a URI and retrieve result – used to submit a form HTTP 9

10. More Methods PUT: Store information in location named by URI DELETE: remove entity identified by URI TRACE: used to trace HTTP forwarding through proxies, tunnels, etc OPTIONS: used to determine capabilities of server, or characteristics of a named resource HTTP 10

11. The Header Lines Request Headers provide information to the server about the client – what kind of client – what kind of content will be accepted – who is making the request Each header line contains – an attribute name followed by a “:” followed by a space and the attribute value HTTP 1.1 requires a Host: header HTTP 11

12. End of the Headers Each header ends with a CRLF ( \r\n ) The end of the header section is marked with a blank line – just CRLF For GET and HEAD requests, the end of the headers is the end of the request! HTTP 12

13. HTTP request message format 13

14. POST A POST request includes some content (some data) after the headers – after the blank line There is no format for the data – just raw bytes A POST request must include a Content- Length line in the headers: – Content-length: 267 HTTP 14

15. HTTP Response ASCII Status Line Headers Section Content can be anything – not just text – typically an HTML document or some kind of image HTTP 15 Status-Line Headers. Content... blank line

16. Response Status Line HTTP-Version Status-Code Message Status Code is 3 digit number (for computers) – 1xxInformational – 2xxSuccess – 3xxRedirection – 4xxClient Error – 5xxServer Error Message is text (for humans) HTTP 16

17. Response Headers Provide the client with information about the returned entity (document) – what kind of document – how big the document is – how the document is encoded – when the document was last modified Response headers end with blank line HTTP 17

18. Content Content can be anything – sequence of raw bytes Content-Length header is required for any response that includes content Content-Type header also required HTTP 18

19. Single Request/Reply The client sends a complete request The server sends back the entire reply The server closes it’s socket If the client needs another document it must open a new connection HTTP 19 This was the default for HTTP 1.0

20. Persistent Connections HTTP 1.1 supports persistent connections – this is the default Multiple requests can be handled over a single TCP connection The Connection: header is used to exchange information about persistence (HTTP/1.1) 1.0 Clients used a Keep-alive: header HTTP 20

21. User-server state: cookies Four components: – 1) cookie header line of HTTP response message – 2) cookie header line in HTTP request message – 3) cookie file kept on user’s host, managed by user’s browser – 4) back-end database at Web site Cookies and privacy: – cookies permit sites to learn a lot about you – you may supply name and e-mail to sites 21

22. Cookies: keeping “state” 22 client server usual http response msg cookie file one week later: usual http request msg cookie: 1678 cookie- specific action access ebay 8734 usual http request msg Amazon server creates ID 1678 for user create entry usual http response Set-cookie: 1678 ebay 8734 amazon 1678 usual http request msg cookie: 1678 cookie- spectific action access ebay 8734 amazon 1678 backend database

23. Cookies (continued) What cookies can bring: – authorization – shopping carts – recommendations – user session state (Web e-mail) How to keep “state”: – protocol endpoints: maintain state at sender/receiver over multiple transactions – cookies: http messages carry state 23

24. Web caches (proxy server) user sets browser: Web accesses via cache browser sends all HTTP requests to cache – object in cache: cache returns object – else cache requests object from origin server, then returns object to client 24 client Proxy server client HTTP request HTTP response HTTP request origin server origin server HTTP response Goal: satisfy client request without involving origin server

25. More about Web caching cache acts as both client and server typically cache is installed by ISP – university, company, residential ISP Why Web caching? reduce response time for client request reduce traffic on an institution’s access link. Internet dense with caches: enables “poor” content providers to effectively deliver content (but so does P2P file sharing) 25

26. Conditional GET Goal: don’t send object if cache has up-to-date cached version cache: specify date of cached copy in HTTP request – If-modified-since: server: response contains no object if cached copy is up-to-date: – HTTP/1.0 304 Not Modified 26 cache server HTTP request msg If-modified-since: HTTP response HTTP/1.0 304 Not Modified object not modified HTTP request msg If-modified-since: HTTP response HTTP/1.0 200 OK object modified

27. Lecture 20 Dynamic Web Servers CPE 401 / 601 Computer Network Systems slides are modified from Dave Hollinger

28. Web Server Talks HTTP Looks at METHOD, URI to determine what the client wants. For GET, URI often is just the path of a file – relative to some directory on the web server Dynamic Web Servers 28

29. GET /foo/blah Dynamic Web Servers 29 usrbinwwwetcfoofungif / blah

30. Dynamic Documents Dynamic Documents can provide: – automation of web site maintenance – customized advertising – database access – shopping carts – date and time service – … Dynamic Web Servers 30

31. Web Programming Writing programs that create dynamic documents has become very important There are a number of general approaches: – Create custom server for each service desired Each is available on different port. – Develop a real smart web server Server Side Includes, scripting, server APIs – Have web server run external programs Dynamic Web Servers 31

32. Custom Server Write a TCP server that watches a “well known” port for requests Develop a mapping from http requests to service requests Send back HTML (or whatever) that is created/selected by the server process Have to handle http errors, headers, etc Dynamic Web Servers 32

33. Drawbacks to Custom Server Approach We might have lots of ideas custom services – Each requires dedicated address (port) – Each needs to include: basic TCP server code parsing HTTP requests error handling headers access control Dynamic Web Servers 33

34. Smart Web Server Take a general purpose Web server (that can handle static documents) and – have it process requested documents as it sends them to the client The documents could contain commands that the server understands – the server includes some kind of interpreter Dynamic Web Servers 34

35. Example Smart Server Have the server read each HTML file as it sends it to the client The server could look for this: some command The server doesn’t send this part to the client, instead it interprets the command and sends the result to the client Everything else is sent normally Dynamic Web Servers 35

36. Server Side Includes Server Side Includes (SSI) provides a set of commands that a server will interpret Typically the server is configured to look for commands only in specially marked documents – so normal documents aren’t slowed down SSI commands are called directives – Directives are embedded in HTML comments Dynamic Web Servers 36

37. SSI Directives A comment looks like this: A directive looks like this: SSI servers keep a number of useful things in environment variables: DOCUMENT_NAME, DOCUMENT_URL echo: inserts the value of an environment variable into the page This page is located at Dynamic Web Servers 37

38. SSI Directives include: inserts the contents of a text file. flastmod: inserts the time and date that a file was last modified. Last modified: exec: runs an external program and inserts the output of the program. Current users: Dynamic Web Servers 38 Danger! Danger! Danger!

39. More Power Some servers support elaborate scripting languages Scripts are embedded in HTML documents, the server interprets the script: – Microsoft Active Server Pages (ASP) JScript, VBScript, PerlScript – Netscape LiveWire JavaScript, SQL connection library. – Many others… Dynamic Web Servers 39

40. Server Mapping and APIs Some servers include a programming interface that allows to extend the capabilities of the server by writing modules Specific URLs are mapped to specific modules instead of to files Dynamic Web Servers 40

41. External Programs Another approach is to provide a standard interface between external programs and web servers – We can run the same program from any web server – The web server handles all the http, we focus on the special service only – It doesn’t matter what language we use to write the external program Dynamic Web Servers 41

42. Common Gateway Interface CGI is a standard interface to external programs supported by most (if not all) web servers – CGI programs are often written in scripting languages (perl, tcl, etc.), The interface that is defined by CGI includes: – Identification of the service (i.e.,external program) – Mechanism for passing the request to the external program Dynamic Web Servers 42

43. Common Gateway Interface CGI is a standard mechanism for: – Associating URLs with programs that can be run by a web server – A protocol (of sorts) for how the request is passed to the external program – How the external program sends the response to the client CGI 43

44. CGI Programming CGI 44 CLIENT HTTP SERVER CGI Program http request http response setenv(), dup(), fork(), exec(),...

45. CGI URLs There is mapping between URLs and CGI programs provided by a web sever – The exact mapping is not standardized web server admin can set it up Typically: – requests that start with /CGI-BIN/, /cgi-bin/ or /cgi/, etc. not to static documents CGI 45

46. HTTP Server - CGI Interaction CGI 46 HTTP SERVER CGI Program stdin stdout Environment Variables

47. Environment Variables The web server sets some environment variables with information about the request The web server fork()s and the child process exec()s the CGI program The CGI program gets information about the request from environment variables CGI 47

48. STDIN, STDOUT Before calling exec(), the child process sets up pipes so that – stdin comes from the web server and – stdout goes to the web server In some cases part of the request is read from stdin Anything written to stdout is forwarded by the web server to the client CGI 48

49. Request Method: Get GET requests can include a query string as part of the URL: GET /cgi-bin/login?mgunes HTTP/1.0 CGI 49 Request Method Resource Name Delimiter Query String