1 Web Programming Based on Notes by D. Hollinger Also Java Network Programming and Distributed Computing, Chs. 9,10 Also Online Java Tutorial, Sun.

2 Netprog HTTP World-Wide Web (Tim Berners-Lee & Cailliau ’92)

3 Netprog HTTP Topics HTTP – HyperText Transfer ProtocolHTTP – HyperText Transfer Protocol HTML – HyperText Markup LanguageHTML – HyperText Markup Language URI – Uniform Resource IdentifiersURI – Uniform Resource Identifiers –URL – Uniform Resource Locators –URN – Uniform Resource Names –URC – Uniform Resource Citations Server-Side ProgrammingServer-Side Programming –HTML Forms Only URLs are widely deployed in today’s Web! See Online Resources

4 HTTP Hypertext Transfer Protocol Refs: RFC 1945 (HTTP 1.0) RFC 2616 (HTTP 1.1)

5 Netprog HTTP HTTP Usage HTTP is the protocol that supports communication between web browsers and web servers.HTTP is the protocol that supports communication between web browsers and web servers. A “Web Server” is a HTTP serverA “Web Server” is a HTTP server We will look at HTTP Version 1.0 +We will look at HTTP Version 1.0 +

6 Netprog HTTP From the RFC “HTTP is an application-level protocol with the lightness and speed necessary for distributed, hypermedia information systems.”

7 Netprog HTTP Transport Independence The RFC states that the HTTP protocol generally takes place over a TCP connection, but the protocol itself is not dependent on a specific transport layer.The RFC states that the HTTP protocol generally takes place over a TCP connection, but the protocol itself is not dependent on a specific transport layer.

8 Netprog HTTP Request - Response HTTP has a simple structure:HTTP has a simple structure: –client sends a request –server returns a reply. HTTP can support multiple request- reply exchanges over a single TCP connection.HTTP can support multiple request- reply exchanges over a single TCP connection.

9 Netprog HTTP Well Known Address The “well known” TCP port for HTTP servers is port 80.The “well known” TCP port for HTTP servers is port 80. Other ports can be used as well...Other ports can be used as well...

10 Netprog HTTP HTTP Versions The original version now goes by the name “HTTP Version 0.9”The original version now goes by the name “HTTP Version 0.9” –HTTP 0.9 was used for many years. Starting with HTTP 1.0 the version number is part of every request.Starting with HTTP 1.0 the version number is part of every request. HTTP is still changing...HTTP is still changing...

11 Netprog HTTP HTTP 1.0+ Request Lines of text (ASCII).Lines of text (ASCII). Lines end with CRLF “\r\n”Lines end with CRLF “\r\n” First line is called “Request-Line”First line is called “Request-Line” Request-Line Headers. Content...

12 Netprog HTTP Request Line Method URI HTTP-Version \r\n The request line contains 3 tokens (words).The request line contains 3 tokens (words). space characters “ “ separate the tokens.space characters “ “ separate the tokens. Newline (\n) seems to work by itself (but the protocol requires CRLF)Newline (\n) seems to work by itself (but the protocol requires CRLF)

13 Netprog HTTP Request Method The Request Method can be:The Request Method can be: GETHEADPUT POSTDELETETRACE OPTIONS future expansion is supported

14 Netprog HTTP Methods GET: retrieve information identified by the URI.GET: retrieve information identified by the URI. HEAD: retrieve meta-information about the URI.HEAD: retrieve meta-information about the URI. POST: send information to a URI and retrieve result.POST: send information to a URI and retrieve result.

15 Netprog HTTP Methods (cont.) PUT: Store information in location named by URI.PUT: Store information in location named by URI. DELETE: remove entity identified by URI.DELETE: remove entity identified by URI.

16 Netprog HTTP More Methods TRACE: used to trace HTTP forwarding through proxies, tunnels, etc.TRACE: used to trace HTTP forwarding through proxies, tunnels, etc. OPTIONS: used to determine the capabilities of the server, or characteristics of a named resource.OPTIONS: used to determine the capabilities of the server, or characteristics of a named resource.

17 Netprog HTTP Common Usage GET, HEAD and POST are supported everywhere.GET, HEAD and POST are supported everywhere. HTTP 1.1 servers often support PUT, DELETE, OPTIONS & TRACE.HTTP 1.1 servers often support PUT, DELETE, OPTIONS & TRACE.

18 Netprog HTTP URI: Uniform Resource Identifier URIs defined in RFC 2396.URIs defined in RFC Absolute URI: scheme://hostname[:port]/pathAbsolute URI: scheme://hostname[:port]/pathhttp:// Relative URI: /pathRelative URI: /path/blah/foo No server mentioned

19 Netprog HTTP URI Usage When dealing with a HTTP 1.1 server, only a path is used (no scheme or hostname).When dealing with a HTTP 1.1 server, only a path is used (no scheme or hostname). –HTTP 1.1 servers are required to be capable of handling an absolute URI, but there are still some out there that won’t… When dealing with a proxy HTTP server, an absolute URI is used.When dealing with a proxy HTTP server, an absolute URI is used. –client has to tell the proxy where to get the document! –more on proxy servers in a bit….

20 Netprog HTTP HTTP Version Number “ HTTP/1.0 ” or “ HTTP/1.1 ” HTTP 0.9 did not include a version number in a request line. If a server gets a request line with no HTTP version number, it assumes 0.9

21 Netprog HTTP The Header Lines After the Request-Line come a number (possibly zero) of HTTP headers.After the Request-Line come a number (possibly zero) of HTTP headers. Each header line contains an attribute name followed by a “:” followed by the attribute value.Each header line contains an attribute name followed by a “:” followed by the attribute value.

22 Netprog HTTP Headers Request Headers provide information to the server about the clientRequest Headers provide information to the server about the client –what kind of client –what kind of content will be accepted –who is making the request There can be 0 headersThere can be 0 headers

23 Netprog HTTP Example HTTP Headers Accept: text/html From: User-Agent: Mozilla/4.0 Referer:

24 Netprog HTTP End of the Headers Each header ends with a CRLFEach header ends with a CRLF The end of the header section is marked with a blank line.The end of the header section is marked with a blank line. –just CRLF For GET and HEAD requests, the end of the headers is the end of the request!For GET and HEAD requests, the end of the headers is the end of the request!

25 Netprog HTTP POST A POST request includes some content (some data) after the headers (after the blank line).A POST request includes some content (some data) after the headers (after the blank line). There is no format for the data (just raw bytes).There is no format for the data (just raw bytes). A POST request must include a Content- Length line in the headers:A POST request must include a Content- Length line in the headers: Content-Length: 267

26 Netprog HTTP Example GET Request GET /~hollingd/testanswers.html HTTP/1.0 Accept: */* User-Agent: Internet Explorer From: Referer: There is a blank line here!

27 Netprog HTTP Example POST Request POST /~hollingd/changegrade.cgi HTTP/1.1 Accept: */* User-Agent: SecretAgent V2.3 Content-length: 35 Referer: stuid= &item=test1&grade=99

28 Netprog HTTP Typical Method Usage GET used to retrieve an HTML document. HEAD used to find out if a document has changed. POST used to submit a form.

29 Netprog HTTP HTTP Response ASCII Status LineASCII Status Line Headers SectionHeaders Section Content can be anything (not just text)Content can be anything (not just text) –typically is HTML document or some kind of image. Status-Line Headers. Content...

30 Netprog HTTP Response Status Line HTTP-Version Status-Code Message Status Code is 3 digit number (for computers)Status Code is 3 digit number (for computers) Message is text (for humans)Message is text (for humans)

31 Netprog HTTP Status Codes 1xx Informational 2xx Success 3xx Redirection 4xx Client Error 5xx Server Error

32 Netprog HTTP Example Status Lines HTTP/ OK HTTP/ Moved Permanently HTTP/ Bad Request HTTP/ Internal Server Error

33 Netprog HTTP Response Headers Provide the client with information about the returned entity (document).Provide the client with information about the returned entity (document). –what kind of document –how big the document is –how the document is encoded –when the document was last modified Response headers end with blank lineResponse headers end with blank line

34 Netprog HTTP Response Header Examples Date: Wed, 30 Jan :48:17 EST Server: Apache/1.17 Content-Type: text/html Content-Length: 1756 Content-Encoding: gzip

35 Netprog HTTP Content Content can be anything (sequence of raw bytes).Content can be anything (sequence of raw bytes). Content-Length header is required for any response that includes content.Content-Length header is required for any response that includes content. Content-Type header also required.Content-Type header also required.

36 Netprog HTTP Single Request/Reply The client sends a complete request.The client sends a complete request. The server sends back the entire reply.The server sends back the entire reply. The server closes it’s socket.The server closes it’s socket. If the client needs another document it must open a new connection.If the client needs another document it must open a new connection.

37 Netprog HTTP Persistent Connections HTTP 1.1 supports persistent connections (this is supposed to be the default).HTTP 1.1 supports persistent connections (this is supposed to be the default). Multiple requests can be handled.Multiple requests can be handled. Most servers seem to close the connection after the first response…Most servers seem to close the connection after the first response…

38 Netprog HTTP Try it with telnet > telnet GET / HTTP/1.0 HTTP/ OK Server: Apache... Response Request Blank Line (end of headers)

39 Netprog HTTP HTTP Proxy Server HTTP Server Browser Proxy

40 Netprog HTTP Tyba: A simple (and incomplete) HTTP Server Implementation in Java See:See:

41 Server-Side Programming

42 Netprog HTTP Web Server Architecture (Berners-Lee & Cailliau ’92)

43 Netprog HTTP Request Method: Get GET requests can include a query string as part of the URL:GET requests can include a query string as part of the URL: GET /program/finger?hollingd HTTP/1.0 Request Method Resource Name Delimiter Query String

44 Netprog HTTP /program/finger?hollingd The web server treats everything before the ‘?’ delimiter as the resource nameThe web server treats everything before the ‘?’ delimiter as the resource name In this case the resource name is the name of a program. (could be a CGI script, a servlet, or your own HTTP server)In this case the resource name is the name of a program. (could be a CGI script, a servlet, or your own HTTP server) Everything after the ‘?’ is a string that is passed to the server program (in the case of CGI and servlets)Everything after the ‘?’ is a string that is passed to the server program (in the case of CGI and servlets)

45 Netprog HTTP Simple GET queries - ISINDEX You can put an tag inside an HTML document.You can put an tag inside an HTML document. The browser will create a text box that allows the user to enter a single string.The browser will create a text box that allows the user to enter a single string. If an ACTION is specified in the ISINDEX tag, when the user presses Enter, a request will be sent to the server specified as the ACTION.If an ACTION is specified in the ISINDEX tag, when the user presses Enter, a request will be sent to the server specified as the ACTION.

46 Netprog HTTP ISINDEX Example Enter a string: Press Enter to submit your query. If you enter the string “blahblah”, the browser will send a request to the http server at foo.com that looks like this: GET /search?blahblah HTTP/1.1

47 Netprog HTTP URL-encoding Browsers use an encoding when sending query strings that include special characters.Browsers use an encoding when sending query strings that include special characters. –Most nonalphanumeric characters are encoded as a ‘%’ followed by 2 ASCII encoded hex digits. –‘=‘ (which is hex 3D) becomes “%3D” –‘&’ becomes “%26”

48 Netprog HTTP More URL encoding The space character ‘ ‘ is replaced by ‘+’.The space character ‘ ‘ is replaced by ‘+’. –Why? The ‘+’ character is replaced by “%2B”The ‘+’ character is replaced by “%2B”Example: “foo=6 + 7” becomes “foo%3D6+%2B+7”

49 Netprog HTTP URL Encoding in Java java.net.URLEncoder classjava.net.URLEncoder class String original = “foo=6 + 7”; System.out.println(URLEncoder.encode(original));foo%3D6+%2B+7

50 Netprog HTTP URL Decoding in Java java.net.URLDecoder classjava.net.URLDecoder class String encoded = “foo%3D6+%2B+7”; System.out.println(URLDecoder.decode(encoded)); foo=6 + 7

51 Netprog HTTP Beyond ISINDEX - Forms Many Web services require more than a simple field in the web form.Many Web services require more than a simple field in the web form. HTML includes support for forms:HTML includes support for forms: –lots of field types –user answers all kinds of annoying questions –entire contents of form must be stuck together and put in the query by the web client.

52 Netprog HTTP Form Fields Each field within a form has a name and a value.Each field within a form has a name and a value. The browser creates a query that includes a sequence of “ name=value” sub-strings and sticks them together separated by the ‘&’ character.The browser creates a query that includes a sequence of “ name=value” sub-strings and sticks them together separated by the ‘&’ character.

53 Netprog HTTP Form fields and encoding 2 fields - name and occupation.2 fields - name and occupation. If user types in “Dave H.” as the name and “none” for occupation, the query would look like this:If user types in “Dave H.” as the name and “none” for occupation, the query would look like this:“name=Dave+H%2E&occupation=none”

54 Netprog HTTP HTML Forms Each form includes a METHOD that determines what http method is used to submit the request.Each form includes a METHOD that determines what http method is used to submit the request. Each form includes an ACTION that determines where the request is made.Each form includes an ACTION that determines where the request is made.

55 Netprog HTTP An HTML Form Name: Occupation: </FORM>

56 Netprog HTTP What the server will get The query will be a URL-encoded string containing the name,value pairs of all form fields.The query will be a URL-encoded string containing the name,value pairs of all form fields. The server program (or a CGI script, or a servlet) must decode the query and separate the individual fields.The server program (or a CGI script, or a servlet) must decode the query and separate the individual fields.

57 Netprog HTTP HTTP Method: POST The HTTP POST method delivers data from the browser as the content of the request.The HTTP POST method delivers data from the browser as the content of the request. The GET method delivers data (query) as part of the URI.The GET method delivers data (query) as part of the URI.

58 Netprog HTTP GET vs. POST When using forms it’s generally better to use POST:When using forms it’s generally better to use POST: – there are limits on the maximum size of a GET query string (environment variable) –a post query string doesn’t show up in the browser as part of the current URL.

59 Netprog HTTP HTML Form using POST Set the form method to POST instead of GET. The browser will take care of the details...

60 Netprog HTTP Server reading POST If the request is a POST, the query is coming in the body of the HTTP request.If the request is a POST, the query is coming in the body of the HTTP request. The “Content-length” header tells us how much data to read.The “Content-length” header tells us how much data to read.

61 HTML Forms (in more detail)

62 Netprog HTTP Form Elements Each HTML form contains the following:Each HTML form contains the following: –, tags –The tag has two required attributes: METHOD specifies the HTTP method used to send the request to the server (when the user submits the form).METHOD specifies the HTTP method used to send the request to the server (when the user submits the form). ACTION specifies the URL the request is sent to.ACTION specifies the URL the request is sent to.

63 Netprog HTTP FORM Method We have seen the two common methods used:We have seen the two common methods used: –GET : any user input is submitted as part of the URI following a “?”. GET foo?name=joe&cookie=oreo HTTP/1.0 –POST : any user input is submitted as the content of the request (after the HTTP headers).

64 Netprog HTTP Sample POST Request POST /dir/foo HTTP/1.0 User-Agent: Netscape Content-Length: 20 Cookie: favorite=chocolatechip ECACChamps: RPI name=joe&cookie=oreo POST /dir/foo HTTP/1.0 User-Agent: Netscape Content-Length: 20 Cookie: favorite=chocolatechip ECACChamps: RPI name=joe&cookie=oreo

65 Netprog HTTP Form ACTION attribute The ACTION attribute specifies the URL to which the request is sent. Some examples:The ACTION attribute specifies the URL to which the request is sent. Some

66 Netprog HTTP Tag Examples Tag Examples <FORM METHOD=“POST” ACTION=“ ACTION=“ <FORM METHOD=“POST” <FORM METHOD=“POST” ACTION=“ ACTION=“ <FORM METHOD=“POST”

67 Netprog HTTP Inside a form Between the and tags you define the text and fields that make up the form.Between the and tags you define the text and fields that make up the form. You can use normal HTML tags to format the text however you want.You can use normal HTML tags to format the text however you want. The fields are defined using tags as well.The fields are defined using tags as well.

68 Netprog HTTP Form Fields There are a variety of types of form fields:There are a variety of types of form fields: –text fields: text, password, textarea –radio buttons –checkboxs –buttons: user defined, submit, reset (clear) –hidden fields

69 Netprog HTTP Input Fields There are a number of field types that allow the user to type in a string value as input.There are a number of field types that allow the user to type in a string value as input. Each field is created using an tag with the attribute TYPE.Each field is created using an tag with the attribute TYPE.

70 Netprog HTTP Input Attributes The TYPE attribute is used to specify what kind of input is allowed: TEXT, PASSWORD, FILE,...The TYPE attribute is used to specify what kind of input is allowed: TEXT, PASSWORD, FILE,... Every INPUT tag must have a NAME attribute.Every INPUT tag must have a NAME attribute.

71 Netprog HTTP TEXT Fields TEXT is the most common type of input:TEXT is the most common type of input: –user can enter a single line of text. –Additional attributes can specify: the maximum string length - MAXLENGTHthe maximum string length - MAXLENGTH the size of the input box drawn by the browser - SIZEthe size of the input box drawn by the browser - SIZE a default value - VALUEa default value - VALUE

72 Netprog HTTP TEXT INPUT Examples <INPUT TYPE=“TEXT” NAME=“PIZZA” NAME=“PIZZA” SIZE=10 SIZE=10 MAXLENGTH=20 MAXLENGTH=20 VALUE=“Pepperoni”> VALUE=“Pepperoni”> <INPUT TYPE=“TEXT” NAME=“PIZZA” NAME=“PIZZA” SIZE=10 SIZE=10 MAXLENGTH=20 MAXLENGTH=20 VALUE=“Pepperoni”> VALUE=“Pepperoni”>

73 Netprog HTTP An example form Your Name: Your Age: </FORM> Your Name: Your Age: </FORM>

74 Netprog HTTP Submission Buttons Another type of INPUT field is the submission button.Another type of INPUT field is the submission button. When a user clicks on a submit button the browser submits the contents of all other fields to a web server using the METHOD and ACTION attributes.When a user clicks on a submit button the browser submits the contents of all other fields to a web server using the METHOD and ACTION attributes.

75 Netprog HTTP Reset Buttons An INPUT of type RESET tells the browser to display a button that will clear all the fields in the form.An INPUT of type RESET tells the browser to display a button that will clear all the fields in the form. <INPUT TYPE=RESET VALUE=“press me to clear form”> VALUE=“press me to clear form”>

76 Netprog HTTP A Complete Form Example Your Name: Your Age: Your Age: </FORM> Your Name: Your Age: Your Age: </FORM>

77 Netprog HTTP Tables and Forms Tables are often used to make forms look pretty - remember that you can use any HTML tags to control formatting of a form.

78 Netprog HTTP Table/Form example <TABLE><TR> Your Name: Your Name: </TR><TR> Your Age: Your Age: </TR><TR> </TR></TABLE></FORM> <TABLE><TR> Your Name: Your Name: </TR><TR> Your Age: Your Age: </TR><TR> </TR></TABLE></FORM>

79 Netprog HTTP Other Inputs CheckboxesCheckboxes –present user with items that can be selected or deselected. Each checkbox has a name and a value and can be initially selected/deselected –Example checkbox definitions:

80 Netprog HTTP Checkbox example Select all the cookies you want to order: Select all the cookies you want to order: Oreo Oreo Oatmeal Oatmeal Chocolate Chip Chocolate Chip </FORM> Select all the cookies you want to order: Select all the cookies you want to order: Oreo Oreo Oatmeal Oatmeal Chocolate Chip Chocolate Chip </FORM>

81 Netprog HTTP Radio Buttons Radio Buttons are like checkbox except that the user can select only one item at a time.Radio Buttons are like checkbox except that the user can select only one item at a time. All radio buttons in a group have the same NAME.All radio buttons in a group have the same NAME.

82 Netprog HTTP Radio Button Example Select all the cookies you want to order: Select all the cookies you want to order: Oreo Oreo Oatmeal Oatmeal ChocolateChip ChocolateChip </FORM> Select all the cookies you want to order: Select all the cookies you want to order: Oreo Oreo Oatmeal Oatmeal ChocolateChip ChocolateChip </FORM>

83 Netprog HTTP Multiline Text The TEXTAREA tag creates an area where the user can submit multiple lines of text.The TEXTAREA tag creates an area where the user can submit multiple lines of text. This is not another type of tag!This is not another type of tag!

84 Netprog HTTP TEXTAREA Attributes Each TEXTAREA tag has attributes NAME, COLS and ROWS.Each TEXTAREA tag has attributes NAME, COLS and ROWS. default text goes here (or can be empty) </TEXTAREA>

85 Netprog HTTP TEXTAREA example Please enter your address in the space provided: Please enter your address in the space provided: </TEXTAREA><BR> </FORM> Please enter your address in the space provided: Please enter your address in the space provided: </TEXTAREA><BR> </FORM> textarea1.html

86 Netprog HTTP Form Submission When the user presses on a SUBMIT button the following happens:When the user presses on a SUBMIT button the following happens: –browser uses the FORM method and action attributes to construct a request. –A query string is built using the (name,value) pairs from each form element. –Query string is URL-encoded.

87 Netprog HTTP Input Submissions For each checkbox selected the name,value pair is sent.For each checkbox selected the name,value pair is sent. For all checkboxes that are not selected - nothing is sent.For all checkboxes that are not selected - nothing is sent. A single name,value pair is sent for each group of radio buttons.A single name,value pair is sent for each group of radio buttons.

88 Netprog HTTP Other Form Field Types There are other form field types:There are other form field types: –SELECT - pulldown menu or scrolled list of choices. –Image Buttons –Push Buttons (choice of submit buttons)

89 Netprog HTTP Hidden Fields Nothing is displayed by the browser.Nothing is displayed by the browser. The name,value are sent along with the submission request.The name,value are sent along with the submission request. <INPUT TYPE=HIDDEN NAME=SECRET NAME=SECRET VALUE=AGENT> VALUE=AGENT>

90 Netprog HTTP Hidden does not mean secure! Anyone can look at the source of an HTML document.Anyone can look at the source of an HTML document. –hidden fields are part of the document! If a form uses GET, all the name/value pairs are sent as part of the URIIf a form uses GET, all the name/value pairs are sent as part of the URI –URI shows up in the browser as the location of the current page