1. (Hypertext Transfer Protocols Headers)
HTTP HEADERS
(Hypertext Transfer Protocols Headers)

2. Request/ Response Messages
GET /announce.html HTTP/1.0
Host:
Request
Server
Client
HTTP/ OK
Content-Type: text/plain
Content-Length:19
Hi! I’m a message
Response

3. HEADERS
Provide information about request or response, or about object sent in the message body.
Headers names are followed by a colon (:), followed by optional whitespace, followed by a value.
Syntax: - “Header-Name: value”
Header name is not case-sensitive.
Headers are terminated by a blank line (CRLF), marking the end of the list of headers & beginning of the entity body.

4. EXAMPLES Accept: image/gif, image/jpg, */* Accept-Encoding: gzip
Connection: Keep-Alive
Host:
Referer:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

5. 5 Main Classes of Headers
General Headers: Used by both clients & servers e.g., Connection, Date etc.
Request Headers: Provides extra information to servers of request e.g., Accept, Host etc.
Response Headers: Provides information to the client e.g. , Server, Age etc.
Entity Headers: Describes size & contents of body, or resources itself e.g., Content Length, Expires etc.
Extension Headers: Are nonstandard headers that have been created by application developers that are not defined in the specification e.g., Client-ip, Cookie etc.

6. Cache & Cookies Headers (Not part of this tutorial )
Cache’s Headers Cookies’s Headers
Age Cookie
Cache-Control Cookie2
Expires Set-Cookie etc.
If-Modified-Since
If-Match
If-None-Match
If-Range
If-Unmodified
Pragma etc.

7. REQUEST HEADERS

8. Accept Tells servers what media types are acceptable.
Its value is a list of media types that client can use.
May include a list of quality values(q values) that tell the server which media is preferred, in case of multiple media type. Q values range from 0.0 to 1.0
“*/*” represents all types, & “image/*” represents all image types.
EXAMPLES: Accept: text/*, image/jpeg;q=1

9. Accept-Language
Allowing clients to inform the server about what languages are acceptable.
EXAMPLES: Accept-Language: en
Accept-Language: en;q=0.5, fr;q=0.8

10. Accept-Charset
Indicate what character sets are acceptable by client for response.
Each charset may be given an associated quality value which represents user's preference. The default value is q=1.
EXAMPLES: Accept-Charset: iso-latin-1;q=0.8

11. Accept-Encoding
Used by clients to tell servers what encoding are acceptable.
EXAMPLES:
Accept-Encoding: compress,gzip
Accept-Encoding: compress;q=0.5, gzip;q=1.0
Accept-Encoding:
Empty Accept-Encoding means only the uuencoded content is acceptable.

12. Authorization Sent by a client to authenticate itself with a server.
A client will include this header in its request after receiving a 401 Authentication Required response from a server.
BASIC SYNTAX: Authorization: authentication-scheme # authentication-param.
EXAMPLE: Authorization: Basic kerjtojrk56kjf

13. From From where the request is coming from.
Gives address of the user making the request.
Used for logging purposes and as a means for identifying the source of invalid or unwanted requests.
Client should not send this header without user's approval, as it might conflict with user's privacy.
EXAMPLE:

14. Max-Forwards
Used only with the TRACE method(allows clients to see how its request looks when it finally makes it to the server), to limit no. of proxies or gateways, or other application intermediaries that a request goes through.
Its value is an integer.
Each application that receives a Trace request with this header should decrement the value before it forwards the request.
If it is missing from Trace request, then there is no maximum no. of forwards.
If value is zero when application receives request, it should send back a 200 OK response, with an entity body containing original request.
EXAMPLE: Max-Forwards :5

15. Host
Used by clients to provide the server with the Internet hostname & port no. of the server from which the client wants the response.
HTTP/1.1 clients must include a Host header in all request.
All HTTP/1.1 servers must respond with the 400 Bad Request status code to HTTP/1.1 clients that do not provide a Host header.
If the URL contains an IP address, the Host header should contain the same address.
Web proxies must add Host headers to request messages before forwarding them.
EXAMPLE: Host:
Host:

16. Interpreting Host Headers
Any origin server that does differentiate resources based on the host, must use the following rules for determining the resource to send on an HTTP/1.1 request:
If the URL is absolute ( Host header ignored, Host is taken from URL.
If the URL doesn’t have a host, the value is obtained from Host header.
If no valid host determined through step 1 or 2, 400(Bad Request) response is send.

18. Referer
Included to let the server know from where the client got the URL .
E.g., If you get home page A & click on a link to go to page B, your browser will insert a Referer header in the request with value A.
Inserted only when you click on links.
Requests for URLs you type yourself will not contain it.
EXAMPLE:
Referer:

19. User-Agent Used by client applications to identify themselves.
Its value is the product name & possibly describing the client application.
For example, browser sends the user-agent string which indicates which browser you are using, its version number, and details about your system, such as operating system and its version.
EXAMPLE: User-Agent: Mozilla/5.0

20. Understanding the User-Agent String

21. The sample user-agent string contains three tokens.
The Compatibility flag-indicates that Internet Explorer is compatible with a common set of features.
The Version token-identifies the browser and its version number.
The Platform token-identifies your operating system and its version number

22. Version token Description
MSIE Internet Explorer 8 (pre-release)
MSIE Internet Explorer 7
MSIE Microsoft Internet Explorer 6
MSIE 6.0b Internet Explorer 6 (pre-release)
MSIE Internet Explorer 5.5
MSIE Internet Explorer 5.01
MSIE Internet Explorer 5
MSIE 5.0b Internet Explorer 5 (pre-release)
MSIE Internet Explorer 4.01

23. Platform token Description
Windows NT Windows Vista
Windows NT Windows XP
Windows NT Windows 2000
Windows NT Microsoft
Windows Windows 98
Windows Windows 95

24. Req/Res from HTTP Analyser(www.google.com)

25. X-Requested-With
When AJAX request is made, X-Requested-With header with the value "XMLHttpRequest" is used.
EXAMPLE:
X-Requested-With: XMLHttpRequest

27. RESPONSE HEADERS

28. Server Identifies the server product.
Can contain additional comments about product.
Product tokens are listed in order of their significance for identifying the application.
If response is being forwarded through a proxy, the proxy application must not modify it.
EXAMPLE:
Server: Microsoft-Internet-Information-Server/1.0

29. Location
Used to redirect the client to a location other than the Request-URI for completion of the request or identification of a new resource.
For 201 (Created) responses, the Location is that of the new resource which was created by the request.
BASIC SYNTAX: Location: absoluteURL
EXAMPLE: Location:

30. Accept-Ranges
Used by servers to tell clients whether they accept requests for ranges of a resource.
EXAMPLES: Accept-Ranges: bytes
Servers that do not accept any kind of range request for a resource may send Accept-Ranges: none
to advise the client not to attempt a range request.

31. Public Allows a server to tell a client what methods it supports.
These methods can be used in future requests by the client.
Proxies need to be careful when they receive a response from a server with the public header.
The header indicates the capabilities of the server, not the proxy, so the proxy needs to edit or remove the list of methods.
EXAMPLE: Public: GET,HEAD,POST,TRACE

32. GENERAL HEADERS

33. Content-Type Tells media type of the object in the message. EXAMPLE:
Content-Type: text/html
Content-Type: application/x-www-form-urlencoded

34. The application/x-www-form-urlencoded type
Content-Type that GET requests must use, and POST requests use by default.
Involves percent encoding format that looks something like: %C3%86.
No official way of determining the character encoding of such a request, since the percent encoding operates on a byte level.

35. Connection
In HTTP 1.0, keep-alive means browser can handle persistent connection.
In HTTP 1.1, persistent connection is by default.
Persistent connections mean that the connection will stay open across transaction, until either the client or the server decides to close them.
EXAMPLE: Connection: close
This token means that the connection is going to be closed after the response is completed.

37. Transfer-Encoding To transfer the HTTP message body safely.
Its value is a list of encodings that were performed on the message body.
If multiple encodings were performed, they are listed in order.
EXAMPLE:
Transfer-Encoding: chunked

38. Transfer-Encoding/ Content-Encoding
Transfer encoding is performed by a server or other intermediary application .
Transfer encoding is applied for architectural reasons & are independent of format of content.
Content-Encoded message just encodes entity section of message while Transfer-Encoded encodes the entire message, changing the structure of the message itself.

39. Chunked Transfer-Encoding
If total size of response is not known, then chunked transfer-encoding is used (only for HTTP 1.1).
Server breaks the complete response into smaller chunks and sends them in series.
A chunked message body contains a series of chunks, followed by a line with "0" (zero), followed by optional footers (just like headers), and a blank line. Each chunk consists of two parts:
size of the chunk data, in hexadecimal followed by CRLF.
the data, followed by CRLF

40. HTTP Response Chunk#1 Chunk#2 Last Chunk
HTTP/ OK <CR><LF>
Content-Type: text/plain <CR><LF>
Transfer-Encoding: chunked<CR><LF>
Trailer:Content-MD5 <CR><LF>
<CR><LF>
HTTP Response
Response
stream
Chunk#1
14<CR><LF>
We hold these truths<CR><LF>
Chunk#2
12<CR><LF>
To be self-evident<CR><LF>
Last Chunk
0<CR><LF>
Trailer*
Content-MD5:njfhrjehn34jbnj5n<CR><LF>
optional

41. Chunked Response Example
HTTP/ OK
Date: Fri, 31 Dec :59:59 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
<blank line>
1a; ignore-stuff-here
abcdefghijklmnopqrstuvwxyz 10
abcdef
some-footer: some-value

42. Response From Page dumps Of A Test Run

43. Date Gives date & time at which the message was created.
This header is required in servers response because the time & date at which the server believes the message was created can be used by caches in evaluating the freshness of a response.
EXAMPLE: Date: Tue, 15 Nov :12:31 GMT

44. ENTITY HEADERS

45. Content-Language
Tells the client the natural language that should be understood in order to understand the object.
If this header is not present in the response, the object is intended for all audiences.
EXAMPLE:
Content-Language: en,fr

46. Content-Location
Included in an HTTP message to give the URL corresponding to the entity in the message.
For objects that may have multiple URLS, a response message can include a Content-Location header indicating the URL of the object used to generate the response.
BASIC SYNTAX:
Content-Location: ( absoluteURL| relativeURL)

47. Content-Encoding
Used to specify whether any encodings have been performed on the object.
Primarily used to allow a document to be compressed without losing the identity of its underlying media type.
EXAMPLE:
Content-Encoding: compress,gzip

48. Content-Length
This header gives the length or size of the entity body.
If the header is in a response message to a HEAD HTTP request, the value of the header indicates the size that the entity body would have been had it been sent.
EXAMPLE: Content-Length: 2417

49. Range
Requests for ranges of a document allow for more efficient requests of large objects or for recovery from failed transfers.
Can also be used to request multiple ranges.
EXAMPLE: Range:bytes=
Range:bytes=1500-
(requesting the remainder of the document after first bytes)

50. EXTENSION HEADER

51. Client-ip
Extension header used by some older clients & some proxies to transmit the IP address of the machine on which the client is running.
TYPE: Extension request header
EXAMPLE: Client-ip:

52. HTTP HEADERS SEND BY NETSTORM

53. NetStorm by default add following headers:
Accept
Host
User-Agent
Connection
Accept-Encoding
Keep-Alive
Content-Length
And along with the headers of the recorded script.

54. Following headers do not capture during Script Recording:
User-Agent
Accept-Encoding
Keep-Alive
Referer
Accept
Connection
Host header is added only in script.detail file but not in the script.capture file.

55. Headers in script.capture
case X153424_jpg:
think_time = pre_page_X153424_jpg();
web_url (X153424_jpg,
METHOD=GET, URL= /thumb/ jpg,
HEADER=Accept-Language: en-US,
HEADER=Cache-Control: no-transform,
HEADER=UA-CPU: x86,
NUM_EMBED=42);
next_page = check_page_X153424_jpg();
break;

56. Headers in script.detail
---- HTTP TX_RAT:0 RX_RAT:0
POST /asp/include/TeaLeafTarget.asp HTTP/1.1
Host:
Cookie: ENDECA=BCOrder=ArtStyle&bc0=c%7cphotography%7c23946;art=presc=1&
addtocart=N&asPurchased=false&sc=1;CustSessionID=E670CFE9345;sessionid=E670CFE9,
Accept-Language: en-us
Content-Type: text/xml
UA-CPU: x86
Content-Length: 1779
Cache-Control: no-cache
<ClientEvent count="1“ Type="PERFORMANCE" SubType="INIT" PageId="ID18H26M37S591"TimeDuration="3753" DateSince1970=" " >
</ClientEvent>
----

57. You can also add headers in script.capture file for script

58. Headers in script.detail file

59. For performance purposes we can also disable headers by adding following keywords in scenario file.
G_DISABLE_ACCEPT_HEADER
G_DISABLE_HOST_HEADER
G_DISABLE_UA_HEADER
G_DISABLE_CONNECTION_HEADER
G_DISABLE_ACCEPT_ENC_HEADER
G_DISABLE_KA_HEADER
G_DISABLE_ALL_HEADER

60. Referer Header By default, NetStorm do not send Referer header.
You can add this header in scenario file by keyword
G_ENABLE_REFERER

61. How Host Header filled by NetStorm
Case 1: Recorded Host, no Server mapping.
For example: URL is then the Host header is only.

62. Case 2: Recorded Host, simple server mapping.
For example: if recorded host is & following simple mapping is used in scenario file:
SERVER_HOST –
then the Host header will be Host:

63. Case 3: Recorded Host, complex mapping.
For example: if recorded host is & following complex mapping is used in scenario file:
SERVER_HOST – –
then the NetStorm will randomly select one IP from these two IP’s & that IP will be the Host header value.

64. Sometimes you may need Recorded Host in Host header.
For example, HPD supports virtual host.
To enable Recorded Host in Host header, add this keyword in scenario file
G_USE_RECORDED_HOST_IN_HOST_HDR
This will give the recorded host in host header irrespective of server mapping.

65. HTTP HEADERS SEND BY NETOCEAN

66. NetOcean by default sends following headers:
Content-Length
Content-Type (If file extension is not in extension.type of conf directory, then Content-Type header is not send in the URL response.)

67. Static/CGI URL
You can also add headers for static/CGI URL which are as follow:
1) GLOBAL HEADER: To send additional headers for all URL accessed, HPD supports additional global headers. These headers can be added in the global_headers file in following directory
cd /var/www/hpd/conf

68. 2) URL SPECIFIC HEADERS FOR STATIC URLS:
You can add additional headers based on the URL in URL control file.
Go to this directoy
/var/www/hpd/control/default/
Make directory as per the path of the URL. E.g.
/custom/samples/url_specific_hdrs/url_specific_hdr.html  
Add desired header in URL control file.
For example:
Content-Encoding: gzip

69. Correlation URL Global header for all correlation Services
Add in correlaton.conf
Service based additional headers
Add in the service.conf file of the service.
For example:
SET_CR_URL_HEADER MyHdr: value

70. Questions?? What are Quality values?
What is the default value of quality values?
What is Trace method?
What is the reason phrase for 201 status code?
What is the name of the header whose value is application/x-www-form-urlencoded?
If the value of Accept-Encoding header is empty then what does it mean?
If no valid host is determined through URL & Host header then what status code will be send by the server?
What are tokens of the user-agent string in User-Agent header?

71. Questions??
How the Transfer-Encoding header differs from the Content-Encoding?
What does each chunk contains in the Chunked Tranfer- Encoding?
What is the form of length value in a chunk of Chunked Tranfer-Encoding?
If the value of Max-Forwards header is zero when application receives the request, then what response should it send back ?

72. THANK YOU