1. Ó 1998 Menascé & Almeida. All Rights Reserved.1 Part III Web and Intranet Performance Issues (Book, chap. 4)

2. Ó 1998 Menascé & Almeida. All Rights Reserved.2 Learning Objectives Present server architecture and performance issues Discuss perception of performance Introduce Web infrastructure components Discuss Web server workload Examine bandwidth, latency, and traffic in the Web Introduce capacity planning questions

3. Ó 1998 Menascé & Almeida. All Rights Reserved.3 Web Server Performance Problems Unpredictable nature of information retrieval and service request over the World-Wide web load spikes: 8 to 10 greater than avg. high variability of document sizes: from 10 3 to 10 7 bytes

4. Ó 1998 Menascé & Almeida. All Rights Reserved.4 Web Server Elements hardware O.S. TCP/IP HTTP server Contents:. HTML. graphics. audio. video. other

5. Ó 1998 Menascé & Almeida. All Rights Reserved.5 HTTP request Method (GET, HEAD, PUT, POST) URI (Universal Resource Indicator), actually URL (a sub-set of URI) Protocol version Additional information (optional): types of documents accepted, authentication, etc.

6. Ó 1998 Menascé & Almeida. All Rights Reserved.6 HTTP response Status line Metainformation (description about information sent) Information (document file, or output from server-side (servlet) application)

7. Ó 1998 Menascé & Almeida. All Rights Reserved.7 Combination of HTTP and TCP/IP HTTP defines a request-response interaction; HTTP is a ``stateless’’ protocol; one connection per object; TCP connection setup overhead; mandatory delays due to the protocols; small Web objects and the TCP ``slow start’’ algorithm

8. Ó 1998 Menascé & Almeida. All Rights Reserved.8 HTTP request-response steps map the server to an IP address (DNS); establish a TCP/IP connection with the server; transmit the request (URL,method,etc); receive the response (HTML text or other information); close the TCP/IP connection.

9. Ó 1998 Menascé & Almeida. All Rights Reserved.9 HTTP 1.0 interaction 0 RTT 1 RTT 2 RTT 3 RTT 4 RTT TCP conn. client sends HTTP req. client parses HTML doc. client sends req. for image image begins to arrive syn ack dat ack syn dat Server residence time Server residence time

10. Ó 1998 Menascé & Almeida. All Rights Reserved.10 HTTP 1.1 interaction 3 RTT 0 RTT 1 RTT image begins to arrive syn ack dat ack dat Server residence time Server residence time TCP conn. client sends HTTP req client parses HTML doc. client sends req. for image ack 2 RTT

11. Ó 1998 Menascé & Almeida. All Rights Reserved.11 HTTP 1.0 and 1.1 interaction 0 RTT 1 RTT 2 RTT 3 RTT 4 RTT TCP conn. client sends HTTP req. client parses HTML doc. client sends req. for image image begins to arrive syn ack dat ack syn dat HTTP 1.0HTTP 1.1 Server time Server time 0 RTT 1 RTT 3 RTT image begins to arrive syn ack dat ack dat Server time Server time TCP conn. client sends HTTP req client parses HTML doc. client sends req. for image ack 2 RTT

12. Ó 1998 Menascé & Almeida. All Rights Reserved.12 Hardware, O.S. Processors (speed, number) Memory (capacity) Disks (speed, capacity) NIC: bandwidth OS reliability, performance, scalability, robustness; TCP/IP implementation!

13. Ó 1998 Menascé & Almeida. All Rights Reserved.13 Contents Contents can heavily impact on performance: web site popularity size of user community (extranet - hard to say; intranet - can be determined/controlled) overall size and type of objects (MM files are 100 to 10,000 times bigger as HTML documents)

14. Ó 1998 Menascé & Almeida. All Rights Reserved.14 Where are the delays? Browser –R’ browser Network –R’ network Server –R’ server User response time: R r –R r = R’ browser + R’ network + R’ server or –R r = R’ cache

15. Ó 1998 Menascé & Almeida. All Rights Reserved. Anatomy of an HTTP transaction End user Client Browser NetworkServer click Data returned from cache Display HTTP Request Data R’ s C R r N1 R’ N2 Server residence time R’ B

16. Ó 1998 Menascé & Almeida. All Rights Reserved.16 Usually R’ cache << R’ network + R’ server p c denotes the fraction of times the data are found in the local cache R’ cache : response time when the data are found in a local cache Exp(R r ) = p c x R’ cache + (1-p c ) x R r Average Response Time

17. Ó 1998 Menascé & Almeida. All Rights Reserved.17 Impact of the Browser’s Cache 20% of the requests are serviced by the local cache local cache response time = 400 msec average response time for remote Web sites = 3 seconds R = p c x R’ cache + (1-p c ) x R r R = 0.20x 0.4 + (1-0.20) x 3.0 R = 2.48 sec

18. Ó 1998 Menascé & Almeida. All Rights Reserved.18 Impact of the Browser’s Cache What if we increase the size of the local cache? Previous experiments show that tripling the cache size would raise the hit ratio to 45%. Thus, R = p c x R’ cache + (1-p c ) x R r R = 0.45x 0.4 + (1-0.45) x 3.0 R = 1.83 sec

19. Ó 1998 Menascé & Almeida. All Rights Reserved.19 Bottlenecks As the number of clients and servers grow, overall performance is constrained by the performance of some components along the path from the client to the server. The components that limit system performance are called bottlenecks

20. Ó 1998 Menascé & Almeida. All Rights Reserved.20 Example of a Bottleneck A home user is unhappy with access times to Internet services. To cut response time down, the user is considering replacing the processor of his/her desktop with one twice as fast. What will be the response time improvement if I upgrade the speed of my desktop computer?

21. Ó 1998 Menascé & Almeida. All Rights Reserved.21 Example of a Bottleneck (cont’d) for an average page: avg. network residence time: 7,500 msec avg. server residence time: 3,600 msec avg browser time: 300 msec R r = R’ browser + R’ network + R’ server = 300+7,500+3,600 R r = 11,400 msec = 11.4 sec

22. Ó 1998 Menascé & Almeida. All Rights Reserved.22 Example of a Bottleneck (cont’d) Percentage of time: %x = R’ x / (R’ browser + R’ network + R’ server ) browser = 300/11,400 = 2.14 % network = 7,500/11,400 = 65.79 % server = 3,600/11,400 = 31.57 %

23. Ó 1998 Menascé & Almeida. All Rights Reserved.23 Example of a Bottleneck (cont.) The CPU upgrade affects mainly the browser time: R’ N browser ~ 1/2 x R’ browser = 1/2 x 300 = 150 msec R N r = R’ N browser + R’ network + R’ server R N r = 150 + 7,500 + 3,600 = 11.25 sec. Therefore if the speed of the PC were doubled, the response time would decrease only by R r /R N r = 11.40/11.25 = 1.3%

24. Ó 1998 Menascé & Almeida. All Rights Reserved.24 WEB workload is hard to forecast (WEB performance depends heavily on workload) Number of WEB clients: 10 7 and rising Randomness of visits large diversity in products (browsers, servers, cache/proxies) unpredictable network delays, due to netw. bandwidth and congestion

25. Ó 1998 Menascé & Almeida. All Rights Reserved.25 Perception of Performance WWW user: fast response time no connection refused Web administrators: high throughput high availability Need for quantitative measurements

26. Ó 1998 Menascé & Almeida. All Rights Reserved.26 WWW Performance Metrics (I) connections/second Mbits/second response time user side server side errors/second

27. Ó 1998 Menascé & Almeida. All Rights Reserved.27 WWW Performance Metrics (II) Web site activity indicators Visit: a series of consecutive Web page requests from a visitor within a given period of time. Hit: any connection to a Web site, including in-line requests, and errors. Request = successful hit. Metrics hits/day visits/day unique visitors/day pages views/day

28. Ó 1998 Menascé & Almeida. All Rights Reserved.28 WWW Performance Metrics (III) Web Advertising Measurements Exposure metrics (visits/day, pages/day) site exposure page exposure banner exposure Interactivity metrics visit duration time inter-visit duration visit depth (total # of pages a visitor is exposed to during a single visit to a Web site)

29. Ó 1998 Menascé & Almeida. All Rights Reserved.29 Example of Performance Metrics The Web site of a travel agency was monitored for 30 minutes and 9,000 HTTP requests were counted. We want to assess the server throughput. 3 types of Web objects –HTML pages: 30% and avg. size of 11,200 bytes –images: 65% and avg. size of 17,200 bytes –video clips: 5% and avg. size of 439,000 bytes

30. Ó 1998 Menascé & Almeida. All Rights Reserved.30 Example of Performance Metrics Throughput in terms of requests: –(No. of requests)/(period of time) = –9,000/(30 x 60) = 5 requests/sec In terms of bits/sec per class –(total requests x class % x avg. size) / (period of time)

31. Ó 1998 Menascé & Almeida. All Rights Reserved.31 Example of Performance Metrics HTML throughput (Kbps) 9,000 x 0.30 x (11,200 x 8) / 1,800 = 131.25 Image throughput (Kbps) 9,000 x 0.65 x (17,200 x 8) / 1,800 = 436.72 Video throughput (Kbps) 9,000 x 0.05 x (439,000 x 8) / 1,800 = 857.42 Total throughput 131.25 + 436.72 + 857.42 = 1,425.39 Kbps

32. Ó 1998 Menascé & Almeida. All Rights Reserved.32 Quality of Service As Web sites become a fundamental component of businesses, quality of service will be one of the top management concerns. The quality of the services provided by a Web environment is indicated by its service levels, namely: response time availability reliability cost

33. Ó 1998 Menascé & Almeida. All Rights Reserved.33 Quality of Service To a user, the source of problems is immaterial users want performance, security, 24H x 7D uptime The problem of quality of service on the Web is exacerbated by the unpredictable nature of interaction of users with Web services. It is usual to see the load of a Web site being multiplied by 8 on the occurrence of a special event. How does management establish the service levels of a Web site?

34. Ó 1998 Menascé & Almeida. All Rights Reserved.34 Quality of Service Typical questions to help to establish the service level of a Web service: –Is the objective of the Web site to provide information to external customers? –Do your mission-critical business operations depend on the World Wide Web? –Do you have high-end business needs for which 24 hours-a-day, 7 days-a-week uptime and high performance are critical, or can you live with the possibility of Web downtime?

35. Ó 1998 Menascé & Almeida. All Rights Reserved.35 QoS metrics examples Dial-up call failure rates Time it takes to log in Web site throughput Web site time-out rate Capacity planning for peak periods is fundamental, due to high variability of workload

36. Ó 1998 Menascé & Almeida. All Rights Reserved. Web infrastructure INTERNET - TCP/IP INFRASTRUCTURE Firewall +Cache/Proxy IntranetTCP/IPIntranetTCP/IP Desktop Computer Browser O.S. Network O.S. Network Hardware Private Web Site Private Web Site HTTP Server O.S. - TCP/IP Hardware Desktop Computer Browser O.S. Network O.S. Network Hardware Desktop Computer Browser O.S. Network O.S. Network Hardware Public Web Site Public Web Site HTTP Server O.S. - TCP/IP Hardware Mirror Web Site Mirror Web Site HTTP Server O.S. - TCP/IP Hardware

37. Ó 1998 Menascé & Almeida. All Rights Reserved.37 Web Proxy Architecture Clients Proxy Server Servers

38. Ó 1998 Menascé & Almeida. All Rights Reserved.38 Web Proxy Architecture Originally introduced for users behind a Firewall. A proxy acts as an agent, representing the server to the client and the client to the server. A proxy accepts request from clients and forwards them to Web servers. Once a proxy receives responses from remote servers, it passes them to clients. Proxies can be configured to cache relayed responses, becoming then a caching proxy.

39. Ó 1998 Menascé & Almeida. All Rights Reserved.39 Web Caching Proxies (I) Caches are always used hierarchically: –OS / disk controller; browser (local) –caching proxy (network) Caching improves speed and network costs, bringing data closer to users + reducing server load Metrics for cache effectiveness: –(document) hit ratio –byte hit ratio –data transferred

40. Ó 1998 Menascé & Almeida. All Rights Reserved.40 Web Caching Proxies (II) Caching adds its own set of problems: –how to be sure that document is up-to-date? (cache consistency, or cache coherence): Expires MIME header (optional) HEAD HTTP request GET if-modified-since - 304 status mesg. (Not Modified) –what documents are worth caching? –For how long?

41. Ó 1998 Menascé & Almeida. All Rights Reserved.41 Web Caching Proxy: an example A large company decided to install a caching proxy server on the corporate intranet. After 6 months of use, management wanted to assess the caching effectiveness. So, we need performance metrics to provide quantitative answer for management. Cache A: we have a cache that only holds small documents, with average size equal to 4,800 bytes. The observed hit ratio was 60%. Cache B: the cache management algorithm was specified to hold medium documents, with average size of 32,500 bytes. The hit ratio was 20%

42. Ó 1998 Menascé & Almeida. All Rights Reserved.42 Web Caching Proxy: an example The proxy was monitored during 1 hour and 28,800 requests were handled in that interval. Let us compare the efficiency of the two cache strategies by the amount of saved bandwidth SavedBandwidth = (No-of-Req.  Hit-Ratio  Size)/Int. SavedBandwidth-A = (28,800  0.6  4,800  8)/3,600 = 180 Kbps SavedBandwidth-B = (28,800  0.2  32,500  8)/3,600 = 406.3 Kbps

43. Ó 1998 Menascé & Almeida. All Rights Reserved.43 Web Mirrors Implies replicating site contents at other servers Requires regular, usually automatic redistribution of contents updates Best combined with DDNS (dynamic DNS) to redirect browsers Purpose: –increase availability, minimize geographical distances –balance server load (DDNS)

44. Ó 1998 Menascé & Almeida. All Rights Reserved.44 Web Server Architecture Need to service more requests at a time –fork a copy per client –multi-thread server program –pool of running services Need to split traffic among machines –DDNS (round-robin; other strategies) –single-IP-image –dynamic HTTP redirection

45. Ó 1998 Menascé & Almeida. All Rights Reserved.45 Workload of Web servers (I) Has unique characteristics: –dynamic WEB pages client-side programs (applets) –JAVA, ActiveX, Jscript, Vbscript, Dynamic HTML server-side programs (servlets) –can severely impact on server performance –JAVA, ActiveX, PERL, C/C++, Shell, ASP, PHP –CGI - Common Gateway Interface, universal interface for non-embedded applications –ISAPI (W/NT only) uses threads to reduce overhead

46. Ó 1998 Menascé & Almeida. All Rights Reserved.46 Workload of Web servers (II) Has unique characteristics (2): –extreme variability notion of class of requests (clustering) - e.g. –small pages (less than 5 kB) –medium documents (5 to 50 KB) –large files (more than 50 KB) Zipf’s law: f ~ 1/  f = frequency of use;  rank of popularity

47. Ó 1998 Menascé & Almeida. All Rights Reserved.47 Workload: dynamic Web pages The Web site of a virtual bookstore receives an average of 20 visitors per second. One out of 10 visitors places an order for books. Each order transaction generates a CGI script, which is executed on the Web server. The Webmaster wants to know what is the CPU load generated by the CGI script. Consider that the average CPU service demand of a CGI script is: D cpu = 120 msec./CGI Using the Service Demand Law: U cpu = X cgi  D cpu

48. Ó 1998 Menascé & Almeida. All Rights Reserved.48 Workload: dynamic Web pages X cgi = cgi = VisitRate  PercentageOfOrders = 20  (1/10) = 2 CGI/sec U cpu = 2  0.12 = 0.24 = 24% What would be the impact of replacing the CGI applications by servlets? Let us assume that Java servlet transactions are 30% less resource-intensive than CGI applications. The CPU utilization due to servlets would be U cpu = 2  0.084 = 0.168 = 16.8%

49. Ó 1998 Menascé & Almeida. All Rights Reserved.49 Novel Features in the WWW The Web exhibits extreme variability in workload characteristics: –Web document sizes vary in the range of 10 2 to 10 6 bytes –The distribution of file sizes in the Web exhibits heavy tails. In practical terms, heavy-tailed distributions indicate that very large values are possible with non-negligible probability. –This reduces statistical meaning of measures, which can be improved by using request classes. Web traffic exhibits a bursty behavior –Traffic is bursty in several time scales (factors of 8 to 10). –It is difficult to size server capacity and bandwidth to support demand created by load spikes.

50. Ó 1998 Menascé & Almeida. All Rights Reserved.50 Novel Features in the WWW The manager of the Web site of a large publishing company is planning the capacity of the network connection. 1 million HTTP operations per day average document requested was 10 KB The required bandwidth (Kbps) is: HTTP op/sec  average size of documents (KB) 11.6 HTTP ops/sec  10 KB/HTTP op = 928 Kbps Assume that protocol overhead is 20%

51. Ó 1998 Menascé & Almeida. All Rights Reserved.51 Novel Features in the WWW The actual throughput required is 928  1.20 = 1.114 Mbps which can be provided by a T1 connection. Assume that management decided to plan for peak load. The hourly peak traffic ratio observed was 5 for some big news event. Then the required bandwidth is: 1.114  5 = 5.57 Mbps which requires four T1 connections.

52. Ó 1998 Menascé & Almeida. All Rights Reserved.52 Capacity Planning of Web Servers It can be used to avoid some of the obvious and most common pitfalls: site congestion and lack of bandwidth. Typical capacity planning questions: Is the corporate network able to sustain the intranet traffic? Will Web server performance continue to be acceptable when twice as many people visit the site? Are servers and network capacity adequate to handle load spikes?

53. Ó 1998 Menascé & Almeida. All Rights Reserved.53 Part III: Summary Web server problems Combination of HTTP and TCP/IP Basic examples using operational analysis Bottlenecks Perception of performance and metrics Quality of Service Web caching proxy