1. E-Commerce Architectures and Technologies Rob Oshana Southern Methodist University

2. Solving Performance Models of E-Business Sites

3. Introduction Solution technique issues –Simplicity –Accuracy –Computational cost –Availability of information about the system under analysis We will deal with simplicity

4. Introduction Two categories of models –System model; treat the system as a black box (function theory) –Component level model; explicitly represent the internal components of the system CMBG; models navigational patterns CSID; describes logical architecture

5. CBMG for the Customer of the Online Toy Store entry homesearch 1.0 0.6 0.4

6. CSID for Search Function of the Online Toy Store C C [1,m1][0.95,m3][1,m4][1,m5] WS C DBWS [0.05,m2] 12 3 45 6 1 -> 2 -> 4 -> 5 -> 6 search function 1 -> 2 -> 3 server overloaded, refused

7. Input Data for the Simple Problem

8. A Performance Modeling Question DB Servers (e.g.mainframes) Intranet/Internet DMZ Layer 1Layer 2Layer 3 Load Balancer Router App Servers Web Servers Firewall Model ?

9. A Modeling Approach Level of detail of a model depends on the questions you are trying to answer Throughput function, Xo(k); gives average throughput of the box as a function of the number of requests, k, in the box –Abstract the internal details of the site architecture

10. Single Queue Approach Single queue model; most elementary form of queuing model Characterize by –Arrival process –Service process –Space for holding the queue of waiting requests –Number of resources that render the service

11. Single Server Model Service process Queuing Space Arrival process Resources

12. Single Queue Approach Queue; a resource (processor, I/O, network) and the queue of requests waiting to use the resource Characterized by S(n); represents the average service time per request with n requests in the queue Number of requests waiting is the queue length

13. Single Queue Approach Load independent resources; average service time does not depend on the load, S(n) = S for all n Load dependent resources; average service time depends on the load, S(n) arbitrary function of n Delay resources; no queuing. Total time is requests service time, and S(n) = S for all n

14. Example Search Engine Server of an online newspaper User population unknown and very large –Arrival rate of requests for search service not influenced by requests that have arrived already and are being serviced (infinite population case)

15. Definitions and Assumptions Homogeneous workload assumption; all requests are statistically indistinguishable, requests present in the server are not important, only the number of requests that are present counts Average service function is a constant (does not depend on number of requests in the system) Average throughput of server, Xo = Ц request/sec

16. Definitions and Assumptions Servers service rate function; –Physical characteristics (processor, etc) –Demands of the workload (DBMS, complexity of search operation, etc) Assume search engine server does not refuse any requests –Infinite queue assumption; enough space for all waiting requests Operational equilibrium; number of requests present in system at the start of observation interval same as at the end

17. Definitions and Assumptions Arrival rate of requests at server = λ requests/sec Service rate = Ц requests/sec or 1/S (service time) Want to compute p k (probability that there are k (k=0,1,..) requests in the server as well as other measures

18. Model Solution Performance metrics required; –Queue length –Average response time –Utilization Look for a general solution State description; number of requests present in the server (waiting or receiving service) Past does not matter (memoryless or Markovian assumption)

19. Model Solution States of the system can be represented as integers, 1, 2, 3,..n State transition diagram can be developed –Each circle is a different state of the system –Transitions between states are physical events and are represented by arrows

20. Model Solution Arrival of new request when the system has k events transitions the system to state k+1 Request completing transitions from k to k-1 Flow equilibrium equation; flow-in = flow-out

21. STD – Infinite Population/Infinite Queue 0 21 k Number of requests in system λ λλ λλ ЦЦЦЦЦ........

22. STD with Boundaries 0 21 k λ λλ λλ ЦЦЦЦЦ........ boundary

23. Model Solution flow-in = flow-out Ц p1 = λ p0 Ц p2 = λ p1 Ц pk = λ pk - 1

24. Model Solution p0 = 1 - λ / Ц This series only converges (has a finite sum) if λ / Ц < 1 (average arrival rate of requests is smaller than the service rate)

25. Example Search engine server 144,000 requests/hour from home page servers Server takes 0.02 sec to process each search request What is server utilization ? How many search requests are on average on the server?

26. Example Server can process Ц requests in 1 sec, one request takes 1/ Ц to complete Service request = 1/ Ц = 1/0.02 = 50 requests/sec Average arrival rate λ = 144,000 request/hour, 40 requests/sec (144,000/3,600) Fraction of time search server idle; 1 – (λ / Ц ) = 1 – (40/50) = 1 – 0.8 = 20%

27. Example Server is utilized 1 – p0 = λ / Ц = 80% of the time Fraction of time there are k requests at the server; Pk = (1 - λ / Ц) (λ / Ц)**k = 0.2 X 0.8**k Utilization, U of the server = 1 – p0 = λ / Ц pk = (1-U)U**k (state distribution depends only on the utilization and not on the individual values of the arrival and service rates)

28. Example Average number of requests N = U / (1-U) = 0.8 / (1 – 0.8) = 4.0 Also, average arrival rate is equal to the average departure rate X = λ

29. Example Average response time, R R = N/X = (U/ λ)/(1-U) = (1/ Ц)/(1-U) = S/(1-U), where S = 1/ Ц is average service time of a request at the server

30. Example Summary When utilization of the server is very low (U close to 0), the average response time = average service time –Expected since no time is spent queuing due to the presence of other requests When U is high (close to 1) denominator goes to 0, R goes to infinity

31. More calculations Average response time at server; R = (1/50) / (1 – 0.8) = 0.1 sec If server is twice as fast; Ц = 100 requests/sec, U = 40/100 = 0.4 R = (1/100) / (1 – 0.4) = 0.017 sec (reduced by 17% of its original value)

32. More calculations If both the arrival rate and the service rate are doubled, U remains the same, U = 0.8 R = (1/100) / (1 – 0.8) = 0.05 sec

33. Another Example Mail server 3,500 requests (sendmail) during working days Concentration during a peak hour –Avg interarrival time = 2.5 sec –Avg interrarrival time at peak = 0.8 sec –Avg message size = 8.2 Kbytes –Avg message size during peak = 7.1 Kbytes

34. Another Example SMTP service processes 2.2 typical messages per second Compute avg response during peak; From simple queue model; Λ = 1 / interrarrival time = 1 / 0.8 = 1.25 messages/sec Ц = 2.2 messages/sec

35. Another Example Mail sever utilization; U = Λ / Ц = 1.25 / 2.2 = 0.568 Average request response time; R = (1 / 2.2) / (1 – 0.568) = 1.05 sec

36. Another Example Assume doubling the mail size increases service time 35% New service time; = (1 / Ц ) X 1.35 = 0.454 X 1.35 = 0.613 sec U = 1.25 X 0.613 = 0.766 R = (0.613) / (1 – 0.766) = 2.62 sec (doubling average mail size increases response time by 149%)

37. Single Queue Model Model Responses Requests Single Queue Web server Data Storage device Requests/ responses

38. Single Queue Model “Whole” e-business site viewed as a black box that receives requests and processes them one at a time –Pessimistic assumption –Multiprogramming allows more Input parameters to calculate RT; –Average service time –Average arrival rate

39. Question What would the response time be if the number of requests during the peak hour grew by a factor of four?

40. Single Queue Model S = 0.048 sec Λ = 18000/3600 = 5 requests/sec U = S X Λ –S is average service time –Λ is average arrival rate Average response time = U = S X Λ = 0.048 X 5 = 0.24

41. Single Queue Model R = S / (1 – U) = 0.048 / (1 – 0.24) = 0.063 sec Average response time for new arrival rate; Λnew = 4 X Λ = 4 Xx 5 = 20 requests/sec U = S X Λ = 0.048 X 20 = 0.96

42. Single Queue Model R = S / (1 – U) = 0.048 / (1 – 0.96) = 1.2 sec Thus, multiplying the arrival rate by four increases the response time by 1900%

43. A more realistic approach: A queuing network Usually there is more than 1 web server Therefore can’t model with a single queue model Must model the most relevant components of the system Use a queuing network model

44. Queuing Network Models Various queues that represent a system are interconnected Network of queues called a queuing network (QN) Level of detail depends on available resources A queue in a QN represents a resource (WS, DS, processor, disk)

45. Queuing Network Models Queuing Network Model (QNM) arranged in the same configuration as the real system These queuing networks can be used to represent systems Connections represent flows of information Service function S(n) used

46. Queuing Network Model Intranet/Internet DMZ Layer 1Layer 2Layer 3 Load Balancer Router App Servers Web Servers Firewall

47. Queuing Network Model

48. Closed models; models that have a fixed number of requests per class Open models; no limits on the number of requests present in the system –Allow requests to arrive, go through the various resources of the site, and leave the system

49. Single Class Open Models All resources are either delay or load independent resources Λ; average arrival rate of requests to the QN K; number of queues Xo; average throughput of the QN (Xo = Λ for equilibrium) Vi; avg # of visits to queue i by request

50. Single Class Open Models Si; avg service time of a request at queue i per visit to the queue Wi; average waiting time of a request at queue i per visit to the queue Xi; average throughput of queue i Ri; average response time of a request at queue I (Ri = Wi + Si)

51. Single Class Open Models R´i; average residence time of a request at queue i (R´i = Vi X Ri) Ro; average response time and equal to the sum of the residence times over all queues Ro = Σ R´i, I = 1,K n i ;average number of requests at queue I waiting or receiving service from any resource at queue i N; average number of requests in the QN

52. Single Class Open Models Arrival Theorem; the average number of requests seen upon arrival to queue i is equal to the average number, n i, of requests in the queue Ri = Si + n i X Si From Littles Law; n i = Xi X Ri Also, Ui = Xi X Si Therefore Ri = Si / (1 – Ui)

53. Single Class Open Models R´i = Di / (1 – Ui) Average number of requests at queue I; n i = Ui / ( 1 – Ui )

54. Example Online investment company Web server accessed by clients through a browser App server that receives transaction requests generated by web server Database server

55. Financial Site: Architecture Customer browser Database server App server Web server internet 100 Mbps

56. Financial Site: CSID for “Show Portfolio” C WSASDBASWS C C 1 2 3 4 5 6 78 [1,m1] [0.95,m3] [0.8,m6][1,m7][1,m8][1,m9] [0.05,m2]

57. Open QN of the Financial Site 1 2 processor disk 3 4 processor disk 5 6 processor disk Web server App server Database server responses

58. Input Data for the Financial Site

59. Example Calculate response time of portfolio transaction Assume Λ = 9 Rsp = R´1 + R´2 + R´3 + R´4 + R´5 + R´6 R´i = Di / ( 1 – Ui )

60. Example Ui = Xo X Di Xo = Λ (for open models) U1 = Λ X D1 = 9 X 0.019 = 0.171 R´1 = 0.019 / ( 1 – 0.171 ) = 0.023 sec

61. Response Time of Financial Site

62. Single-Class Closed Model Situations where we want to model systems with a fixed and finite number of requests in the system –Maximum level of multiprogramming under heavy load (multithreaded web server, or intranet system with a number of clients sending requests to WS) Solution technique is Mean Value Analysis

63. Single-Class Closed Model Denote variables as functions Ri(n) is residence time at queue i when there are n requests in system Compute Response time per visit to resource i; Ri(n) = Si + Wi(n) = Si + n i (n) X Si = Si [ 1 + n i (n) ], n i (n) = average number of requests found in the queue by an arriving request

64. Single-Class Closed Model Arrival Theorem (applied to closed systems); avg # of requests seen upon arrival to queue i when there are n requests in the QN is equal to the avg # of requests in queue i in a QN with n-1 requests (with the arrival request to queue i removed from the QN – cannot find itself in the queue) n i (n) = n i ( n – 1)