Closed Queuing Networks (Mean Value Analysis)

Closed Queuing Networks Arise in two situations Arise in two situations When “source” of requests is explicitly modeled. That means, there is one node that models the clients. When “source” of requests is explicitly modeled. That means, there is one node that models the clients. When there is a limit on the number of requests that can circulate in a server network, and the assumption is that this limit has been reached and being maintained due to a high load. When there is a limit on the number of requests that can circulate in a server network, and the assumption is that this limit has been reached and being maintained due to a high load. (Implicitly, there is a queue at the “entrance” to the server network which is always non-empty). (Implicitly, there is a queue at the “entrance” to the server network which is always non-empty).

Consider the “client-server” situation (class example)

Parameters of the queuing network ➲ For this queuing network, the parameters are: ➲ Load parameters: Number of clients (M), client think time (EXP(lambda)) ➲ Think time is assumed to be exponential ➲ Server parameters: Routing matrix (X), service rates (mu_i) for all the servers ➲ Service time is assumed to be exponential

Metrics of the queuing network Given server parameters, and think timr we may be interested in: Given server parameters, and think timr we may be interested in: How many clients can the server system support? How many clients can the server system support? What is the maximum achievable throughput of the system? What is the maximum achievable throughput of the system? What is the bottleneck server? What is the bottleneck server?

Analysis Technique Calculate relative visit counts Calculate relative visit counts Apply Mean Value Analysis Apply Mean Value Analysis

Relative Visit Counts Since there is no actual notion of “exit” from the network, in the case of closed QNs, we cannot really find “expected number of visits until exit” from the network Since there is no actual notion of “exit” from the network, in the case of closed QNs, we cannot really find “expected number of visits until exit” from the network We can only talk about the number of visits made to a server S i relative to the number of visits made to a server S j We can only talk about the number of visits made to a server S i relative to the number of visits made to a server S j Thus visit counts in closed QNs are called relative visit counts Thus visit counts in closed QNs are called relative visit counts

Relative Visit Counts Suppose V_i ‘s are the expected number of visits made to node i during a certain observation period. Suppose V_i ‘s are the expected number of visits made to node i during a certain observation period. Then as explained in class, the equation Then as explained in class, the equation V = V X V = V X expresses the relationships between the visit counts expresses the relationships between the visit counts This is a linearly dependent set of equations This is a linearly dependent set of equations

Relative Visit Counts Thus, we can choose to express V_i’s in terms of one “chosen” V_j. Thus, we can choose to express V_i’s in terms of one “chosen” V_j. In the example given, it makes sense for this chosen V_j to be V_C, the visit count to the client node In the example given, it makes sense for this chosen V_j to be V_C, the visit count to the client node Further, if we set the value for this V_C to 1, the relative visit counts to the other servers have a good “explanation” Further, if we set the value for this V_C to 1, the relative visit counts to the other servers have a good “explanation” They are the number of visits made to the server nodes by a request from a client, before going back to the client node They are the number of visits made to the server nodes by a request from a client, before going back to the client node

Relative Visit Count Calculation As done in class As done in class Once we have the relative visit counts for the case of V_C=1, we can find response time, if we knew the per-visit response time of a request to a node Once we have the relative visit counts for the case of V_C=1, we can find response time, if we knew the per-visit response time of a request to a node

Arrival Theorem For finding this, we use a very big result in queueing systems, called the “Arrival Theorem”. It states for a closed QN with “load level” M, a request arriving at a node i “sees” the state of the queue as if the queuing network has one less customer in the system. For finding this, we use a very big result in queueing systems, called the “Arrival Theorem”. It states for a closed QN with “load level” M, a request arriving at a node i “sees” the state of the queue as if the queuing network has one less customer in the system. In other words, it sees the state of the system as if it itself is not there in the system, but just observing the system In other words, it sees the state of the system as if it itself is not there in the system, but just observing the system

Mean Value Analysis: Terms Each of the following are defined for a load level M Each of the following are defined for a load level M N i (M) = average queue length at node i N i (M) = average queue length at node i T i (M) = throughput at node i T i (M) = throughput at node i R i (M) = response time at node I R i (M) = response time at node I T sys (M) = system throughput T sys (M) = system throughput R sys (M) = Actual response time R sys (M) = Actual response time R cycle (M) = Cycle Time R cycle (M) = Cycle Time Note that T sys (M) and R sys (M) are values that are relative to the visit counts. Other values are not relative to visit counts. Note that T sys (M) and R sys (M) are values that are relative to the visit counts. Other values are not relative to visit counts.

Mean Value Analysis Assume that we know all the above quantities at level M-1. Can we derive them at M? Then we have a recursion. Assume that we know all the above quantities at level M-1. Can we derive them at M? Then we have a recursion. Using arrival theorem, we can write the response time of a request arriving at node i as: Using arrival theorem, we can write the response time of a request arriving at node i as: R i (M) = [ 1 + N i (M-1)] * tau_i R i (M) = [ 1 + N i (M-1)] * tau_i This is because arriving request is expected to “see” N i (M-1) requests in front of it which must be processed before it, and then its own service time must be counted for calculating response time. This is because arriving request is expected to “see” N i (M-1) requests in front of it which must be processed before it, and then its own service time must be counted for calculating response time. Note that the client node is an exception. For the client node Note that the client node is an exception. For the client node R C (M) = think_time, for all load levels M R C (M) = think_time, for all load levels M

MVA…contd Now we have response time at each node, and we have the visit counts. Thus Now we have response time at each node, and we have the visit counts. Thus R cycle (M) = V C R C (M) + V 0 R 0 (M) + V 1 R 1 (M) + V 2 R 2 (M) R cycle (M) = V C R C (M) + V 0 R 0 (M) + V 1 R 1 (M) + V 2 R 2 (M) R sys (M) = V 0 R 0 (M) + V 1 R 1 (M) + V 2 R 2 (M) R sys (M) = V 0 R 0 (M) + V 1 R 1 (M) + V 2 R 2 (M) Now, as explained in class, we apply little’s law to the whole system and get system throughput Now, as explained in class, we apply little’s law to the whole system and get system throughput T sys (M) = M / R cycle (M) T sys (M) = M / R cycle (M)

MVA…contd Now we have system throughput at load level T sys (M). We can get node throughput by multiplying by visit counts: Now we have system throughput at load level T sys (M). We can get node throughput by multiplying by visit counts: T i (M) = V i T sys (M) T i (M) = V i T sys (M) Now thet we have node throughput, and recalling that we first found response time at node i, we can apply Little’s Law at node level: Now thet we have node throughput, and recalling that we first found response time at node i, we can apply Little’s Law at node level: N i (M) = T i (M) * R i (M) N i (M) = T i (M) * R i (M) Recursion is complete! Recursion is complete!

MVA…contd So if we have to find, say, system response time at load level 3, we start with So if we have to find, say, system response time at load level 3, we start with N i (0) = 0 and proceed with finding metrics at load levels 1, 2, 3 iteratively. N i (0) = 0 and proceed with finding metrics at load levels 1, 2, 3 iteratively.

Example in Spreadsheet Example in spreadsheet represents a server system which processes two types of requests: “orders” and complaints (“trouble tickets”) Example in spreadsheet represents a server system which processes two types of requests: “orders” and complaints (“trouble tickets”) If modeled as closed QN, model a node of M clients who are interacting with this system If modeled as closed QN, model a node of M clients who are interacting with this system Client issues either request for “order” or “trouble ticket” (w. p. pO, pT respectively) Client issues either request for “order” or “trouble ticket” (w. p. pO, pT respectively) These go to different web servers (W1, W2) These go to different web servers (W1, W2) Request may have to visit application servers (A1, A2). Multiple such loops are possible. Request may have to visit application servers (A1, A2). Multiple such loops are possible. Picture on next page Picture on next page

Example in Spreadsheet

Example in spreadsheet: activity for you Calculate visit count based on the topology shown Calculate visit count based on the topology shown Express each visit count in terms of Vc and set Vc to 1 Express each visit count in terms of Vc and set Vc to 1 In the spreadsheet, all cells coloured yellow are input parameters. Everything else is calculated. In the spreadsheet, all cells coloured yellow are input parameters. Everything else is calculated. Check that the visit counts as coded in the spreadsheet match your calculations. Check that the visit counts as coded in the spreadsheet match your calculations. The MVA iteration is coded in the rows. Row i values are calculated using the N i of Row i-1. The MVA iteration is coded in the rows. Row i values are calculated using the N i of Row i-1.