Performance improvement through Active Idleness José A.A. Moreira Agilent Technologies Germany Carlos F.G. Bispo Instituto de Sistemas e Robótica Instituto Superior Técnico Portugal

MED Rhodes, Greece 2/23 Outline Motivation Active Idleness The Time Window Controller Implications on Performance Conclusions

MED Rhodes, Greece 3/23 Motivation: Queuing network Multiple servers Multiple types of customers Different routs for each type through the network Each type of customer may visit same server more than once A type of customer is composed of several classes One per operation needed Each server processes different classes with different processing time distributions – no redundant servers Each type of customer arrives to the system randomly in time but to one unique point and exits the system at a specific server Customers queue up in front of the server they require next

MED Rhodes, Greece 4/23 Motivation: The concept of load Assume there are K different classes of customers Some may belong to the same type of customer Define as k the first moment of the arrival rate distribution of class k, for k = 1, 2, …, K, to its server Define as  k the first moment of the service time distribution of class k, for k = 1, 2, …, K Assume there are I different servers Let c(i) designate the constituency of server i, for i = 1, 2,..., I Class k belongs to c(i) if server i processes it Then, the load of server i is  k  c(i) k  k

MED Rhodes, Greece 5/23 Motivation: Admission and scheduling A policy to control the entry of new customers in the system is said to be an admission policy A policy to decide which customer a server has to process next is said to be a scheduling policy An open network does not have an admission policy Otherwise the system is termed as a closed network All networks must have a scheduling policy If they exist, both policies affect the arrival rates of classes to their servers In an open network only the internal arrival rates are affected We address open networks with distributed policies (why?)

MED Rhodes, Greece 6/23 Motivation: Stability There are many alternative ways of defining stability for queuing networks For our purposes, suffices to say that a network is stable if the expected queue lenght of all servers is finite or if the expected time to flow through the network is finite for all types of customers Connected to the above, one can also say that a network is stable if all the internal arrival rates match the external arrival rates on a short term basis This is not a precise concept, serves only to provide intuition

MED Rhodes, Greece 7/23 Motivation: Traffic Intensity Condition The necessary condition for stability states that if a network is stable then the load imposed on each server by the classes it serves is below unity Is the TIC also sufficient? For many years it was thought so, as long as the scheduling policy does not keep servers idle in the presence of customers – non-idling policies Also termed by some authors as work-conserving policies In the late 80’s and early 90’s some examples were published where, although the TIC holds, the networks are unstable when controlled by non-idling policies

MED Rhodes, Greece 8/23 Motivation: Dai’s network Parameters 1 = 1  1 =  2 =  3 =  4 =  5 =  6 = Loads Server 1: 0.9 Server 2: 0.9 Scheduling First Come First Serve Class 1Class 2 Class 3 Class 4 Class 6Class 5

MED Rhodes, Greece 9/23 Motivation: Simulation results

MED Rhodes, Greece 10/23 Active Idleness: What conditions stability? The network alone? The network plus the scheduling policy? What is the network alone? Just the servers? Servers and routs of customers? The above plus the arrival rates and processing times? What should condition stability? Are we interested in determining if a pair topology/scheduling policy is stable? Or are interested in determining if a given topology is stabilizable?

MED Rhodes, Greece 11/23 Active Idleness: Control Theory From a control theory perspective Is a system stable when left alone? If a system is unstable, can we stabilize it through the adequate choice of control? Is a network stable when left alone? NEVER – needs control policy (admission and/or scheduling) Do we want to know if a network can be stabilized through the adequate choice of scheduling policy? Does that entail finding the right policy? Or does it entail someting else?

MED Rhodes, Greece 12/23 Active Idleness: Wondering From the examples published It is odd that always working in the presence of customers does not ensure stability whenever the TIC holds – for distributed policies There are starvation periods being created, which translate into short term loss of capacity, that will not be recovered Should we blame it on the policy alone? Or is there something else? I wonder... What if we drop the requirement of always using non-idling policies? What if we allow servers to stop in the presence of customers? Is there a long term capacity gain due to controlled short term losses of capacity?

MED Rhodes, Greece 13/23 Active Idleness: Wondering still Surely you are kidding... The network is unstable and you want to stabilize it by wasting capacity!!!... Find a job somewhere else... Well... We are wasting capacity for the fact that we do not filter the burstiness of the arrival processes Some servers get lots of customers of a given class, work on them and the next server will get a handfull of new customers, but some other server is not getting anything, thus the waste

MED Rhodes, Greece 14/23 Active Idleness: Key to stability? Active Idleness is perhaps one key Don’t be passive about staying idle Chose your moments of idleness and perhaps you will not have to be idle for such long periods Filter burstiness Block classes that are flooding the server, allowing for some others to get through If a class gets blocked and there are others, work on them If there are no other classes, what is the rush of working on a class that you have been working for a significant amount of time?

MED Rhodes, Greece 15/23 The Time Window Controller It is one possible implementation of the AI concept Time window of size T (finite) Processing History - Look T units into the past and compute the fraction of time each class used its server Maximum time fraction – assign a maximum value to each class Blocking – occurs when a class has exceeded its maximum time fraction Server only “sees” classes which are not blocked, the blocked ones are assumed not be present while they remain blocked Not a new scheduling policy decisions are still made by the same criteria as before, but only on the visible classes As time progresses, classes leave the blocking status and become elegible to be processed according to the policy

MED Rhodes, Greece 16/23 Implications on Performance Simulation results - stability

MED Rhodes, Greece 17/23 Implications on Performance So we manage to stabilize unstable networks What if the pair network/scheduling policy is already stable? Is there any advantage on using the Time Window Controller? Take the same network as before and use Last Buffer First Serve We know this pair to be stable when the TIC holds Use some metric to evaluate performance Higher cost to earlier stages Average queue lenght

MED Rhodes, Greece 18/23 Implications on Performance Simulation results – improvement over stable Higher cost to earlier stages

MED Rhodes, Greece 19/23 Implications on Performance Active Idleness as a function of fraction Higher cost to earlier stages

MED Rhodes, Greece 20/23 Implications on Performance Simulation results – improvement over stable Average bufer lenght

MED Rhodes, Greece 21/23 Conclusions: What was done Proposed to use Active Idleness as a way to ensure stabilizability Described a controller implementing the concept Shown the dramatic improvement for an unstable system Shown the potential to optimize performance, irrespective of the fact that original system is or is not stable

MED Rhodes, Greece 22/23 Conclusions: What remains to do Would like/need to develop an efficient way to tune the controller Brute force parameter tunning Infinitesimal Perturbation Analysis Other... What if buffers are finite? What if there is a set-up time associated with changing classes? How about spliting and merging? What is the best distributed scheduling policy with AI? In general, is it the case that the best policy is non-idling?

MED Rhodes, Greece 23/23 Performance Improvement through Active Idleness José A.A. Moreira Carlos F.G. Bispo