Queueing Models with Multiple Classes CSCI 8710 Tuesday, November 28th Kraemer

The Need for Multiple-Class Models As you may recall, motivations for constructing multiple-class models for heterogeneous workloads include:  To represent different QoS or SLA workload classes  To (more) accurately model requirements of the workload Example: server  Heavy user  Light user Averaging together to create “medium” user is a less accurate model for predictive purposes

Multiclass modeling Choosing “right” number of classes is difficult  Too few -> inaccurate  Too many -> too complex Downside of multiclass modeling:  Difficult to obtain parameters for each class

Example: Proposed SLA Agreement Proposal  Risk Portfolio analysis transactions Average RT: 3 hours $15 per transaction  Purchase transactions Average RT < 1 sec. $0.50 per transaction  Browsing transactions Average RT < 2 sec $0.10 per transaction

Modeling the Proposed System Class 1: Risk portfolio analysis  Closed, defined by service demands and number of processes in execution during the “peak hour” Class 2: Online purchase transactions  Open, defined by service demands and average arrival rate during a “peak minute” Class 3: Browsing transactions  Open, defined by service demands and average arrival rate during a “peak minute”

Simple Two-Class Model

During peak hours, system is under heavy load  Four transactions are in execution almost all the time  More than 4 -> thrashing, so keep 5th & higher in system entry queue (blocking)  Observed mix: 3Q, 1U

Simple Two-Class Model Assume:  Service time per disk visit is same for Q&U  U has more visits per transaction

Note: scheduling matters for multiclass Choice of scheduling discipline affects performance modeling for multiclass  First-come, first served (FCFS)  Round-Robin (RR)  Shortest Job Next (SJN)  Shortest Remaining Time (SRT)  Earliest Deadline (ED)  Least Laxity (LL)

Simple Two-Class Model One approach: construct an equivalent single-class model  (We did this before the break)  Useful, in that the single-class approach “pessimistically bounds the performance of the multiclass model”[Dowdy]  Problematic, in that it doesn’t permit the solution of many “what-if” scenarios

Assumptions: BCMP Specify a combination of service-time distributions and scheduling disciplines that yield multiclass product-form queueing networks that “lend themselves to efficient model solution techniques”  Open, closed, or mixed networks are allowed

Assumptions: BCMP Service centers with FCFS discipline  Service time distribution must be exponential with same mean for all classes May have different visit ratios  Service rate can be load_dependent But can only depend on total number of customers at the server, not on the number of customers of any particular class

Assumptions: BCMP Service centers with PS discipline  n customers, each receives service at rate 1/n  Each class may have distinct service time dist.  Reasonable approximation for RR

Assumptions: BCMP Service centers with infinite servers (IS)  Never any waiting for a server  a.k.a. - delay server or no queueing

Assumptions: BCMP Service centers with LCFS-PR discipline  Each class may have distinct service time dist.  Can be used to model servers at which high- priority interrupts require immediate but small amounts of service

Assumptions: BCMP Open networks:  Exponentially distributed inter-arrival time  No bulk arrivals

Notation K: number of devices (service centers) R: number of classes of customers M r : number of terminals of class r Z r : think time of class r N r : class r population r : arrival rate of class r S i,r : avg. service time of class r customers at device i V i,r : avg. visit ratio of class r customers at device i D i,r : avg. service demand of class r customers at device i; D i,r = S i,r * V i,r

Notation, continued R i,r : avg. response time per visit of class r customers at device i R’ i,r : avg. residence time of class r customers at device i; R’ i,r = V i,r * R i,r X i,r : class r throughput at device i X 0,r : class r system throughput R r : class r response time

Closed Models typically used for batch jobs and interactive jobs = (1,3) for the update/query example of earlier

Multiple class closed model

Multiclass MVA relies on 3 basic equations applied to each class First: apply Little’s Law separately to each class of customers

Multiclass MVA Second:  Apply Little’s Law and the Forced Flow Law to each service center:

Mulitclass MVA The sum up customers of all classes at device i to get the total number of customers at that device:

Multiclass MVA Note: mean response time of a class r customer at service center i is own mean service time at that device plus time to service mean backlog seen upon its arrival. Therefore:

“backlog” backlog = average queue length at device i seen by arriving class r customer for delay server => 0 for PS or LCFS-PR => an “inflation factor”

Closed Models: Exact Solution Algorithm Observation: queue length is 0 when no customers are in the network Also:

Exact MVA for Multiclass:

Closed Models: Case Study

1.What is the predicted increase in the throughput of query transactions if the load of the update class is moved to off- peak hours? Assume we still have 4 transactions – now all query transactions. Solve single-class model with 4 queries, get tput=5.275 tps … which is an increase of 28.87%

Closed Models: Case Study 2.Realizing that disk 1 is the bottleneck and disk 2 is lightly loaded, which is the predicted RT if the total I/O load of query transactions is moved to disk 2? shift value of D 2,q to D 3,q, solve new model. Results:  X 0,q = tps  X 0,u = tps  R q = sec  R u = sec

Open Models number of customers (transactions, processes, requests) varies dynamically

Multiclass Open Models

Analysis of Multiclass Open Models In steady state, the throughput of class r equals its arrival rate:  X 0,r = r ( ) The application of Little’s Law to each device gives Eq :

Analysis of Multiclass Open Models The avg. residence time for the entire execution is R’ i,r =V i,r R i,r Using the Forced Flow Law and Eq. ( ) from the previous slide, the throughput of class r is given by Eq

Analysis of Multiclass Open Models Using Eq and , the average queue length per device becomes (Eq ):

Analysis of Multiclass Open Models Combining the Utilization Law and the Forced Flow Law, the utilization of device i by class r customers can be written as (Eq ):

Analysis of Multiclass Open Models To computer average number of class r customers in service center i, need R’ i,r as function of the input parameters.

Analysis of Multiclass Open Models In an open system, the population is infinite, so the arriving customer sees the overall steady-state distribution. Thus,

Analysis of Multiclass Open Models

Notice that the expression in the [] on the previous slide doesn’t depend on class r. Thus, for any two classes r and s, we have:

Analysis of Multiclass Open Models Taking as class s all classes combined, we can rewrite n i,r as:

Analysis of Multiclass Open Models Applying Little’s Law to the previous formula, we obtain:

Summary

Open Model: Case Study distributed environment diskless clients connected to file server via high-speed LAN file server = 1 CPU, 1 disk(large) Question: what is the predicted performance of the file server if the number of diskless clients doubles?

Open Model: Case Study 1.Workload characterization, monitor for 1 hour: read requests write requests other requests – 3600 CPU utilization: 32% (9% R, 18%W, 5%O) disk utilization: 48% (20%R, 20%W, 8%O)

Open Model: Case Study From per-class utilizations and throughputs, can calculate D i,r, and then V disk,r, based on disk service times quoted by manufacturer. Can calculate V proc,r as 1 initial CPU visit plus one more CPU visit for every I/O visit

Case Study – Open/multi

Alternative Configurations