1. Lecture 4: Algorithmic Methods for G/M/1 and M/G/1 type models
Dr. Ahmad Al Hanbali
Department of Industrial Engineering
University of Twente

2. Lecture 4
This Lecture deals with continuous time Markov chains with infinite state space as opposed to finite space with skip-free in one direction as opposed to QBDs Lecture 3
Objective: To find equilibrium distribution of the Markov chain
Lecture 4: G/M/1 and M/G/1 type models

3. Background (1): G/M/1 queue
Interarrival time of jobs is arbitrary distribution, 𝐹 𝐴 (𝑡) with mean 1/𝜆, and inter-arrival times are iid
Jobs service times are iid exponential rvs with rate 𝜇. Inter- arrivals and service times are independent
Service discipline can be Fisrt-In-First-Out (FIFO)
Under above assumptions, the nbr of jobs in G/M/1 queue at arbitrary time 𝑡 DOES NOT form a Markov chain
Let 𝑁( 𝑡 𝑖 ) denote the number of jobs in the queue just before the 𝑖-th arrival. The process {𝑁( 𝑡 𝑖 ), 𝑖=0,1,…} is a discrete- time Markov chain
Let 𝑎 𝑛 denote the probability that exactly 𝑛 jobs are served during an inter-arrival time given there are at least 𝑛 jobs present at the start of inter-arrival. Then 𝑎 𝑛 reads
𝑎 𝑛 = 0 ∞ 𝜇𝑡 𝑛 𝑛! 𝑒 −𝜇𝑡 𝑑 𝐹 𝐴 (𝑡) , 𝑛=0,1,…, and 𝑏 𝑖 = 𝑘>𝑖 𝑎 𝑘
The last integration is due to exponential distribution of service times.
- b_i is the proba. that more than i jobs are served during an inter-arrival time
Lecture 4: G/M/1 and M/G/1 type models

4. Background (2): G/M/1 queuei
i+1
i-1 1
𝑎 0
𝑎 1
𝑎 2
𝑎 𝑖
𝑏 𝑖
The transition probability matrix is given
𝑃= 𝑏 0 𝑏 1 𝑏 2 𝑏 3 𝑏 4 ⋮ 𝑎 0 𝑎 1 𝑎 2 𝑎 3 𝑎 4 ⋮ 0 𝑎 0 𝑎 1 𝑎 2 𝑎 3 ⋮ 𝑎 0 𝑎 1 𝑎 2 ⋮ 𝑎 0 𝑎 1 ⋮ … … … … ⋱ ⋱ .
For stable case with 𝜆<𝜇 we find
𝑝 𝑖 = 1−𝜎 𝜎 𝑖 , (Geometric distribution)
where 𝜎 is the unique root in (0,1) of 𝜎=𝐸 𝑒 −𝜇 1−𝜎 𝐴 .
𝑝 𝑖 is the probability that on moment of arrivals there are i jobs in the system.
Note the p_i is the geometric distribution.
Note 𝜋 𝑖 the equilibrium probability at arbitrary point can be found using a crossing level argument. 𝜆 𝑝 𝑖−1 =𝜇 𝜋 𝑖 (See Tijms book section 2.7 note that there p and 𝜋 are interchanged)
Lecture 4: G/M/1 and M/G/1 type models

5. Definition G/M/1-type processes: skip-free process to the right
A 2-dimensional irreducible continuous time Markov process with states (𝑖,𝑗), where 𝑖=0,…,∞ and 𝑗= 0,…,𝑚−1
Subset of state space with common 𝑖 entry is called level 𝑖 (𝑖>0) and denoted 𝑙(𝑖)={(𝑖,0),(𝑖,1),…,(𝑖,𝑚−1)}. 𝑙(0)={(0,0),(0,1),…,(𝑖, 𝑚 0 −1)}. This means state space is ∪ 𝑖≥0 𝑙(𝑖)
Transition rate from (𝑖,𝑗) to (𝑖′,𝑗′) is equal to zero for 𝑖 ′ −𝑖≥ 2
For 𝑖>0, transition rate between states in 𝑙(𝑖) and from 𝑙(𝑖) to 𝑙 𝑖+1 , 𝑙 𝑖−1 , …,𝑙(0) are independent of 𝑖
Lecture 4: G/M/1 and M/G/1 type models

6. Skip-free to the right process
Order the states lexicographically, i.e., 0,0 ,…, 0, 𝑚 0 , 1,0 ,…, 1,𝑚 , 2,0 ,…, 2,𝑚 ,…, the generator of the skip-free process has the following form:
𝑄= 𝐵 00 𝐵 10 𝐵 20 𝐵 30 ⋮ 𝐵 01 𝐵 11 𝐴 2 𝐴 3 ⋱ 0 𝐴 0 𝐴 1 𝐴 2 ⋱ 𝐴 0 𝐴 1 ⋱ 𝐴 0 ⋱ … ⋱ ⋱ ⋱ ⋱
where 𝐴0 and 𝐴2 are nonnegative 𝑚-by-𝑚 matrices; 𝐴𝑖 aisre square matrices of size 𝑚 0 +1; 𝐵 00 is square matrix of size 𝑛+1; 𝐵 𝑖0 𝑚 0 +1-by-𝑛+1 and 𝐵 01 𝑛+1-by- 𝑚 nonnegative matrices. Note, ( 𝐵 00 + 𝐵 01 )𝑒=0, ( 𝐵 𝐵 11 + 𝐴 0 )𝑒=0, and ( 𝐵 0𝑖 + 𝑙=0 𝑖 𝐴 𝑙 )𝑒=0
Lecture 4: G/M/1 and M/G/1 type models

7. Stability of G/M/1-type process
Let 𝐴= 𝑖=0 ∞ 𝐴 𝑖 . 𝐴 is the generator describing transitions of the M/G/1-type process between level states (i.e., in the vertical direction)
Theorem: Assume the Markov chain with generator 𝐴 is irreducible with equilibrium distribution, 𝜋𝐴=0, 𝜋𝑒=1. The G/M/1-type process is ergodic if and only if
𝜋𝐴0𝑒<𝜋 𝑖=1 ∞ 𝑖−1 𝐴 𝑖 𝑒 (mean drift condition)
Interpretation: 𝜋𝐴0𝑒 is mean drift from level 𝑖 to 𝑖+1. 𝜋 𝑖=1 ∞ 𝑖−1 𝐴 𝑖 𝑖=1 ∞ 𝑖−1 𝐴 𝑖 𝑒 is the mean drift from level 𝑖 to levels smaller than 𝑖 for large 𝑖
It is assumed here the A is a generator of an irreducible Markov chain. For less restrictive assumption on A see book of Latouche & Ramaswami Section 7.3
Lecture 4: G/M/1 and M/G/1 type models

8. Equilibrium distribution of G/M/1-type processes
Let 𝑝𝑛=(𝑝(𝑛,0),..,𝑝(𝑛,𝑚−1)) and 𝑝=(𝑝0,𝑝1,…) then equilibrium equation 𝑝𝑄=0 reads
𝑖=0 ∞ 𝑝 𝑖 𝐵 𝑖0 = 0, 𝑝0 𝐵 01 +𝑝1 𝐵 11 + 𝑖=2 ∞ 𝑝 𝑖 𝐴 𝑖 = 0,
𝑖=0 ∞ 𝑝 𝑛+𝑖 𝐴 𝑖 = 0, 𝑛≥1
Theorem: if the G/M/1-type process is ergodic the equilibrium probability distribution then reads
𝑝 𝑛 =𝑝1 𝑅 𝑛−1 , 𝑛≥1,
where 𝑅 is the minimal nonnegative solution of the matrix equation 𝑖=0 ∞ 𝑅 𝑖 𝐴 𝑖 =0
Interpretation: 𝑅=𝐴0𝑁 same as in QBD process
Similarly to QBD processes 𝑅=𝐴0𝑁, where matrix 𝑁 records the expected sojourn time in the states of 𝑙(𝑖), i>2, given the process starts in one of 𝑙(𝑖) states at time zero, before the first visit to 𝑙(𝑖−1)
Lecture 4: G/M/1 and M/G/1 type models

9. Equilibrium distribution of G/M/1-type processes(cnt'd)
A direct result of the previous theorem is that spectral radius of 𝑅 is < 1 and (𝐼−𝑅) is nonsingular
Lemma: The stationary probability vectors 𝑝0 and 𝑝1
is the normalized unique solution of 𝑥 0 , 𝑥 1 𝐵 𝑅 = 0,0 , where 𝐵 𝑅 is the generator given by
𝐵 𝑅 = 𝐵 00 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 𝐵 11 + 𝑖=1 ∞ 𝑅 𝑖−1 𝐴 𝑖 .
Normalization is done by letting 𝑥 0 𝑒 𝑚 0 +𝑥1 𝐼−𝑅 −1 𝑒 𝑚 =1. Note 𝑒 𝑚 0 and 𝑒 𝑚 are column vectors of ones with size 𝑚 0 and 𝑚, respectively
Proof: follows by inserting 𝑝 𝑛 =𝑝1 𝑅 𝑛−1 in the balance equations
Note that B_00 is invertible because eventually the process will leave level 0. Therefore 𝑝0=− 𝑝 1 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 . Inserting in the second equation we get that:
𝑝 1 − 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 𝐵 01 + 𝐵 11 + 𝑖=2 ∞ 𝑅 𝑖−1 𝐴 𝑖 =0. The normalization condition can be rewritten as 𝑝 1 (− 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 𝑒 𝑛 + 𝐼−𝑅 −1 𝑒 𝑚 )=1. Let matrix X be − 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 𝐵 01 + 𝐵 11 + 𝑖=2 ∞ 𝑅 𝑖−1 𝐴 𝑖 − 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 𝐵 01 + 𝐵 11 + 𝑖=2 ∞ 𝑅 𝑖−1 𝐴 𝑖 but with column i − 𝑖=1 ∞ 𝑅 𝑖−1 𝐵 𝑖0 𝐵 00 −1 𝑒 𝑛 + 𝐼−𝑅 −1 𝑒 𝑚 . Then we get that 𝑝 1 = 𝑒 𝑖 𝑋 −1 , 𝑒 𝑖 a row vector with i-th entry 1 and rest zero.
Lecture 4: G/M/1 and M/G/1 type models

10. Special case: GI/PH/1 queue
Consider the case 𝐵 01 = 𝐴 0 and 𝐵 11 = 𝐴 1
The matrix 𝐵 𝑖0 is a rank one matrix satisfying
𝐵 𝑖0 =− 𝑗=0 𝑖 𝐴 𝑗 𝑒𝛽 .
where 𝛽 is the initial state probability vector of the service time phase-type distribution
Lemma: The stationary probability vector of GI/PH/1 embedded at the moment of arrivals
𝑝 𝑖 = 𝑝 0 𝑅 𝑘 ,
where 𝑝 0 = 𝛽 𝐼−𝑅 −1 𝑒 −1 𝛽.
Proof: in this case 𝐵 𝑅 will be of rank one
Lecture 4: G/M/1 and M/G/1 type models

11. 𝑅 𝑘+1 =−( 𝐴 0 + 𝑖=2 ∞ 𝑅 𝑘 𝑖 𝐴 𝑖 ) 𝐴 1 −1 , with 𝑅0=0
Finding R
Rearrange the equation of the rate matrix:
𝑅=− 𝐴0+ 𝑖=2 ∞ 𝑅 𝑖 𝐴 𝑖 𝐴 1 −1
Fixed point equation solved by successive substitution
𝑅 𝑘+1 =−( 𝐴 0 + 𝑖=2 ∞ 𝑅 𝑘 𝑖 𝐴 𝑖 ) 𝐴 1 −1 , with 𝑅0=0
It can be shown that 𝑅𝑘→ 𝑅 for 𝑘→∞
In many queueing systems for large 𝐾, 𝐴 𝐾 ≈0. Truncating 𝑖=2 ∞ 𝑅 𝑘 𝑖 𝐴 𝑖 ≈ 𝑖=2 𝐾 𝑅 𝑘 𝑖 𝐴 𝑖
A1 is invertible since it is a generator of a transient Markov chain states 𝑙(𝑛),
Lecture 4: G/M/1 and M/G/1 type models

12. Definition of M/G/1-type processes: skip-free process to the left
A 2-dimensional irreducible continuous time Markov process with states (𝑖,𝑗), where 𝑖=0,…,∞ and 𝑗= 0,…,𝑚−1
Subset of state space with common 𝑖 entry is called level 𝑖 (𝑖>0) and denoted 𝑙(𝑖)={(𝑖,0),(𝑖,1),…,(𝑖,𝑚−1)}. 𝑙(0)={(0,0),(0,1),…,(𝑖, 𝑚 0 −1)}. This means state space is ∪ 𝑖≥0 𝑙(𝑖)
Transition rate from (𝑖,𝑗) to (𝑖′,𝑗′) is equal to zero for 𝑖 ′ −𝑖≤− 2.
For 𝑖>0, transition rate between states in 𝑙(𝑖) and from 𝑙(𝑖) to 𝑙 𝑖−1 , 𝑙 𝑖+1 , …, are independent of 𝑖
Lecture 4: G/M/1 and M/G/1 type models

13. Skip-free to the left process
Order the states lexicographically, i.e., 0,0 ,…, 0, 𝑚 0 , 1,0 ,…, 1,𝑚 , 2,0 ,…, 2,𝑚 ,…, the generator of the QBD has the following form:
𝑄= 𝐵 00 𝐵 ⋮ 𝐵 01 𝐴 1 𝐴 0 0 ⋱ 𝐵 02 𝐴 2 𝐴 1 𝐴 0 ⋱ 𝐵 03 𝐴 3 𝐴 2 𝐴 1 ⋱ … … … … ⋱
𝐴𝑖 is square matrix of size 𝑚+1; 𝐵 00 is square matrix of size 𝑚 0 +1; 𝐵 0𝑖 𝑛+1-by- 𝑚 0 +1 and 𝐵 10 𝑚 0 +1-by-𝑛+1 nonnegative matrices. Note, the sum of the rows elements should be zero.
Lecture 4: G/M/1 and M/G/1 type models

14. Stability of M/G/1-type processes
Let 𝐴= 𝑖=0 ∞ 𝐴 𝑖 . 𝐴 is the generator describing transitions of the M/G/1-type process in the horizontal direction.
Theorem: Assume the Markov chain with generator 𝐴 is irreducible with equilibrium distribution, 𝑖.𝑒. 𝜋𝐴=0, 𝜋𝑒=1, and with 𝑖=1 ∞ 𝑖 𝐵 0𝑖 𝑒. The M/G/1-type process finite is ergodic if and only if
𝜋𝐴0𝑒>𝜋 𝑖=2 ∞ 𝑖−1 𝐴 𝑖 𝑒
See Theorem of fundamentals of matrix analytical methods Qi-Ming He or Neuts 1980
Lecture 4: G/M/1 and M/G/1 type models

15. Equilibrium distribution of skip-free to the left processes
Assume M/G/1-type process is ergodic. The minimal nonnegative solution 𝐺 of
𝑖=0 ∞ 𝐴 𝑖 𝐺 𝑖 =0,
is then stochastic, i.e., 𝐺𝑒=𝑒,
Interpretation: 𝑘,𝑙 -element of 𝐺 represents probability to jump for the first time to level 𝑖−1 by entering state 𝑗, 𝑖 ≥1, given process starts in (𝑖,𝑘) at time 0
Theorem (Matrix analytic): assume that M/G/1-type process is ergodic then the steady states probabilities give
𝑝 𝑖 =− 𝑝 0 𝐵 𝑖 + 𝑘=1 𝑖−1 𝑝 𝑘 𝐴 𝑖+1−𝑘 𝐴 1 −1 , 𝑖=2,3,…,
where, 𝐵 𝑖 = ∑ 𝑘=0 ∞ 𝐵 0𝑖+𝑘 𝐺 𝑘 , 𝐴 𝑖 = ∑ 𝑘=0 ∞ 𝐴 𝑖+𝑘 𝐺 𝑘 , 𝑛=1,2,….
- It is assumed here the A is a generator of an irreducible Markov chain. For less restrictive assumption on A see book of Neuts 1989, Structured Markov Chains of M/G/1-type of Latouche & Ramaswami Section 7.3.
- Note 𝐴 1 is non-singular because 𝐴 1 e=− A 0 e≤0 and different from zero vector.
Lecture 4: G/M/1 and M/G/1 type models

16. Finding 𝐺
In many queueing systems for large 𝐾, 𝐴 𝐾 ≈0. Therefore, it is reasonable to truncate the infinite sum of 𝐺 𝑙+1 at K
Using this fact, 𝐴 𝐾 and 𝐵 𝐾 →0, the following recursion (backward) can used
𝐵 𝑖 = 𝐵 0𝑖 + 𝐵 𝑖+1 𝐺
𝐴 𝑖 = 𝐴 𝑖 + 𝐴 𝑖+1 𝐺
The matrix 𝐺 can be found recursively as follows
𝐺 𝑙+1 =− 𝐴 1 −1 𝐴 0 + 𝑖=2 ∞ 𝐴 𝑖 𝐺 𝑙 𝑖 , 𝑙=1,2,…, with 𝐺 0 =0.
It can be shown that 𝐺𝑘→𝐺 for 𝑘→∞
Lecture 4: G/M/1 and M/G/1 type models

17. 𝐵 𝐺 = 𝐵 00 𝐵 10 ∑ 𝑘=0 ∞ 𝐵 01+𝑘 𝐺 𝑘 ∑ 𝑘=0 ∞ 𝐴 1+𝑘 𝐺 𝑘 .
Probabilities 𝑝 0 and 𝑝 1
Theorem: The stationary probability vectors 𝑝0 and 𝑝1
is the normalized unique solution of 𝑥 0 , 𝑥 1 𝐵 𝑅 = 0,0 , where 𝐵 𝐺 is the generator of an irreducible chain given by
𝐵 𝐺 = 𝐵 00 𝐵 ∑ 𝑘=0 ∞ 𝐵 01+𝑘 𝐺 𝑘 ∑ 𝑘=0 ∞ 𝐴 1+𝑘 𝐺 𝑘 .
Normalization should be done such 𝑖=0 ∞ 𝑝 𝑖 𝑒=1
Lecture 4: G/M/1 and M/G/1 type models

18. Finding 𝑝 0 : special case
Let 𝐵= 𝑖=0 ∞ 𝐵 0𝑖 and 𝑏= 𝑖=1 ∞ 𝑖 𝐵 0𝑖 𝑒
Let 𝑎= 𝑖=1 ∞ 𝑖 𝐴 𝑖 𝑒
Assume 𝐴= 𝑖=0 ∞ 𝐴 𝑖 is irreducible and let 𝜋𝐴=0, 𝜋𝑒=1.
Let 𝜇=𝜋𝑎
Theorem: Assume the generators of skip-free to the left process and 𝐴 are irreducible. If the skip-free to the left process is ergodic, then
𝜇= 𝑝 0 𝜇𝑒−𝑏+𝐵 𝐴+𝑒𝜋 −1 (𝜇𝑒−𝑎𝑒)
Proof: On board
For proof see book of book of Bini, Latouche, Meini, “Numerical Methods for Structured Markov Chains” Theorem 4.8 for Discrete Time M/G/1-type processes.
𝑝 1 can be found using the recursion on slide 16 which will holds in the case for 𝑖=1,2,…
Lecture 4: G/M/1 and M/G/1 type models

19. References
R. Nelson. Matrix geometric solutions in Markov models: a mathematical tutorial. IBM Technical Report 1991.
G. Latouche and V. Ramaswami (1999), Introduction to Matrix Analytic Methods in Stochastic Modeling. SIAM.
I. Mitrani and D. Mitra, A spectral expansion method for random walks on semi-infinite strips, in: R. Beauwens and P. de Groen (eds.), Iterative methods in linear algebra. North-Holland, Amsterdam (1992), 141–149.
M.F. Neuts (1981), Matrix-geometric solutions in stochastic models. The John Hopkins University Press, Baltimore.
Lecture 4: G/M/1 and M/G/1 type models