1. Solutions Markov Chains 1
1) Given the following one-step transition matrices of a Markov chain, determine the classes of the Markov chain and whether they are recurrent. a. b. 1 2 3
All states communicate, all states recurrent 1 2 3
Communicate {0}, {1, 2}, {3}
states 0, 1, 2 recurrent
state 3 transient

2. Solutions Markov Chains 3
2) The leading brewery on the West Coast (A) has hired a TM specialist to analyze its market position. It is particularly concerned about its major competitor (B). The analyst believes that brand switching can be modeled as a Markov chain using 3 states, with states A and B representing customers drinking beer produced from the aforementioned breweries and state C representing all other brands. Data are taken monthly, and the analyst has constructed the following one-step transition probability matrix.
A B C
What are the steady-state market shares for the two major breweries?
Soln:
(1)
(2)
(3)
Let,
Then, from (1),
(4)

3. Solutions Markov Chains 4
2) (cont.)
(1)
(2)
(3)
(4) p C B = - 3 2
from (2),
from (4)

4. Solutions Markov Chains 5
2) (cont.)
But,
Note: This could also be checked by Pn for some large n.

5. Solutions Markov Chains 6
Louise Ciccone, a dealer in luxury cars, faces the following weekly demand distribution.
Demand 0 1 2
Prob
She adopts the policy of placing an order for 3 cars whenever the inventory level drops to 2 or fewer cars at the end of a week. Assume that the order is placed just after taking inventory. If a customer arrives and there is no car available, the sale is lost. Show the transition matrix for the Markov chain that describes the inventory level at the end of each week if the order takes one week to arrive. Compute steady state probabilities and the expected number of lost sales per week.
Soln
S= inventory level at week’s end = 1, 2, 3, 4, 5
note: we can have at most 5 cars in the lot, 2 cars at week’s end followed by no sales followed by the arrival of 3 cars at the end of the week.
note: we could start with 0 cars, but once an arrival of 3 cars comes we will never get back to 0 cars since we never sell more than 2 and we get below 2 at time t, 3 arrive at t+1

6. Solutions Markov Chains 7
Soln cont.
State Event End State Probability
5 sell
5 sell
5 sell
4 sell
4 sell
4 sell
3 sell
3 sell
3 sell
2 3 arrive at week’s end
sell
sell
sell
1 3 arrive at week’s end
sell
sell since demand of 1 or 2 deletes
inventory during the week

7. Solutions Markov Chains 8
Soln cont.
Let,
Continue substitutions and renormalize or